Back to Our Insights
Forbes First published on Forbes.com on October 2, 2020
October 2, 2020Investment Management

The Perils of Confusing Predictive Models With Explanatory Models

Models are simplified depictions of reality. Source: Getty Images

Models are important for understanding the current and future states of the world and we use many, for instance the capital asset pricing model, to help us understand markets and investing.

But most models only either explain or predict: they don’t do both. In the investment industry, people commonly confuse an explanatory model with a predictive one. This can be an expensive mistake.

A model is a simplified representation of some aspect of reality, such as a diagram, a mathematical formula, a story or an analogy. Because they are simplifications of reality, models, cannot be 100% correct. The classic example of this is a map. For it to be 100% correct it would have to be the same size as the the territory itself. So as George Box once put it, “all models are wrong, but some models are useful.”

Predictive Models Are Not Easy to Build

In 2006, when they were in the business of mailing DVDs, Netflix announced a $1 million prize to whomever could improve their algorithms that recommended movies to users. Participants were given access to Netflix’s (anonymized) data on the movies users had rented and the ratings they had given them. The winning team was the first to improve on Netflix’s recommendation engine by at least 10%. The contest was eventually won by a team from AT&T, but it was a hard-fought contest against 20,000 other teams, and it took three years for the AT&T team to reach an improvement of 10.06%.

Predictive models like this one, can suggest what movie, book, or music you might like, but they do not explain why people like the movies they do. Netflix models don’t explain why I might love Napoleon Dynamite but not like Dumb and Dumber (which is the case).

Scott Page, in his fantastic book The Model Thinker, provides these examples of models that predict without explaining:

“Deep-learning algorithms can predict product sales, tomorrow’s weather, price trends, and some health outcomes, but they offer little in the way of explanation. Such models resemble bomb-sniffing dogs. Even though a dog’s olfactory system can determine whether a package contains explosives, we should not look to the dog for an explanation of why the bomb is there, how it works, or how to disarm it.”

Models That Explain

Other models are developed to explain the world. They provide cause and effect. For example, geologists and geophysicists have developed a plate tectonic model of the earth that explains the occurrence of earthquakes and volcanoes. However, the plate tectonic model can’t predict when major earthquakes or volcanoes will occur. Likewise, evolution is a model that explains the change of species over time but it cannot predict how a species will change in the future.

The social sciences, such as economics and sociology, produce mainly explanatory models. Good models exist to explain why the economy and people behave as they do, but it is difficult to predict how the economy and people will behave in specific instances in the future. Good predictive models are rare in the social sciences.

Models and the Financial Markets

Unfortunately, we tend to conflate the two types of models and use explanatory models to try to predict the future.

A good example is price-to-earnings ratios (P/E). P/E ratios can explain whether the market or a stock has become more or less expensive on an absolute or relative basis. This is useful information. However, P/E ratios have very low predictive ability. Research from Vanguard concludes that P/E ratios have a less than a 0.1 R2 to the next year’s return and about 0.4 to the next ten years. Thus, an investor who relies on P/E ratios as a signal to buy or sell is conflating explanation with prediction, and betting the future on a weak indicator.

Similarly, many investors follow the CBOE’s Volatility Index (known as the VIX), which is a measure of stock market volatility. The VIX describes the current volatility in the stock market and provides insight about what future volatility may be. However, it does not predict future stock market returns. Investors who use the VIX as a forward-looking gauge of returns are using the VIX incorrectly.

Reliable predictive models are rare in economics and investments because the economy and stock market are complex adaptive systems. Being a complex adaptive systems means that the market and the economy are comprised of intelligent actors who observe what happens and what other actors do, and react accordingly. This phenomenon of intelligent actors watching and reacting to each other creates feedback loops which render predicting the path of future events nearly impossible.

Why economic predictive models fail was summarized well by statistician H. Eugene Stanley of Boston University: “The economy is a very complex system—like the weather—that we understand bits of. You sure as heck can’t decide on a Monday whether the weather will be nice on the coming weekend. No one can predict where the market will be at the end of the week.” Thus, models that can reliably predict what the stock market and the economy will do in the future do not exist. And they quite possibly never will.

St. Louis Trust & Family Office is an independent, multi-family office and trust company that advises clients on more than $13 billion of investment assets and more than $15 billion of total wealth. Founded in 2002, St. Louis Trust & Family Office provides holistic, high-touch client service including customized, independent investment management and a full range of family office and fiduciary services. The firm serves a limited number of clients with substantial wealth in order to maintain very low client-to-employee ratios. Visit stlouistrust.com to explore how the firm manages complexity with unmatched expertise and focuses on Family, Always.

Back to Our Insights