Models are important for understanding the current and future states of the world and we use many, for instance the capital asset pricing model, to help us understand markets and investing.
But most models only either explain or predict: they don’t do both. In the investment industry, people commonly confuse an explanatory model with a predictive one. This can be an expensive mistake.
A model is a simplified representation of some aspect of reality, such as a diagram, a mathematical formula, a story or an analogy. Because they are simplifications of reality, models, cannot be 100% correct. The classic example of this is a map. For it to be 100% correct it would have to be the same size as the the territory itself. So as George Box once put it, “all models are wrong, but some models are useful.”
Predictive Models Are Not Easy to Build
In 2006, when they were in the business of mailing DVDs, Netflix announced a $1 million prize to whomever could improve their algorithms that recommended movies to users. Participants were given access to Netflix’s (anonymized) data on the movies users had rented and the ratings they had given them. The winning team was the first to improve on Netflix’s recommendation engine by at least 10%. The contest was eventually won by a team from AT&T, but it was a hard-fought contest against 20,000 other teams, and it took three years for the AT&T team to reach an improvement of 10.06%.
Predictive models like this one, can suggest what movie, book, or music you might like, but they do not explain why people like the movies they do. Netflix models don’t explain why I might love Napoleon Dynamite but not like Dumb and Dumber (which is the case).
Scott Page, in his fantastic book The Model Thinker, provides these examples of models that predict without explaining:
“Deep-learning algorithms can predict product sales, tomorrow’s weather, price trends, and some health outcomes, but they offer little in the way of explanation. Such models resemble bomb-sniffing dogs. Even though a dog’s olfactory system can determine whether a package contains explosives, we should not look to the dog for an explanation of why the bomb is there, how it works, or how to disarm it.”
Models That Explain
Other models are developed to explain the world. They provide cause and effect. For example, geologists and geophysicists have developed a plate tectonic model of the earth that explains the occurrence of earthquakes and volcanoes. However, the plate tectonic model can’t predict when major earthquakes or volcanoes will occur. Likewise, evolution is a model that explains the change of species over time but it cannot predict how a species will change in the future.
The social sciences, such as economics and sociology, produce mainly explanatory models. Good models exist to explain why the economy and people behave as they do, but it is difficult to predict how the economy and people will behave in specific instances in the future. Good predictive models are rare in the social sciences.
Models and the Financial Markets
Unfortunately, we tend to conflate the two types of models and use explanatory models to try to predict the future.
A good example is price-to-earnings ratios (P/E). P/E ratios can explain whether the market or a stock has become more or less expensive on an absolute or relative basis. This is useful information. However, P/E ratios have very low predictive ability. Research from Vanguard concludes that P/E ratios have a less than a 0.1 R2 to the next year’s return and about 0.4 to the next ten years. Thus, an investor who relies on P/E ratios as a signal to buy or sell is conflating explanation with prediction, and betting the future on a weak indicator.
Similarly, many investors follow the CBOE’s Volatility Index (known as the VIX), which is a measure of stock market volatility. The VIX describes the current volatility in the stock market and provides insight about what future volatility may be. However, it does not predict future stock market returns. Investors who use the VIX as a forward-looking gauge of returns are using the VIX incorrectly.
Reliable predictive models are rare in economics and investments because the economy and stock market are complex adaptive systems. Being a complex adaptive systems means that the market and the economy are comprised of intelligent actors who observe what happens and what other actors do, and react accordingly. This phenomenon of intelligent actors watching and reacting to each other creates feedback loops which render predicting the path of future events nearly impossible.
Why economic predictive models fail was summarized well by statistician H. Eugene Stanley of Boston University: “The economy is a very complex system—like the weather—that we understand bits of. You sure as heck can’t decide on a Monday whether the weather will be nice on the coming weekend. No one can predict where the market will be at the end of the week.” Thus, models that can reliably predict what the stock market and the economy will do in the future do not exist. And they quite possibly never will.
