The Fragility of Prediction Models: Are We Bound to Fail?

The Fragility of Prediction Models: Are We Bound to Fail?Predicting the Future: What If Our Models Fail? A significant volume of content is dedicated to predicting the future, often based on models that are regarded as dependable predictive tools. These models are usually derived from historical examples, which are then used to create a framework of how the world operates. The assumption is that these historical events are not anomalies, but rather indicative of patterns that will repeat themselves in the future. The Complexity of History History is multifaceted and therefore open to interpretation. The victors or survivors often write histories to serve their interests, such as protecting their reputation or covering up their mistakes. Subsequent historians may unearth new information and present fresh interpretations that reflect the spirit of their era. Those making predictions usually start with a model, like Keynesian economics, and then look for data to support their chosen model. It's relatively uncommon for an analyst to start with a mass of often incomplete and contradictory data points and make a prediction about the future without proposing a model. Without a theoretical model, skeptics can easily argue that the data was a one-off and therefore has little predictive value. The Reliance on Models This reliance on models leads to a strong attachment to them. Without a model, history can quickly devolve into a series of seemingly random events devoid of causal patterns. Our innate desire to piece together causal chains as a means to develop predictive tools leads us to seek universal models of how the world works, even when the data is scarce and our understanding is limited. When we lack evidence and understanding of unseen dynamics, we may jump to conclusions, such as attributing thunder to angry gods. We like to think that we've advanced far beyond such causal errors and that our understanding is now essentially god-like. We take it for granted that our models are proven, and the only debate is which model will be more successful in predicting the future. The Potential Failure of Our Models But what if none of our models are as predictively useful as we believe? What if the Keynesian economic model completely misses the mark, and following that model is simply doing more of the same failed actions? Maybe competing models will also fall short. The prospect that all of our models will fail to accurately predict the future rarely crosses our minds. It nullifies the entire project of making accurate predictions and planning our responses. If we accept the possibility that the next few years cannot be accurately predicted for various reasons, then our plans and our thinking must be contingent and flexible. We must be prepared to abandon our entire structure of models, data, and expectations, and respond without any confidence in the models we've committed to and are reluctant to give up. This is challenging because it requires substantial humility and a willingness to admit, "I was wrong, the models I've staked my entire career on are incorrect." The Role of Keystones in Systems Consider the role of keystones in arches and ecosystems. We understand that removing the keystone from an arch causes the arch to collapse. However, we're surprised when removing a species from an ecosystem causes the ecosystem to collapse because we didn't recognize that the species was the keystone species of that self-organizing system. Our ability to identify the many keystones in sprawling, complex systems is not as god-like as we believe. This is the source of the centuries-long debate about what caused the collapse of the western Roman Empire. The empire was likely beset by what we now call a polycrisis, a set of independent crises that fed back into one another, exacerbating the overall situation from manageable to overwhelming. Predictions and Their Shaky Foundations This is why I doubt any of the predictions about the future of the global and US economies, geopolitics, etc., will prove accurate. Every prediction is based, either explicitly or implicitly, on a model with shaky foundations and therefore shaky causality. These models fail to identify the keystones in each complex subsystem that makes up the system the model is attempting to represent. The Fall of the Roman Empire provides a keystone of insight into western Rome's collapse: the elite's complacent belief in Rome's eventual success and their inability to recognize the novel challenges they faced. The current situation goes largely unrecognized. The technical-managerial experts all share a complacent acceptance of things as they are, without a single new idea. Their confidence in their models is so great that they see no need for new ideas. Bottom Line Instead of piling up more predictions, it might be more prudent to start accumulating humility and preparing to discard all the old models and solutions before they sink the lifeboat. What are your thoughts on this matter? Do you think our current models are sufficient to predict the future, or do you agree that we might need to rethink our approach? Share this article with your friends and let us know your thoughts. Don't forget to sign up for the Daily Briefing, which is every day at 6pm.

Some articles will contain credit or partial credit to other authors even if we do not repost the article and are only inspired by the original content.

Some articles will contain credit or partial credit to other authors even if we do not repost the article and are only inspired by the original content.