Problems with economic models

Most economic models rest on a number of assumptions that are not entirely realistic. For example, agents are often assumed to have perfect information, and markets are often assumed to clear without friction. Or, the model may omit issues that are important to the question being considered, such as externalities. Any analysis of the results of an economic model must therefore consider the extent to which these results may be compromised by inaccuracies in these assumptions, and there is a growing literature debunking economics and economic models.

Restrictive, unrealistic assumptions

Provably unrealistic assumptions are pervasive in neoclassical economic theory (also called the "standard theory" or "neoclassical paradigm"), and those assumptions are inherited by simplified models for that theory. (Any model based on a flawed theory, cannot transcend the limitations of that theory.) Joseph Stiglitz' 2001 Nobel Prize lecture reviews his work on information asymmetries, which contrasts with the assumption, in standard models, of "perfect information". Stiglitz surveys many aspects of these faulty standard models, and the faulty policy implications and recommendations that arise from their unrealistic assumptions. Stiglitz writes: (pp. 519–20)

"I only varied one assumption—the assumption concerning perfect information—and in ways which seemed highly plausible. ... We succeeded in showing not only that the standard theory was not robust—changing only one assumption in ways which were totally plausible had drastic consequences, but also that an alternative robust paradigm with great explanatory power could be constructed. There were other deficiencies in the theory, some of which were closely connected. The standard theory assumed that technology and preferences were fixed. But changes in technology, R & D, are at the heart of capitalism. ... I similarly became increasingly convinced of the inappropriateness of the assumption of fixed preferences. (Footnote: In addition, much of recent economic theory has assumed that beliefs are, in some sense, rational. As noted earlier, there are many aspects of economic behavior that seem hard to reconcile with this hypothesis.)"

Economic models can be such powerful tools in understanding some economic relationships that it is easy to ignore their limitations. One tangible example where the limits of economic models allegedly collided with reality, but were nevertheless accepted as "evidence" in public policy debates, involved models to simulate the effects of NAFTA, the North American Free Trade Agreement. James Stanford published his examination of 10 of these models.

The fundamental issue is circularity: embedding one's assumptions as foundational "input" axioms in a model, then proceeding to "prove" that, indeed, the model's "output" supports the validity of those assumptions. Such a model is consistent with similar models that have adopted those same assumptions. But is it consistent with reality? As with any scientific theory, empirical validation is needed, if we are to have any confidence in its predictive ability.

If those assumptions are, in fact, fundamental aspects of empirical reality, then the model's output will correctly describe reality (if it is properly "tuned", and if it is not missing any crucial assumptions). But if those assumptions are not valid for the particular aspect of reality one attempts to simulate, then it becomes a case of "GIGO" – Garbage In, Garbage Out".

James Stanford outlines this issue for the specific Computable General Equilibrium ("CGE") models that were introduced as evidence into the public policy debate, by advocates for NAFTA:

"...CGE models are circular: if trade theory holds that free trade is mutually beneficial, then a quantitative simulation model based on that theoretical structure will automatically show that free trade is mutually beneficial...if the economy actually behaves in the manner supposed by the modeler, and the model itself sheds no light on this question, then a properly calibrated model may provide a rough empirical estimate of the effects of a policy change. But the validity of the model hangs entirely on the prior, nontested specification of its structural relationships ... [Hence, the apparent consensus of pro-NAFTA modelers] reflects more a consensus of prior theoretical views than a consensus of quantitative evidence."

Commenting on Stanford's analysis, one computer scientist wrote,

"When simulating the impact of a trade agreement on labor, it seems absurd to assume a priori that capital is immobile, that full employment will prevail, that unit labor costs are identical in the U.S. and Mexico, that American consumers will prefer products made in America (even if they are more expensive), and that trade flows between the U.S. and Mexico will exactly balance. Yet a recent examination of ten prominent CGE models showed that nine of them include at least one of those unrealistic assumptions, and two of the CGE models included all the above assumptions.

This situation has been alleged to bear a disturbing resemblance to computer-assisted intellectual dishonesty. Human beings have always been masters of self-deception, and hiding the essential basis of one's deception by embedding it in a computer program surely helps reduce what might otherwise become an intolerable burden of cognitive dissonance."

In commenting on the general phenomenon of embedding unrealistic "GIGO" assumptions in neoclassical economic models, Joseph Stiglitz is only slightly more diplomatic: (pp. 507–08)

"But the ... model, by construction, ruled out the information asymmetries which are at the heart of macro-economic problems. Only if an individual has a severe case of schizophrenia is it possible for such problems to arise. If one begins with a model that assumes that markets clear, it is hard to see how one can get much insight into unemployment (the failure of the labor market to clear)."

Despite the prominence of Stiglitz' 2001 Nobel prize lecture, the use of arguably misleading neoclassical models persisted in 2007, according to these authors:

" ... projected welfare gains from trade liberalization are derived from global computable general equilibrium (CGE) models, which are based on highly unrealistic assumptions. CGE models have become the main tool for economic analysis of the benefits of multilateral trade liberalization; therefore, it is essential that these models be scrutinized for their realism and relevance. ... we analyze the foundation of CGE models and argue that their predictions are often misleading. ... We appeal for more honest simulation strategies that produce a variety of plausible outcomes."

The working paper, "Debunking the Myths of Computable General Equilibrium Models", provides both a history, and a readable theoretical analysis of what CGE models are, and are not. In particular, despite their name, CGE models use neither the Walrass general equilibrium, nor the Arrow-Debreus General Equilibrium frameworks. Thus, CGE models are highly distorted simplifications of theoretical frameworks—collectively called "the neoclassical economic paradigm"—which—themselves—were largely discredited by Joseph Stiglitz.

In the "Concluding Remarks" (p. 524) of his 2001 Nobel Prize lecture, Stiglitz examined why the neoclassical paradigm—and models based on it—persists, despite his publication, over a decade earlier, of some of his seminal results showing that Information Asymmetries invalidated core Assumptions of that paradigm and its models:

"One might ask, how can we explain the persistence of the paradigm for so long? Partly, it must be because, in spite of its deficiencies, it did provide insights into many economic phenomena. ... But one cannot ignore the possibility that the survival of the [neoclassical] paradigm was partly because the belief in that paradigm, and the policy prescriptions, has served certain interests."

In the aftermath of the 2007–2009 global economic meltdown, the profession's alleged attachment to unrealistic models is increasingly being questioned and criticized. After a weeklong workshop, one group of economists released a paper highly critical of their own profession's allegedly unethical use of unrealistic models. Their Abstract offers an indictment of fundamental practices:

"The economics profession appears to have been unaware of the long build-up to the current worldwide financial crisis and to have significantly underestimated its dimensions once it started to unfold. In our view, this lack of understanding is due to a misallocation of research efforts in economics. We trace the deeper roots of this failure to the profession’s focus on models that, by design, disregard key elements driving outcomes in real-world markets. The economics profession has failed in communicating the imitations, weaknesses, and even dangers of its preferred models to the public. This state of affairs makes clear the need for a major reorientation of focus in the research economists undertake, as well as for the establishment of an ethical code that would ask economists to understand and communicate the limitations and potential misuses of their models."

Omitted details

A great danger inherent in the simplification required to fit the entire economy into a model is omitting critical elements. Some economists believe that making the model as simple as possible is an art form, but the details left out are often contentious. For instance:

• Market models often exclude externalities such as pollution. Such models are the basis for many environmentalist attacks on mainstream economists. It is said that if the social costs of externalities were included in the models their conclusions would be very different, and models are often accused of leaving out these terms because of economist's pro-free market bias.
• In turn, environmental economics has been accused of omitting key financial considerations from its models. For example, the returns to solar power investments are sometimes modelled without a discount factor, so that the present utility of solar energy delivered in a century's time is precisely equal to gas-power station energy today.
• Financial models can be oversimplified by relying on historically unprecedented arbitrage-free markets, probably underestimating the chance of crises, and under-pricing or under-planning for risk.
• It is possible that any missing variable as well as errors in values of included variables can lead to erroneous results.
• Model risk: There is a significant amount of model risk inherent in the current mathematical modeling approaches to economics that one must take into account when using them. A good economic theory should be built on sound economic principles tested on many free markets, and proven to be valid. However, empirical facts have been alleged to indicate that the principles of economics hold only under very limited conditions that are rarely met in real life, and there is no scientific testing methodology available to validate hypotheses. Decisions based on economic theories that are not scientifically possible to test can give people a false sense of precision, and that could be misleading, leading to build up logical errors.
• Natural economics: Economics is concerned with both 'normal' and 'abnormal' economic conditions. In an objective scientific study one is not restricted by the normality assumption in describing actual economies, as much empirical evidence shows that some "anomalous" behavior can persist for a long time in real markets e.g., in market "bubbles" and market "herding".