Reference class forecasting

Reference class forecasting predicts the outcome of a planned action based on actual outcomes in a reference class of similar actions to that being forecasted. The theories behind reference class forecasting were developed by Daniel Kahneman and Amos Tversky. They helped Kahneman win the 2002 Nobel Prize in Economics.

Kahneman and Tversky (1979a, b) found that human judgment is generally optimistic due to overconfidence and insufficient consideration of distributional information about outcomes. Therefore, people tend to underestimate the costs, completion times, and risks of planned actions, whereas they tend to overestimate the benefits of those same actions. Such error is caused by actors taking an "inside view," where focus is on the constituents of the specific planned action instead of on the actual outcomes of similar ventures that have already been completed.

Kahneman and Tversky concluded that disregard of distributional information, that is, risk, is perhaps the major source of error in forecasting. On that basis they recommended that forecasters "should therefore make every effort to frame the forecasting problem so as to facilitate utilizing all the distributional information that is available" (Kahneman and Tversky 1979b, p. 316).

Using distributional information from previous ventures similar to the one being forecasted is called taking an "outside view." Reference class forecasting is a method for taking an outside view on planned actions.

Reference class forecasting for a specific project involves the following three steps:

1. Identify a reference class of past, similar projects.
2 Establish a probability distribution for the selected reference class for the parameter that is being forecasted.
3 Compare the specific project with the reference class distribution, in order to establish the most likely outcome for the specific project.

Flyvbjerg and Cowi (2004) have developed the theory of reference class forecasting into a method that is now used in practical policy and planning. The first instance of reference class forecasting in practice was a forecast carried out in 2004 by the UK government of the projected capital costs for an extension of the Edinburgh Tram (Flyvbjerg 2006). Since then the method has been applied to numerous other projects in the UK, including the £15 (US$29) billion Crossrail project in London. The Netherlands, Denmark, and Switzerland have also implemented various types of reference class forecasting.

In 2005, the American Planning Association (APA) endorsed reference class forecasting and recommended that planners should never rely solely on conventional forecasting techniques:

"APA encourages planners to use reference class forecasting in addition to traditional methods as a way to improve accuracy. The reference class forecasting method is beneficial for non-routine projects ... Planners should never rely solely on civil engineering technology as a way to generate project forecasts" (the American Planning Association 2005).

Forecasts based on reference class forecasting and the outside view have been found to produce more accurate results than conventional forecasts based on the inside view. This is explained by the fact that reference forecasting bypasses human bias, including optimism bias and strategic misrepresentation.

References

American Planning Association, 2005, "JAPA Article Calls on Planners to Help End Inaccuracies in Public Project Revenue Forecasting", news release, April 7.
Flyvbjerg, B., "From Nobel Prize to Project Management: Getting Risks Right." Project Management Journal, vol. 37, no. 3, August 2006, pp. 5-15.
Flyvbjerg, B. and Cowi, 2004, Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document (London: UK Department for Transport).
Kahneman, D. and Tversky, A., 1979a, "Prospect theory: An analysis of decisions under

risk." Econometrica, 47, pp 313-327

Kahneman, D. and Tversky, A., 1979b, "Intuitive Prediction: Biases and Corrective Procedures." In S. Makridakis and S. C. Wheelwright, Eds., Studies in the Management Sciences: Forecasting, 12 (Amsterdam: North Holland).
Lovallo, D. and Kahneman, D., 2003, "Delusions of Success: How Optimism

Undermines Executives' Decisions," Harvard Business Review, July, pp 56-63

See also

References