Curbing Optimism Bias and Strategic Misrepresentation in Planning: Reference Class Forecasting in Practice

Absract

The American Planning Association recently endorsed a new forecasting method called reference class forecasting, which is based on theories of planning and decision-making that won the 2002 Nobel prize in economics. This paper details the method and describes the first instance of reference class forecasting in planning practice. First, the paper documents that inaccurate projections of costs, demand, and other impacts of plans are a major problem in planning. Second, the paper explains inaccuracy in terms of optimism bias and strategic misrepresentation. Third, the theoretical basis is presented for reference class forecasting, which achieves accuracy in projections by basing them on actual performance in a reference class of comparable actions and thereby bypassing both optimism bias and strategic misrepresentation. Fourth, the paper presents the first case of practical reference class forecasting, which concerns cost projections for planning of large transportation infrastructure investments in the UK, including the Edinburgh Tram and London's £15 billion Crossrail project. Finally, potentials for and barriers to reference class forecasting are assessed.

Acknowledgments

Bent Flyvbjerg would like to acknowledge the contributions of Daniel Kahneman and Dan Lovallo to the underlying ideas in this article. The author would also like to thank David Banister, Christian Brockmann, Antonio Estache, Phil Goodwin, Peter Hall, Christoph Lieb, Dan Lovallo, Hugo Priemus, Emile Quinet, Mateo Turro, José Manuel Vassallo, and Roger Vickerman.

Notes

1. Inaccuracy is measured in percentages as (actual outcome/forecast outcome – 1)×100. The base year of a forecast for a project is the time of decision to build that project. An inaccuracy of zero indicates perfect accuracy. Cost is measured as construction costs in constant prices. Demand is measured as number of vehicles for roads and number of passengers for rail.

2. The closest thing to an outside view in large infrastructure forecasting is Gordon and Wilson's (1984) use of regression analysis on an international cross section of light-rail projects to forecast patronage in a number of light-rail schemes in North America.

3. The fact that this is, indeed, the first instance of practical reference class forecasting has been confirmed with Daniel Kahneman and Dan Lovallo, who also know of no other instances of practical reference class forecasting. Personal communications with Daniel Kahneman and Dan Lovallo, author's archives.