German Election Forecasting: Comparing and Combining Methods for 2013

Abstract

The present study reviews the accuracy of four methods for forecasting the 2013 German election: polls; prediction markets; expert judgement; and quantitative models. On average, across the two months prior to the election, polls were most accurate, with a mean absolute error of 1.4 percentage points, followed by quantitative models (1.6), expert judgement (2.1) and prediction markets (2.3). In addition, the study provides new evidence for the benefits of combining forecasts. Averaging all available forecasts within and across the four methods provided more accurate predictions than the typical component forecast. The error reductions achieved through combining forecasts ranged from 5 per cent (compared to polls) to 41 per cent (compared to prediction markets). The results conform to prior research on US presidential elections, which showed that combining is one of the most effective methods to generating accurate election forecasts.

ACKNOWLEDGMENTS

Mario Haim helped developing the PollyVote.de website and collecting the forecast data. Bettina Zerwes collected the names for the expert survey. This work was supported by an LMU excellent research fellowship from the Center for Advanced Studies at LMU Munich.

SUPPLEMENTAL DATA AND RESEARCH MATERIALS

The underlying research materials for this article can be accessed at http://dx.doi.org/10.7910/DVN/GERMANPOLLYVOTE2013.

ABOUT THE AUTHOR

Andreas Graefe is a research fellow at the Center for Advanced Studies and the Department of Communication Science and Media Research at LMU Munich. His research interests focus on the development and validation of forecasting methods in different fields, in particular, political forecasting. His work is published, among others, in Public Opinion Quarterly, Journal of Business Research, International Journal of Forecasting and Journal of Behavioral Decision Making. Dr Graefe also created the research focus ‘Political Forecasting' at LMU's Center for Advanced Studies and he runs the PollyVote for forecasting German and US elections.

Notes

1. See, for example, the special symposiums in Political Methodologist 5/2, American Politics Research 24/4 and PS: Political Science and Politics 34/1, 37/4, 41/4 and 45/4.

2. J.E. Berg and T.A. Rietz, ‘Market Design, Manipulation, and Accuracy in Political Prediction Markets: Lessons from the Iowa Electronic Markets’, PS: Political Science & Politics 47/2 (2014), pp.293–6.

3. M.S. Lewis-Beck and C. Tien, ‘Voters as Forecasters: A Micromodel of Election Prediction’, International Journal of Forecasting 15/2 (1999), pp.175–84; M.S. Lewis-Beck and M. Stegmaier, ‘Citizen Forecasting: Can UK Voters See the Future?’, Electoral Studies 30/2 (2011), pp.264–8; A. Graefe, ‘Accuracy of Vote Expectation Surveys in Forecasting Elections’, Public Opinion Quarterly 78/S1 (2014), pp.204–32.

4. A. Graefe, J.S. Armstrong, R.J. Jones, Jr and A.G. Cuzán, ‘Combining Forecasts: An Application to Elections', International Journal of Forecasting 30/1 (2014), pp.43–54.

5. J.M. Bates and C.W.J. Granger, ‘The Combination of Forecasts', OR 20/4 (1969), pp.451–68; R.T. Clemen, ‘Combining Forecasts: A Review and Annotated Bibliography’, International Journal of Forecasting 5/4 (1989), pp.559–83; J.S. Armstrong, K.C. Green and A. Graefe, ‘Golden Rule of Forecasting: Be Conservative', Journal of Business Research (forthcoming), available from www.goldenruleofforecasting.com.

6. J.S. Armstrong, ‘Combining Forecasts', in J.S. Armstrong (ed.), Principles of Forecasting: A Handbook for Researchers and Practitioners (New York: Springer, 2010), pp.417–39.

7. Graefe et al., ‘Combining Forecasts'.

8. Armstrong, ‘Combining Forecasts'.

9. Bates and Granger, ‘The Combination of Forecasts'.

10. M. Blumenthal, ‘Polls, Forecasts, and Aggregators', PS: Political Science & Politics 47/2 (2014), pp.297–300.

11. A. Graefe, J.S. Armstrong, R.J. Jones, Jr and A.G. Cuzán, ‘Accuracy of Combined Forecasts for the 2012 Presidential Election: The PollyVote', PS: Political Science & Politics 47/2 (2014), pp.427–31.

12. Graefe et al., ‘Combining Forecasts’.

13. D. Rothschild, ‘Combining Forecasts for Elections: Accurate, Relevant, and Timely', International Journal of Forecasting (2014), doi: 10.1016/j.ijforecast.2014.08.006.

14. D.S. Hillygus, ‘The Evolution of Election Polling in the United States’, Public Opinion Quarterly 75/5 (2011), pp.962–81.

15. P.W. Rhode and K.S. Strumpf, ‘The Long History of Political Betting’, in L. Vaughan Williams and D.S. Siegel (eds), The Oxford Handbook of the Economics of Gambling (Oxford: Oxford University Press, 2014), pp.560–86.

16. P.W. Rhode and K.S. Strumpf, ‘Historical Presidential Betting Markets’, Journal of Economic Perspectives 18/2 (2004), pp.127–41.

17. J.E. Berg and T.A. Rietz, ‘Market Design, Manipulation, and Accuracy in Political Prediction Markets: Lessons from the Iowa Electronic Markets’, PS: Political Science & Politics 47/2 (2014), pp.293–6.

18. S. Kernell, ‘Life Before Polls: Ohio Politicians Predict the 1828 Presidential Vote’, PS: Political Science and Politics 33/3 (2000), pp.569–74.

19. See, for example, the models published for US presidential elections since 1992 in special symposiums in Political Methodologist 5/2, American Politics Research 24/4 and PS: Political Science and Politics 34/1, 37/4, 41/4 and 45/4.

20. B. Jérôme, V. Jérôme-Speziari and M.S. Lewis-Beck, ‘A Political-Economy Forecast for the 2013 German Elections: Who to Rule with Angela Merkel?’, PS: Political Science & Politics 46/3 (2013), pp.479–80.

21. H. Norpoth and T. Gschwend, ‘Chancellor Model Picks Merkel in 2013 German Election', PS: Political Science & Politics 46/3 (2013), pp.481–2.

22. M. Kayser and A. Leininger, ‘A Benchmarking Forecast and Post-Mortem of the 2013 Bundestag Election', Paper presented at the joint DVPW-Arbeitskreise conference Die Bundestagswahl 2013 (‘Politik und Kommunikation’ and ‘Wahlen und politische Einstellungen’), Berlin, 5–6 June 2014, available from https://www.researchgate.net/publication/262767623.

23. P. Selb and S. Munzert, ‘Forecasting the 2013 Bundestag Election Using Data from Various Polls', SSRN Working Paper, doi: 10.2139/ssrn.2313845.

24. Only election.de provided forecasts for each party. For the remaining models, the forecasts for the individual parties were calculated by using the distribution of the PollyVote forecast from the preceding day. All data and calculations are publicly available: A. Graefe (2015) ‘Replication data for: German election forecasting: Comparing and combining methods for 2013, Harvard Dataverse Network’, available from http://dx.doi.org/10.7910/DVN/GERMANPOLLYVOTE2013.

25. R.T. Clemen, ‘Combining Forecasts: A Review and Annotated Bibliography’, International Journal of Forecasting 5/4 (1989), pp. 559–83.

26. A. Graefe, H. Küchenhoff, V. Stierle and B. Riedl, ‘Limitations of Ensemble Bayesian Model Averaging for Forecasting Social Science Problems’, International Journal of Forecasting (2014), doi: 10.2139/ssrn.2266307.

27. Graefe, ‘Replication data for: German election forecasting: Comparing and combining methods for 2013’.

28. R.S. Erikson and C. Wlezien, The Timeline of Presidential Elections: How Campaigns Do (and Do Not) Matter (Chicago, IL: University of Chicago Press).

29. L.-M. Schaffer and G. Schneider, ‘Die Prognosegüte von Wahlbörsen und Meinungsumfragen zur Bundestagswahl 2005’, Politische Vierteljahresschrift 46/4 (2005), pp.674–81; J. Groß, ‘Märkte und Prognosen’, in N. Braun, M. Keuschnigg and T. Wolbring (eds), Wirtschaftssoziologie II: Anwendungen (München: Oldenbourg, 2012), pp.111–26.