The Polls and the U.S. Presidential Election in 2020 …. and 2024

Abstract

Arguably, the single greatest determinant of U.S. public policy is the identity of the president. And if trusted, polls not only provide forecasts about presidential-election outcomes but can act to shape those outcomes. Looking ahead to the 2024 U.S. presidential election and recognizing that polls before the 2020 presidential election were sharply criticized, we consider whether such harsh assessments are warranted. Initially, we explore whether such polls as processed by the sophisticated aggregator FiveThirtyEight successfully forecast actual 2020 state-by-state outcomes. We evaluate FiveThirtyEight’s forecasts using customized statistical methods not used previously, methods that take account of likely correlations among election outcomes in similar states. We find that, taken together, the pollsters and FiveThirtyEight did an excellent job in predicting who would win in individual states, even those “tipping point” states where forecasting is more difficult. However, we also find that FiveThirtyEight underestimated Donald Trump’s vote shares by state to a modest but statistically significant extent. We further consider how the polls performed when the more primitive aggregator Real Clear Politics combined their results, and then how well single statewide polls performed without aggregation. It emerges that both Real Clear Politics and the individual polls fared surprisingly well.

KEYWORDS:

• Forecast accuracy
• Political polling
• U.S. presidency
• U.S. elections

Acknowledgments

The authors are grateful to the editors and reviewers for their thoughtful suggestions.

Disclosure Statement

The authors have no potential competing interests.

Notes

1 In FiveThirtyEight’s nine swing states most likely to “tip” the election, the median weight it accorded to polls in its final 2020 forecast was 97%.

2 FiveThirtyEight’s modeling allows for correlated forecasts across states, but it does not disclose how, and our own approach to correlation is probably different. But test of a model need not be predicated on treating all its assumptions as correct (e.g., someone evaluating a model that assumes the earth is flat is not required to do likewise).

3 Suppose that state-by-state win/loss outcomes for Trump are positively correlated. Then a Trump victory in state A could moderately increase his chance of winning in state B, where FiveThirtyEight assigns him a 25% chance of victory, and also do so in state C (75% chance). But then the conditional probability of a win/loss error would go up from 25% at B, but this error probability would go down from 25% at C. Thus, relative to independence, the net effect on ${\sigma }^2(Z)$ of these two opposite movements could well be modest.

4 While the projected Biden/Trump difference in electoral votes could fluctuate around its mean of 85 for these 47 states, that circumstance would not meaningfully alter this approximate analysis.

5 For example, suppose that a model correctly assumes that the candidate has a 50% chance of winning in each state, but that the various outcomes have strong positive correlation. Then the percentage actually won could be polarized towards 100% or 0%, and the unbiased estimates of 50% could appear highly inaccurate.