What We Relearned and Learned from the 2016 Elections: Comment on Gelman and Azari

1. Introduction

I largely agree with Andrew Gelman and Julia Azari on the fully 19 things that they say we learned from the 2016 elections. I want to comment here, first, on what we in fact have relearned, and then elaborate further on what Gelman and Azari have emphasized. I end with what I would add regarding lessons for the 2020 election.

2. Lessons Relearned

How soon we forget, and Gelman and Azari did not mention what baseball legend and language master Yogi Berra would have reminded us regarding the 2016 election polling: (1) “It's ‘de ja vu’ all over again!” And (2) “…But the similarities are different!” (see Shapiro 2017a). This election hearkened back to the 1936 and especially the 1948 elections in which pollsters—as both pollsters and pundits—demonstrated unadulterated arrogance or hubris. In 1936 the folks at The Literary Digest magazine flaunted the prediction based on their multiple million ballot straw poll (that had been mailed to their subscribers and names from telephone, car registration and other lists—which had a distinctively upper status bias) that Alfred Landon would defeat President Franklin Roosevelt. The poll had gotten the winner right in every election from 1916 through FDR in 1932, so what could go wrong? Everything, thanks to the political realignment in which lower status voters missed in the straw poll disproportionately broke toward the Democrat Roosevelt. That year the more “scientific” (that is, engaging in something closer, but still far from, probability sampling) pollsters George Gallup, Elmo Roper, and Archibald Crossley predicted an easy Roosevelt victory and put the Digest to shame (it went out of business not long afterward). But Crossley and Gallup—who was then and still is the most famous of the lot—still underestimated Roosevelt's vote (60.7%) by fully 7 percentage points (compared to the Digest's 20 points), and Gallup continued to underestimate Roosevelt's vote in the next two election. So something was still amiss in the polls. The question of poll accuracy during this time, as the pollsters announced their predictions, got some attention, including calls for congressional investigation of the polls (on this forgotten and not well-remembered point, see especially Fried (2012) here and the discussion that follows). The arrogance associated with these predictions came home to roost infamously in the 1948 election, when all the pollsters predicted that Thomas Dewey would defeat Harry Truman by 5 percentage points or more (whereas the results were close to the opposite), leading to the famous dead wrong Chicago Tribune headline, “Dewey Defeats Truman.”

The embarrassment and vehement criticism of pundits, predictors, and pollsters in the 2016 election was on par with what happened in 1948. To get a flavor of the pollsters arrogance, and their shame, after the 1948 election, read the preface of Lindsay Rogers' book, The Pollsters (1949; Rogers had coined the term “pollster,” as a pejorative like “huckster,” before the 1948 debacle). The lesson relearned here is that, in the language of Gelman and Azari, “bias is a thing” and the perceived hubris of those predicting elections on the basis of polls has serious consequences.

The further lesson is that such errors in polling lead to investigations. For the 1948 fiasco, the Social Science Research Council commissioned a report which was done quickly and comprehensively (Mosteller et al. 1949). Two of the problems with the polls that the report cited are no longer issues today: There were selection biases in the 1948 polls with in-person interviewers in the field given discretion in filling quota samples. This underestimated support due to missing lower status voters. This serious problem was corrected by adopting methods closer to full probability. The second problem, referred to as “tentative,” was that polling had ended two weeks before the election, so late shifts in voting were missed. Today polling has been extended up until election-day.

But the other problems that the report cited for 1948 have remained problems—indeed, bigtime in the 2016 election and they were of most concern for state polling: The difficulty in allocating the “undecided” responses, and the problem that we would now refer to of identifying actual voters in order to estimate the electorate on election day. In addition, and most important with respect to understanding the claims of pundits and predictors, as Gelman and Azari aptly cover: the exaggerated impression of poll accuracy and the lack of general knowledge and understanding of probabilities based on polls results.

3. Adding to Gelman and Azari's Discussion

While the 2016 election seems like “de ja vu’ all over again,” some of “the similarities are different.” The American Association for Public Opinion Research already had a committee in place before the 2016 election to study the pre-election polls, so it was mobilized to work under greater pressure and responsibility (Ad Hoc Committee on 2016 Election Polling 2017), as the SSRC committee had done in 1949. As was also the case after the 1948 election (as Fried 2012 astutely described), the survey research profession and its networks has collectively engaged in rebounding from its mistakes and re-establishing its credibility, to insure its survival. Like its counterpart in 1948, the AAPOR committee did excellent work—and with more data and better technology and statistical tools. Gelman and Azari do not cite this report but they cover several if not most of the issues it raised. While the AAPOR report did not flag the perceived arrogance of the pundits, predictors, etc., it heavily emphasized the underestimation of Donald Trump's support in the states, which Gelman and Azari examine neatly. Nonresponse bias is indeed “a thing,” as they say, in that based on the exit polls it looks like voters broke for Trump in the last week in the states that he won narrowly, and this was not picked up adequately in the polls. Many state polls underestimated Trump's (often referred to as “working class”) support by failing to weight by education, that is, due to nonresponse biases there was underrepresentation of Trump's noncollege degree supporters in polls that did not correct with weights.

There was indeed systematic underestimation in the state polls, but I would add that the difference between the national and state polling is telling. The national polls were generally accurate by historical stands: Hillary Clinton winning by an average of 3–4 points compared to her actual 2 point win of the popular vote. This was based on what evidently were a good number of high quality polls: realclearpolitics.com, for example, reported the results of fully 11 polls ending November 6–7, 2017. This was not the case in the key states—Wisconsin, Michigan, and Pennsylvania—in which the underestimation of Trump voters stunned virtually everyone: realclearpolitics.com reported on only 2 polls ending Nov. 1–2 and none after that in Wisconsin; only 4 polls ending Nov. 3–4 and 1 for Nov. 6 in Michigan; and 4 ending Nov. 3–4 and 2 ending Nov. 5 in Pennsylvania. The polling problem in these states was both the quantity and quality of polls. The lesson here is that polls cost money, and the media and other pollsters have to make choices on how to expend limited polling resources, and this has led to more and better national polls than state polls. The result in 2016, then, was that polling data bearing directly on the Democrats' “blue wall” was off the mark. This wall assumed the above three states were solidly Democratic based on several recent elections. The problem Al Gore had in 2000 was fixed in that New Hampshire has become Democratic since then (Gore could have won in 2000 had he taken NH—he would not have needed Florida), and because Colorado and Virginia have become Democratic, Clinton could have won the 2016 election without winning Iowa, Ohio, Florida, and North Carolina. But this assumes Wisconsin, Michigan, and Pennsylvania were in the bag. This all raises questions regarding the future of polling. The conclusion, however, that can be drawn from the AAPOR report and elsewhere (see Keeter et al. 2017) is that, overall, national surveys can still be done sufficiently well and accurately—though expensively—even with low response rates. Estimating/weighting to known census demographic characteristics is more accurate than attempting to weight to what is an unknown electorate in an upcoming election.

Turning to the substantive aspects of the election, Gelman and Azari are on target in affirming that polarization is real, partisanship is important, and that there is an authoritarian dimension in American politics. While the emotional tenor and perhaps the substance of partisan conflict reached a high in the 2016 election due to Trump's campaign, I would argue that this substantially ideological conflict would have been just as great whoever the Republican candidate was. It had been growing for years and even further during the Obama presidency (see Shapiro and Bloch-Elkon 2006; Bafumi and Shapiro 2009; Shapiro 2017b). While this conflict may have been largely elite-driven in developing and in later penetrating to the level of public opinion, the mass base of the Republican party more so than for the Democrats has exerted increasing pressure toward maintaining this level of partisan conflict (see Hetherington 2001; Grossman and Hopkins 2016). Gelman and Azari's point about public opinion no longer exclusively following elite opinion, is well taken. Also regarding polarization, even before the 2016 election, it was becoming increasing evident that partisan conflict led to increasing partisan biases in perceptions of facts and reality, though the 2016 campaign and the first year of the Trump administration have made these distortions, with its disagreement about real news versus “fake news,” all too clear among partisan elites and their publics; this has raised questions about the “democratic competence” of Americans and their leaders (see Page and Shapiro 1992; Shapiro and Bloch-Elkon 2008).

In emphasizing the appeal to a noticeable proportion of Trump's authoritarianism, Gelman and Azari lead us to the question of what decisively drove voters to elect Trump? Researchers have been actively trying to sort this out and it will be interesting to see what consensus emerges: Was it voters' authoritarianism, racial resentment, or the relative deprivation of white voters without college degrees (see Hetherington and Weiler 2009; Tesler 2016; Sides, Tesler, and Vavreck 2016; Pettigrew 2017). Or was it Trump's appeal to the populist “sweet spot” of economic liberalism and social conservatism (Buckley 2017)?

4. Lesson for the 2020 Election?

In Gelman's and Azar's conclusion, I would only take issue with their assertion that: “From the Democrats end, the challenge will be to broaden their geographic base of support so they can win in an electoral college which currently favors whites.” This is in fact a contentious question for the Democratic party—indeed, one that could well split the party. A strategy for appealing to white voters has more relevance to gaining back seats in the House of Representative, to holding on to and gaining Senate seats, as well as overcoming the substantial Republican advantage in state legislature and governorship for which the geographic distribution of voters and to a lesser extent gerrymandering has given Republicans an advantage. Bolstered by the success of Bernie Sanders' presidential campaign, by pointing to Clinton's loss, and by evidence that the Democrats had a more general “working class” problem, not just one with whites in this group (see Greenberg 2017), the Democratic left wants to focus on holding on to and mobilizing further the party's base, and not to moderate its positions on social, big government, and inequality issues in order to appeal to white voters. The question is, then, what is the best strategy for the Democrats to run up the big numbers they need to make gains in Congress and in the states?

In contrast, winning the presidential election is a smaller numbers game. For one, the Democrats did win the presidential popular voter in 2016. For another, there is a debate about how many white voters switched from Obama in 2012 to Trump in 2016. That is, were these in fact largely Republican oriented voters who voted for Obama and then returned to the GOP? If so, the number of white Trump Democrats was overall very small nationally (see Milbank 2017). But the fact that the numbers are small nationally is not relevant (indeed, they did not put Trump over the top nationally); what matters is whether they are large enough in specific states to make up the 70,000 vote difference to put Wisconsin, Michigan, and Pennsylvania in the blue state column? If the estimates of Trump Democrats are indeed small, the gap could be made up better in other ways by mobilizing and persuading voters with characteristics associated with the Democrats' base. African-American voter turnout fell off in 2016 and while millennials' turnout rose by 3 percentage points, it was still fully 13 points less than older age groups (Krogstad and Lopez 2017). In addition, according to the exit polls Trump did slightly better among black voters, Latinos, Asian-American, and millennials than Mitt Romney did in 2012. It may well be easier to regain the blue wall by mobilizing and persuading these voters without having to alter the Democratic party's programmatic efforts, than to go after white voters. These two strategies might not necessarily be mutually exclusive to win the presidency, but the elections nationally are another story (see Greenberg 2017).

Acknowledgments

I thank Robert Jervis and Robert Erikson for helpful discussions.