A fake news inoculation? Fact checkers, partisan identification, and the power of misinformation

ABSTRACT

Previous research finds that misinformation is often difficult to correct once a person accepts it as truth. Nonetheless, a few studies have shown evidence that fact-checkers can help lower an individual’s susceptibility to believing false news and rumors. Our study builds on this research by examining the fact-checking inoculation effect on political misinformation (also known as “fake news”) that circulated on the Internet in the months following the election of President Donald Trump. Using an experimental design, we find only selected instances of inoculation effects. Instead, our results are consistent with previous studies that show individuals are more likely to accept or reject misinformation based on whether it is consistent with their pre-existing partisan and ideological beliefs. However, partisanship and ideology played a much stronger role in predicting believability for fake news stories critical of Democrats than stories critical of Republicans.

KEYWORDS:

• Fake news
• misinformation
• public opinion
• partisanship
• ideology

Acknowledgements

The authors gratefully acknowledge the Thomas Harriot College of Arts and Sciences (THCAS) at East Carolina University for the services and infrastructure support that it provides to the Center for Survey Research. The authors also wish to thank Randy Knebel of the THCAS Center for Survey Research for his research and technical assistance on this project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 In the case of this experiment, these workers served as potential subjects who were asked to participate in a short political survey that would take 8–10 minutes in exchange for payment of $0.70 for their successful participation. In order to ensure successful participation, two questions were included in the experiment that had a correct answer. Specifically, we asked “Is Donald Trump President of the United States,” (yes, no), and “Please select True.” (true, false). Subjects who incorrectly answered those items were assumed to be randomly answering questions, and thus denied the ability to complete the task. Eleven subjects failed the attention check. The only other restriction on participation was that workers must have an IP address from the United States. Our institution’s IRB approved the study.

2 The experiment did not include all of the text for each story. Instead, we used screen shots of the title, subtitle, and first few sentences. We did this in an effort to keep the visual organization of the stories similar, and to maintain the appearance that the stories had when shared as links on social media. Of course, we acknowledge that the similarities in the presentations are not perfect, which is a drawback to using actual news stories compared to structured experimental stimuli. We contend, however, that the similarities across the stories are strong. (To view these items, please see https://goo.gl/zQGV6p.)

3 Some scholars have raised concerns about the external validity of adult convenience samples, such as those drawn from Mturk (see, e.g., Krupnikov and Levine 2014). Nonetheless, others report “considerable similarity” between results obtained from convenience and nationally representative population-based samples (Mullinix et al. 2015). Mturk samples, in particular, are more diverse than student samples, which scholars have long drawn from for experimental research (Berinsksy, Huber, and Lenz 2012). Mturk samples also have been shown to be valid for research on political ideology (Clifford, Jewell, and Waggoner 2015). For this reason, the use of MTurk among experimental political scientists has gained popularity and increasing scholarly acceptance as a cost-effective and valid method of inquiry.

4 As is typically the case with MTurk samples, our sample does not perfectly mirror the attributes of the general public as a whole, but our sample and the American population do align on several key demographics – age, gender and income, specifically. However, our sample is disproportionately white, well-educated, and further to the political left ideologically than the American public.

5 We also include controls for political ideology (liberal, moderate, conservative) party identification (Democrat, Independent/no affiliation, and Republican), age, race, gender, educational attainment, and income (see the Appendix for question wording and response categories). Since political ideology and party identification are both measures of one latent concept – political attitudes – multicollinearity in our regression models is a possibility. To explore this possibility we estimated the correlation between political ideology and party identification (r = .769). Although this is a strong correlation, as expected, it is not close to 1 and reflects both the strong link between ideology and party identification and the fact that it is not uncommon for a person to hold ideologies that do not necessarily align with general party principals. Therefore, we believe it is important to include both political ideology and party identification in our models. To ensure that multicollinearity is not biasing our results, we estimated variance inflation factors (VIF) for all models, with all VIF <4, which easily meets the threshold of VIF <10 (Kennedy 2008).

6 Identifying as white or identifying as female are also strongly predictive of believability of stories critical of Republicans, although in opposite directions, likely due to the strong correlation of race and gender with political perspectives and behavior.

7 Examination of these scales warrant simple ordinary least squares analyses due to the continuous nature of the variables. We believe that these aggregated indices best reflect how a person consumes and processes fake news given that news information is not a piecemeal process, but a process which combines a stream of information into a general perspective and understanding on national and global events, with prior information informing our interpretation of new information.

8 We also conducted a direct comparison of the inoculation condition to the control group (dropping all other cases). The significance of the inoculation effect did not differ from the models presented in Table 5.

9 In asking this question, we acknowledge the concerns about social desirability effects (see e.g., Presser 1990). Nonetheless, additional research finds that declining survey response rates – not question wording – is the primary cause of over-reporting of voting behavior (Burden 2000). In addition, Holbrook and Krosnick (2010) find that internet surveys are less susceptible to such social desirability response bias.