A platform penalty for news? How social media context can alter information credibility online

ABSTRACT

Growing concern about dubious information online threatens the credibility of legitimate news. We examine two possible mechanisms for this effect on social media. First, people might view all news on social media as less credible. Second, questionable information elsewhere in a news feed might discredit legitimate news coverage. Findings from a preregistered experiment confirm that people see information on Facebook as less credible than identical information on news websites, though the effect is small, suggesting that observational data overstates this platform penalty. Prior exposure to low (versus high) credibility information on Facebook also reduces engagement with a target article, but not its perceived credibility. However, exploratory analyses show that the effects of prior exposure to low credibility information vary depending on the plausibility of the target article, decreasing the credibility of a less plausible article (a spillover effect) but increasing the credibility of a more plausible one (a contrast effect).

KEYWORDS:

• Social media
• news consumption
• news credibility
• Facebook

Acknowledgement

Agadjanian is a political science Ph.D. student at the University of California, Berkeley. Nyhan is the James O. Freedman Presidential Professor of Government at Dartmouth College ([email protected]). Other coauthors are or were undergraduate students at Dartmouth. We thank the Dartmouth Center for the Advancement of Learning for generous funding support.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/19331681.2022.2105465

Notes

1. All hypotheses were preregistered with Evidence in Governance and Politics (EGAP) prior to data collection (the preregistration is provided in Online Appendix C and is available online at https://osf.io/6dzeu along with full replication materials).

2. The distribution of participant demographics by condition is provided in Online Appendix B; these characteristics did not vary significantly across the experimental conditions.

3. The original articles were modified to fit the desired tone and length by credibility type. We also randomized whether participants were assigned to see the KKK or Kansas article first. See Online Appendix A for full details.

4. There was a minor error in the instrument. The first response option for “How biased would you expect the article on the Kansas law to be?” was “Not at all credible” instead of “Not at all biased.” This question is not relevant to the main analysis.

5. All deviations are labeled below; full replication data and code will be made available online after publication. Online Appendix C contains the entire preregistration document. Per the preregistration, Online Appendix B shows robustness of results when using ordered probit models (for ordered dependent variables) and when using respondent random effects. We also confirm that treatment effects pool across stimuli and question order, as there are no differences in treatment effects by question order, between versions of the low and high credibility articles, and between versions of the placebo content (results available upon request).

6. Table B3 in Online Appendix B shows that we cannot reject the null hypothesis of no difference in treatment effects in a pooled model including an interaction effect.

7. Unlike the results for H2, credibility effects on article engagement were largely consistent across target articles. The only exception was the likelihood of sharing the article after exposure to the low credibility article, which decreased marginally more for the fajitas article than the wrestler article ($p<.10$; see Table B11).