Effects of Competing Statistical and Testimonial Evidence in Debates About Science

ABSTRACT

Communication technologies have expanded the range of actors who participate in public debates about science. When experts communicate with the public, scientifically-derived statistical evidence competes with the testimony of non-experts. This study investigates how competing statistical and testimonial evidence affect attitudes toward an issue and the debating speakers. Our findings suggest an advantage to asserting statistical evidence in competitive debates about science; a dissenting lay person is considered more credible when asserting statistical evidence in response to an expert’s testimony than when they assert testimonial evidence. Additionally, prior support for the issue affects evaluations of speakers and issue attitudes.

KEYWORDS:

• Competition
• evidence
• debate
• credibility

Data availability statement

The data that support the findings are available from the corresponding author, upon reasonable request.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Samples recruited from MTurk tend to differ from those recruited using probability sampling methods on a number of different demographics including age, gender, income, and political ideology (Mullinix et al., 2015). However, research indicates strong similarities in experimental treatment effects in public opinion studies between convenience and nationally representative population-based samples, suggesting MTurk samples are valuable in theory building (see Mullinix et al., 2015). Furthermore, our confidence in the appropriateness of the sample for our study is bolstered by the distribution of prior beliefs about nuclear power, with the mean (4.00) falling at the exact midpoint of the 7-point scale.

2 Bai (2018) suggests that bots on MTurk can be detected by repeating GPS locations. We followed Bai’s suggestion, to search for “88639831” in your data […]. This is the number after the decimal point for the latitude of a GPS location. “This location was seen in multiple studies,” and responses from these locations appear to be random. We found 92 identical GPS location codes, and removed these from subsequent analyses.

3 An a priori power analysis indicated that with a sample size of 800, we would be able to detect a small to medium effect at .9 power. Given known attrition due to bot accounts on MTurk, we overrecruited to ensure we reached this sample size.

4 Findings from an integrative PROCESS approach and SEM model (not reported) find a very similar pattern of results to those presented here.

5 When running analyses with only those respondents who correctly identified at least one actor’s form of evidence (n = 704), we find a similar pattern of results, but additional associations between evidence condition and expert evaluations. Evidence condition affected evaluations of the lay person in a similar manner; lay people are evaluated more positively in the ETLS condition compared to all other conditions. Among those who correctly identified at least one evidence-source pairing, the expert sees slightly more negative evaluations in the ETLS condition, compared with ESLT and ETLT conditions. This is the only substantive difference in results between analyses of the full body of data and those respondents who correctly identified at least one actor’s evidence form, and may reflect an assumption on the part of participants that scientists will draw on statistical data. See Appendix 5 for full results of analyses on subset data.