Naturalistic Fallacy Errors and Behavioral Science News: The Effects of Editorial Content and Cautions on Readers' Moral Inferences and Perceptions of Contributors

Abstract

Naturalistic fallacy (NF) errors occur when empirical descriptions of nature are presumed to dictate moral positions or courses of action. Two research summaries prepared as media essays for lay audiences were used to explore whether brief editorial warnings—placed at either the beginning or the end—would discourage NF errors. For both research topics, warnings yielded a “sleeper effect,” with initially suppressed NF errors returning to the level of no-warning controls over a 2-week period. Warning location, however, did not impact errors. Readers also estimated the moral views of essay authors and cited researchers to be similar to their own views and similar to each other's. Better author–researcher differentiation occurred when authors and not the researchers were recognized as responsible for the writing in essays that asserted moral claims.

ACKNOWLEDGMENTS

Thanks go to students in Willamette University's Research Methods and Analysis II course, who assisted in study design and data collection for this work.

Notes

¹Copies of the essay materials are available from the author.

Note. All proportions are based on the number out of 116 participants answering “true” to the statement. Total naturalistic fallacy (NF) scores are on a 0–4 scale based on the number of NF items for an essay each participant marked as true. Total Knowledge scores are on a 0–6 scale based on the number of correct answers to the six true–false factual questions (not listed in the table).

Note. Total naturalistic fallacy (NF) scores are on a 0–4 scale based on the number of NF items for an essay each participant marked as “true.” Overall means correspond to the respective main effect values. None of the effects involving claim placement were significant (all ps > .2 for the Advertising essay and all ps > .7 for the Conservatism essay); breakdowns by the placement variable are therefore omitted.

^a n = 39.

^b n = 38.

Note. Total naturalistic fallacy (NF) scores are on a 0–4 scale based on the number of NF items for an essay each participant marked as “true.” Overall means correspond to the respective main effect values.

^a n = 19.

^b n = 20.

Note. Total naturalistic fallacy (NF) scores are on a 0–4 scale based on the number of NF items for an essay each participant marked as “true.”

^a Composite scores for the Advertising essay perceptions averaged responses to the following three items:

1. The researchers (essay writer)think(s) their (the) research shows that advertisers should be discouraged from including “junk food” commercials in children's programs.

2. The researchers (essay writer) believe(s) that, because the research shows children to be swayed by sugared food ads, advertisers must be taking unfair advantage of children.

3. The researchers (essay writer) think(s) their (the) study results show that some form of legal constraints on advertisers is essential.

^b Composite scores for the Conservatism essay perceptions averaged responses to the following three items:

1. The researchers (essay writer) think(s) their (the) research shows that people should be on guard against the kinds of needs and motives to which conservative beliefs appeal.

2. The researchers (essay writer) believe(s) their (the) data demonstrate that people ought to base their political beliefs on reasons rather than on needs and motives.

3. The researchers (essay writer) feel(s) their (the) study results show that holding to a conservative political ideology is rather unhealthy psychologically.

^c n = 39.

^d n = 38.

*p < .05. **p < .01.

Note. NF = naturalistic fallacy.

^a n = 39.

^b n = 38.

^c n = 116.

*p < .05. **p < .01.

²A 3 (moral claim) × 2 (claim placement) × 2 (time of testing) mixed ANOVA performed on responsibility ratings yielded a marginally significant main effect for time, F(1, 110) = 3.64, p = .06,  = .012, MSE = 1.39, with the mean shifting toward the scale midpoint from initial testing (M = 5.60) to posttesting (M = 5.30). No other main or interaction effects approached significance, all ps > .25. Thus, there was no evidence to suggest that journalists were assigned more responsibility when the essays went beyond a straight summary of the research findings to either caution against value-based interpretations or to argue for a particular moral conclusion.

³Space does not permit a more detailed discussion of the author–researcher perception data and its relationship to perceived responsibility for writing; more complete analyses are available from the author. It is worth noting, however, that any confusion over writing responsibility that did occur did not appear to be closely linked to careless reading overall. At initial testing, perceived responsibility scores were uncorrelated with scores on the true–false essay comprehension or “knowledge” composites (r = .10, ns, for Advertising and r = .00, ns, for Conservatism). At posttesting, there was only a weak association between greater responsibility attributed to journalists and better performance on the knowledge composites (r = .18, p = .05 for Advertising and r = .20, p = .03 for Conservatism).

⁴Although this might appear to simply be a confirmation of the research null, it is important to note that low statistical power would have favored missing any sleeper effect at all. Yet, in fact, lower NF errors in the warning condition increased at a statistically significant level over time for both essay topics. The interactions that would have qualified these effects based on claim placement, however, were far from significant with Fs < 1. Lack of power remains a potential concern, especially for detecting small but incomplete attenuation of a sleeper effect. Nevertheless, the significant NF increases after warnings (especially when failure to detect NF increases over time would have supported claims for sustained benefits of such cautions) suggests that their poor long-term benefit was not merely an artifact of low power.