How Should Scientific Journals Handle ‘Big If True’ Submissions?

Abstract

How can scientific journals satisfy an admirable desire for open-mindedness and aversion to censorship while minimizing the publication of junk science? We consider this question in the context of the Bem (2011) paper reporting extra-sensory perception among Cornell students, which received a lot of attention in part because the editors made the decision to publish the article despite extreme skepticism of its claims. We consider the reasons, good and bad, for journals to publish such papers, and then we propose an alternative way in which the journal could publish without seeming to endorse outlandish claims. Our proposal is to flip the standard scheme of scientific publication by privileging data rather than strong conclusions presented with an air of certainty. This proposal could work for the publication of "big if true" claims more generally.

Additional information

Notes on contributors

Andrew Gelman

Andrew Gelman is a professor of statistics and political science and director of the Columbia University Applied Statistics Center. He has received many awards, including the Outstanding Statistical Application Award from the American Statistical Association and the award for best article published in the American Political Science Review. He has coauthored many books; his most recent is Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do. He is editor of the Ethics and Statistics column.