A snapshot look at replication and statistical reporting practices in psychology journals

ABSTRACT

Current efforts started in 2012 by the Association for Psychological Science (APS) appear to be different from previous arguments against null hypothesis statistical testing (NHST), which remained largely rhetorical without specific actions for compliance by researchers in psychology. The APS advocacy involves specific promising implementation tactics. The present study examined the impact of those efforts on replication and statistical reporting practices in four psychology journals from 2011 and 2015. The results showed that amidst increased reporting of NHST statistics in 2015 compared to 2011 and an absence of power reporting in the behavioral journals, there was increased reporting of actual replications in Psychological Science, paradoxically surpassing Journal of the Experimental Analysis of Behavior, of CIs in all four journals, and of error bars on graphs in Cognition and Behavioural Processes. These trends suggest need for additional efforts at propagating the APS initiatives to ensure greater impact in the broader psychological community. Additionally, psychologists from all domains need to become advocates of best practices for sustainable impact.

KEYWORDS:

• Psychological science
• replication practices
• statistical reporting practices
• behavioral journals
• cognitive journals
• NHST
• estimation

Acknowledgments

We thank Arshiya Mariam for conducting some tallies on select issues of PS.

Disclosure statement

No potential conflict of interest was reported by the authors.

Author Contributions

AI completed tallies on JEAB and PS and MF completed tallies on COG and BP. AI and MF jointly prepared earliest drafts of the paper; AI performed tally checks, conducted additional analyses, and developed themes in the introduction and discussion.

Ethical Considerations

This study involved no human participants or animals and therefore did not require an Institutional Review Board approval.

Supplementary Material

Supplemental data for this article can be accessed here.