How well are open science practices implemented in industrial and organizational psychology and management?

ABSTRACT

To address the low reproducibility and replicability of research, Open Science Practices (OSPs) have been developed. Yet, despite increasing awareness of their potential benefits, there has been only little implementation. As journals can act as gatekeepers for scientific discoveries, a potential tendency not to mention OSPs on their websites may help to explain this implementation gap. Therefore, we examined the implementation of OSPs and potential barriers in industrial and organizational psychology and management (IOP/management) journals. Study 1 examined whether and how N = 257 journal websites referred to OSPs. We found that most journals did not mention OSPs. Specifically, only two (1.0%), five (2.5%), and 14 (6.9%) IOP/management journals mentioned preregistration, registered reports, and explicitly welcomed replications, respectively. Study 2 investigated perceived barriers to implementing OSPs with a survey among editors of the IOP/management journals from Study 1. Among the 40 responding editors, 14, 10, and five attributed the lacking implementation of OSPs to a lesser suitability of OSPs for qualitative research, missing authority, and missing familiarity with OSPs, respectively. Based on our findings, the implementation gap could be mitigated by developing new and refining extant OSPs, starting bottom-up initiatives (e.g., researchers directly contacting publishers), and increasing the availability of information on OSPs.

KEYWORDS:

• open science practices
• reproducibility
• replicability
• academic reward system

Disclosure statement

No potential conflict of interest was reported by the author(s).

Author contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Ann-Kathrin Torka, Jens Mazei and Joachim Hüffmeier. The first draft of the manuscript was written by Ann-Kathrin Torka and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/1359432X.2023.2206571.

Notes

1. In the online supplement (Table S1), we present definitions of the individual OSPs and explain their potential benefits.

2. We preregistered to cover all OSPs, which would have also included Open Access (OA) publishing. However, as OA publishing does not necessarily reduce QRPs, we deviated from our preregistration and did not include OA publishing in our analysis.

3. COPE was established to educate and support editors and publishers of scholarly journals on ethical issues during the publication process (COPE, 2020).

4. The results of these analyses differ from those reported in the manuscript. However, the implementation rates nevertheless remain low. For instance, only 6% of IOP journals mentioned preregistration and only 13% offered RRs as a publication option. Moreover, 74% of all IOP journals did not mention replications on their websites.

5. A reviewer kindly suggested conducting additional analyses with the journals’ recent listing in the “Academic Journal Guide” (AJG) of the Chartered Association of Business Schools (CABS), because the journals’ impact factor may be contingent on sub-disciplines’ citation norms (e.g., because there are more journals and more citing researchers in some sub-disciplines). These additional analyses are provided in the online supplement (Table S3).

6. We contacted editors-in-chief instead of associate editors because we were interested in the journals’ communicated policies on their websites, not in the editorial practice (e.g., the review process, for which associate editors are often more directly responsible).

7. We would like to thank an anonymous reviewer for suggesting this analysis.

Additional information

Funding

No funding was received for conducting this study.