How open is communication science? Open-science principles in the field

ABSTRACT

Open-science principles, such as open data, shared materials, and pre-registration, are expected to encourage a culture of replication in scientific research. Yet, with its topical and methodological heterogeneity, communication science has been described to fall short of such principles. We analyze the extent to which open-science principles were used in publications from 20 leading communication journals between 2010 and 2020, and compare the results to benchmarks from psychology. Results show that open-science principles were little used in communication science with some variation across methods, but were more consistently used in psychology papers. There was no relationship with scientific impact. This suggests a need for greater attention to open-science principles in communication science, while considering their appropriateness for different study designs.

KEYWORDS:

• Open data
• open materials
• pre-registration
• experiment
• content analysis
• meta-academic research
• computational communication science

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Pre-registration is considered an open-science practice as it forces scholars to a-priori label their research as exploratory or confirmatory. Moreover, it helps to reduce the leeway for HARKing and p-hacking, contributes to less underpowered studies, and might help to reduce the significance of publication bias and the file-drawer effect (Dienlin et al., 2020).

2 Communication Monographs; Communication Research; Communication Theory; Cyberpsychology, Behavior, and Social Networking; Human Communication Research; Information Communication & Society; International Journal of Advertising; International Journal of Press/Politics; Journal of Advertising; Journal of Communication; Journal of Computer-Mediated Communication; Journal of Health Communication; Management Communication Quarterly; Media Psychology; New Media & Society; Political Communication; Public Opinion Quarterly; Public Understanding of Science; Research on Language and Social Interaction; Science Communication.

3 Journal of Experimental Psychology: Learning, Memory, and Cognition; Journal of Personality and Social Psychology; Psychological Science.

4 Based on their submission guidelines all journals encourage open science practices. The Journal of Personality and Social Psychology and the Journal of Experimental Psychology implemented mandatory open science requirements after our sampling period (1 July 2021 for Journal of Personality and Social Psychology and 1 August 2022 for Journal of Experimental Psychology). As such, these policy changes are not likely to affect our analysis. Public Opinion Quarterly, however, implemented their new open science policy in mid-2019 and estimated first articles following the guidelines to be published in mid-2020 (see Plutzer, 2019). As our sampling ended in August 2020, we do not expect the new policy of POQ to substantially influence the analysis.