Registered Reports in the Journal of Sports Sciences

We published a recent editorial on sample size estimation (Abt et al., 2020), which was one in a series of editorials over many years intended to encourage authors to improve the rigour of studies submitted to the Journal of Sports Sciences (Nevill et al., 2004; Winter et al., 2014). We continue this encouragement now with the introduction of the Registered Report submission format.

As highlighted in our editorial (Abt et al., 2020), and by many others (Caldwell et al., 2020; Ioannidis et al., 2014; Munafò et al., 2017), there are a number of problems that plague modern science; publication bias, questionable research practices, and underpowered studies to name but a few. Jones and Brewer (1972) reported that for studies published in Research Quarterly (now Research Quarterly for Exercise and Sport) the calculated power for different effect sizes was “disturbingly low” – 0.15, 0.54, and 0.83 for small, medium, and large effect sizes, respectively. These findings have been largely confirmed by similar studies in 1977, 2000, and 2020 that explored the same issues across numerous journals (Christensen & Christensen, 1977; Kinney et al., 2020; Speed & Andersen, 2000). Collectively, these studies suggest that the power of a typical study published in our discipline has a high probability of detecting a large effect, a medium probability of detecting a medium effect, and a very low probability of detecting a small effect. As noted by Speed and Andersen (2000), small effects in the sport sciences are often associated with the difference between winning and losing. As such, it is potentially important to be able to detect important effects, however small. Yet, underpowered studies are just one problem that sport and exercise scientists need to deal with. Questionable research practices and publication bias have been reported across a range of scientific disciplines (Ioannidis et al., 2014), and we are not immune (Caldwell et al., 2020).

So, what are the solutions? A number of recommendations have been made over the last decade to improve the rigour of research and to minimise the introduction of these problems into the literature (Caldwell et al., 2020; Munafò et al., 2017). Two of these are the use of preregistration and the Registered Report submission format. Preregistration allows reviewers, editors, and readers to transparently evaluate the capacity of a test to falsify a prediction (Lakens, 2019). Preregistration (ideally a priori) involves publishing a timestamped protocol that outlines all data collection procedures together with a data analysis plan (Nosek et al., 2018). The data analysis plan usually involves identifying what are termed “confirmatory” and “exploratory” analyses (Caldwell et al., 2020). Publishing the data collection and analysis protocols prior to data collection allows authors to minimise the temptation to engage in questionable research practices (e.g., p-hacking, HARKing). Preregistration encourages greater transparency and rigorous reporting of study methods and analyses (Toth et al., 2020). Although we strongly support the use of preregistration, it is still open to abuse and manipulation (Ikeda et al., 2019; Yamada, 2018). The Registered Report format takes preregistration one step further by formalising the registration process and allowing reviewer feedback prior to data collection.

Unlike the traditional peer-review process, which involves peer-review after data collection, the Registered Report format involves peer-review both before and after data collection (Chambers, 2013; Impellizzeri et al., 2019). This two-stage process not only has the benefits of preregistration but also allows authors to receive feedback on the proposed methods and data analysis before data collection, when they have the ability to modify these features. With a post-data peer-review process it is too late to make changes to the method, refine the research question, and/or hypotheses. Moreover, the Registered Report format has the potential to reduce publication bias by allowing editors to gauge the value of a study without being biased by the results. In an exploratory analysis, Allen and Mehler (2019) reported that the number of null findings increased from ~5-20% with traditional peer-review to ~60% with the Registered Report format. In a similar vein, Scheel et al. (2021) reported that positive findings reduced from 96% in the standard literature to 44% in Registered Reports.

Although the Open Science Framework lists over 260 journals that offer the Registered Report format (Open Science Framework, n.d.), only a few of these are in the sport and exercise sciences, namely, Human Movement Science (Dessing & Beek, 2015), Science and Medicine in Football (Impellizzeri et al., 2019), Psychology of Sport and Exercise, and Registered Reports in Kinesiology. Like the other journals in our field that have implemented the Registered Report format, we encourage authors to embrace the methods of open science to improve the quality of their research. We hope authors will engage with this new submission type and we look forward to helping them through the two-stage process. Improving research practices is everyone’s responsibility – authors, journals, editors, and reviewers. At the Journal of Sports Sciences, we want to play our part in enabling researchers to do their best work, and we believe that the Registered Report format is a positive step towards that goal.

Additional information

Funding

No Funding is associated with this study.