Recommendations to medical journals on ways to encourage statistical experts to review submissions

In recent years, there has been a decline in the quality of statistical analyses conducted on various fields of medicine in the submitted articles¹. According to Zapf et al., biostatisticians take part in the preparation of appropriate and effective study designs and monitor their course. They are responsible for planning, conducting data analysis, and interpreting and publishing the results². The latest published results show that only 39% of the 2600 accepted articles on various aspects of COVID-19 are statistically correct, which unfortunately may jeopardize the credibility of biomedical research³. A 1998 study by Goodman et al. found that 33% of 114 biomedical journals surveyed used a statistical review for all original articles, and 46% used a statistical review at the editor's discretion⁴. Unfortunately, no significant changes took place over the next 20 years. In an article from 2020, published in PLOS ONE, the authors stated that only 34% (36/107) of journal editors rarely or never use a specialized statistical review. Furthermore, 55% of the surveyed respondents said they paid for a statistical reviewer⁵, however, according to Goodman and colleagues, it was about one-third, which is much less⁴. The conducted research is limited in terms of the sparse number of respondents. Due to the low number of statistical reviewers, rewarding this group of people is often necessary. However, these types of people are employed primarily by more prestigious magazines that have the financial resources for this specific purpose. For this reason, it seems advisable to publish recommendations for medical journals regarding ways to find and encourage statistical reviewers to conduct an expert review of the submitted manuscript regarding the performed statistical analysis. Otherwise, there is a high probability that the situation will not change in the near future.

The first advised recommendation is to provide reviewers vouchers for reviews conducted for open access journals. A financial coupon could be provided for the review, which could then be used on an approved article. The processing fee for an article (APC) in an open-access journal would be reduced by the value of the voucher. If you are reviewing more than one article, you could use them together. Magazines and journals would also need to set a fair limit of monthly reviews per reviewer to prevent you from appearing as though you want to collect as many vouchers as possible. Many scientists do not have university funding sources that would allow them to pay for an approved article. For this reason, obtaining this type of voucher would be a factor motivating statistical reviewers to conduct expert reviews.

The next recommendation is that journals consider paying even a small sum for conducting a single statistical review. This applies to several types of necessary situations, such as obtaining ambiguous reviews or the editorial office having serious doubts about the quality of the statistical analysis performed. Providing statistical reviewers with even a small payment for an expert review would increase the quality of statistical analyses conducted in medical journals. As time passes, more editors would already have a statistical reviewer on the editorial board of their journal.

Another recommendation has to do with the academic career of statistical reviewers. They undergo periodic academic evaluations at their universities, covering research, teaching and organizational activities. In order to encourage reviewers to conduct regular statistical reviews, editors of medical journals could, at the request of an interested reviewer, issue an appropriate certificate containing not only the number of articles reviewed but also a detailed description of the work performed by the reviewer. This applies to aspects pertaining to the subject of reviewed articles, the number of reviews and their quality, including a detailed description of the reliability of the work performed. Such opinions about the work performed by the reviewer could be additionally used, for example, when applying for a higher academic degree. The certificates issued so far only contain the number of reviewed articles, and providing the reviewer with a slightly more in-depth description would allow for a more reliable depiction of their additional scientific activity. This, in turn, simplifies the decision-making process for scientific councils related to the scientific career of a specific person. Appreciation for statistical reviewers could be given on the Publons platform, enabling scientists to discuss and share reviews and edit achievements and research. Publons also allows you to archive written reviews and be noticed and invited by publishers to review articles.

Another recommendation concerns including statistical reviews of people starting their scientific careers, i.e. those who have completed basic classes in biostatistics. According to the authors of PLOS ONE, statistical reviewers do not necessarily have to be statisticians with a PhD degree. They point out that most statistical reviewers have received doctoral training in a quantitative discipline, which may include areas such as statistics, epidemiology, computer science, health research, and economics⁵. Encouraging young people with a PhD to conduct this type of review could reduce the current problem faced by the editors of many medical journals. They could first send the journal editorial staff proof of completed training in the discussed area. Thanks to this activity, the young scientific community would become even more familiar with the current scientific research and gain statistical experience. In the future, new members of the editorial board of journals, based on reviews conducted for many years, would no longer experience problems with finding a statistical reviewer. Including the young scientific generation trained in biostatistics in the course of, for example, doctoral studies could contribute to reducing the current problem many editors face in the future. One of the proposed recommendations could be to complete a course in biostatistics/medical statistics, i.e. after meeting an additional condition. It could consist in properly reviewing the articles submitted to several types of medical journals in terms of all aspects related to the statistical analysis conducted by the authors. An exemplary group of doctoral students, a month or two months before completing their course, would conduct a statistical review under the supervision of their course director at the request of the journal editor, thus confirming their knowledge acquired during the course. According to the results presented at the International Statistical Congress, research conducted on a group of 25,300 students indicated that it is a more interesting form of statistical knowledge compared to traditional examinations consisting in conducting individual statistical tests step by step, thus causing stress related to remembering long stages of conducting an analysis⁶.

To sum up, one should at least gradually try to implement the above-mentioned recommendations. Thanks to this type of action, the problem faced by editors at the moment, especially those of less prestigious journals, would be alleviated in the future. You have to start somewhere so that you can visualize the effect of applying these recommendations in a few years. Otherwise, the problem will, unfortunately, only intensify.

Transparency

Declaration of funding

This paper was not funded.

Declaration of financial/other relationships

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgements

None.