Statistical Guidance to Authors at Top-Ranked Journals across Scientific Disciplines

Figures & data

Fig. 1 Frequency of journals offering some statistical guidance (circles) or having a dedicated statistical guidance section (diamonds) in each scientific discipline (bars not highlighted, n = 15 journals per discipline) and each higher-level scientific domain (bars highlighted; Health & Life Sciences: n = 165 journals; Multidisciplinary: n = 15; Social Sciences: n = 30; Physical Sciences: n = 90; Formal Sciences, n = 30). Data is presented in descending order of frequency first by domain, then by discipline within domains.

Fig. 2 Frequency of journals offering guidance on 20 prespecified statistical topics. Data is stratified by scientific domain (Health & Life Sciences: n = 165 journals; Multidisciplinary: n = 15; Social Sciences: n = 30; Physical Sciences: n = 90); the Formal Sciences are not shown because no journals provided statistical guidance. Data is presented in descending order of overall frequency across all domains. CCV = categorizing continuous variables.

Table 1 Number of journals offering various levels of endorsement for six “hotly debated” statistical topics.

Topic	Explicit	Implicit	Implicit	Explicit
	endorsement	endorsement	opposition	opposition
Confidence intervals	85	4	1^#	0
Sample size planning/	67	5	0	0
justification
Effect sizes	62	4	0	0
p-values	10	77	0	1*
Statistical significance	9	35	5	3
Bayesian statistics	0	39	0	0

Explicit endorsement = the journal advises or instructs authors to use this method whenever possible/appropriate. Implicit endorsement = the journal provides advice on the method, implying endorsement, but does not explicitly advise that the method should be used. Implicit opposition = the journal advises that they would prefer the method is not used, but does not explicitly rule it out. Explicit opposition = the journal advises that the method should not be used.

NOTE: Some opposition was caveated as indicated by the symbols below.

^# Confidence intervals are implicitly opposed by this journal only in the context of small sample sizes.

* p-values are opposed by this journal only when there were no prespecified multiplicity corrections.

Data Availability Statement

All data, materials, and analysis scripts related to this study are publicly available on the Open Science Framework repository (https://dx.doi.org/10.17605/OSF.IO/JRX7D). To facilitate reproducibility this manuscript was written by interleaving regular prose and analysis code using knitr (Xie 2017) and papaja (Aust and Barth 2020), and is available in a Code Ocean container (https://doi.org/10.24433/CO.6131540.v1) which recreates the software environment in which the original analyses were performed.