Opinionated Practices for Teaching Reproducibility: Motivation, Guided Instruction and Practice

Figures & data

Fig. 1 Infamous comic from https://xkcd.com that highlights the difficulty of learning and using the version control software Git.

Fig. 2 LaTeX errors are often cryptic to new learners.

Table 1 Case studies that illustrate the significant real world consequences of reproducibility failures in data analyses.

Reproducibility error	Consequence	Source(s)
Limitations in Excel data formats	Loss of 16,000 COVID case records in the UK	(Kelion 2020)
Automatic formatting in Excel	Important genes disregarded in scientific studies	(Zeeberg et al. 2004; Ziemann, Eren, and El-Osta 2016)
Deletion of a cell caused rows to shift	Mix-up of which patient group received the treatment	(Wallensteen et al. 2018)
Using binary instead of explanatory labels	Mix-up of the intervention with the control group	(Aboumatar and Wise 2019)
Using the same notation for missing data and zero values	Paper retraction	(Whitehouse et al. 2021)
Incorrectly copying data in a spreadsheet	Delay in the opening of a hospital	(Picken 2020)

Kelion, L. (2020), Excel: Why using Microsoft’s Tool caused Covid-19 Results to be Lost,” BBC News. Available at https://www.bbc.com/news/technology-54423988.

 Google Scholar

Zeeberg, B. R., Riss, J., Kane, D. W., Bussey, K. J., Uchio, E., Linehan, W. M., et al. (2004), “Mistaken Identifiers: Gene Name Errors can be Introduced Inadvertently When Using Excel in Bioinformatics,” BMC Bioinformatics, 5, 80.

 PubMed Web of Science ®Google Scholar

Ziemann, M., Eren, Y., and El-Osta, A. (2016), “Gene Name Errors are Widespread in the Scientific Literature,” Genome biology, 17, 1–3.

 PubMed Web of Science ®Google Scholar

Wallensteen, L., Zimmermann, M., Sandberg, M. T., Gezelius, A., Nordenström, A., Hirvikoski, T., Lajic, S. (2018), “Retraction notice to ‘Evaluation of Behavioral Problems after Prenatal Dexamethasone Treatment in Swedish Adolescents at Risk of cah’ [Hormones and Behavior 85c (2016) 5–11],” Hormones and Behavior, 103, 140.

 PubMed Web of Science ®Google Scholar

Aboumatar, H., and Wise, R. A. (2019), “Notice of Retraction. Aboumatar et al. Effect of a Program Combining Transitional Care and Long-term Self-management Support on Outcomes of Hospitalized Patients With Chronic Obstructive Pulmonary Disease: A Randomized Clinical Trial. JAMA. 2018;320(22):2335–2343,” JAMA, 322, 1417–1418.

 PubMed Web of Science ®Google Scholar

Whitehouse, H., François, P., Savage, P. E., Currie, T. E., Feeney, K. C., Cioni, E., Purcell, R., Ross, R. M. Larson, J., Baines, J., ter Haar, B., Covey, A., Turchin, P. (2021), “Retraction Note: Complex Societies Precede Moralizing Gods Throughout World History,” Nature, 595, 320–320. DOI: 10.1038/s41586-021-03656-3.

 PubMed Web of Science ®Google Scholar

Picken, A. (2020), Spreadsheet Error led to Edinburgh Hospital Opening Delay, BBC News.

 Google Scholar