Opinionated Practices for Teaching Reproducibility: Motivation, Guided Instruction and Practice

References

• Aboumatar, H., and Wise, R. A. (2019), “Notice of Retraction. Aboumatar et al. Effect of a Program Combining Transitional Care and Long-term Self-management Support on Outcomes of Hospitalized Patients With Chronic Obstructive Pulmonary Disease: A Randomized Clinical Trial. JAMA. 2018;320(22):2335–2343,” JAMA, 322, 1417–1418.
   PubMed Web of Science ®Google Scholar
• Adriaanse, M. A., van Oosten, J. M. F., de Ridder, D. T. D., de Wit, J. B. F., and Evers, C. (2011), “Planning What not to Eat: Ironic Effects of Implementation Intentions Negating Unhealthy Habits,” Personality & Social Psychology Bulletin, 37, 69–81.
   PubMed Web of Science ®Google Scholar
• Allaire, J., Xie, Y., McPherson, J., Luraschi, J., Ushey, K., Atkins, A., Wickham, H., Cheng, J., Chang, W., Iannone, R. (2021), “rmarkdown: Dynamic Documents for R [Computer software manual],” available at https://github.com/rstudio/rmarkdown (R package version 2.8).
   Google Scholar
• Anaconda. (2020), Anaconda Software Distribution, Anaconda Inc. Available at https://docs.anaconda.com/.
   Google Scholar
• Baggerly, K. A., and Coombes, K. R. (2009), “Deriving Chemosensitivity from Cell Lines: Forensic Bioinformatics and Reproducible Research in High-throughput Biology,” The Annals of Applied Statistics, 3, 1309–1334.
   Web of Science ®Google Scholar
• Bailey, D. H., and Borwein, J. J. (2013). The Reinhart-Rogoff Error – or How Not to Excel at Economics. Available at http://theconversation.com/the-reinhart-rogoff-error-or-how-not-to-excel-at-economics-13646.
   Google Scholar
• Bonney, K. M. (2015), “Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains,” Journal of Microbiology & Biology Education, 16, 21–28.
   PubMed Web of Science ®Google Scholar
• Bryan, J., Attali, D., Konrad, B., Jackman, S., and Hester, J. (n.d.). Chapter 36 Burn It All down — Happy Git and GitHub for the useR, Available at https://happygitwithr.com/burn.html.
   Google Scholar
• Carlson, B. (2012), “Putting Oncology Patients at Risk,” Biotechnology Healthcare, 9, 17–21.
   PubMedGoogle Scholar
• Carlson, J. A., and Schodt, D. W. (1995), “Beyond the Lecture: Case Teaching and the Learning of Economic Theory,” The Journal of Economic Education, 26, 17–28. DOI: 10.1080/00220485.1995.10844853.
   Web of Science ®Google Scholar
• The Carpentries. (n.d.), https://carpentries.org/index.html.
   Google Scholar
• Chamberlain, S., and Dobbyn, A. (2020), “cowsay: Messages, Warnings, Strings with ASCII Animals [Computer Software Manual],” https://CRAN.R-project.org/package=cowsay(Rpackageversion0.8.0)
   Google Scholar
• Committee on Reproducibility and Replicability in Science; Board on Behavioral, Cognitive, and Sensory Sciences; Committee on National Statistics; Division of Behavioral and Social Sciences and Education; Nuclear and Radiation Studies Board; Division on Earth and Life Studies; Board on Mathematical Sciences and Analytics; Committee on Applied and Theoretical Statistics; Division on Engineering and Physical Sciences; Board on Research Data and Information; Committee on Science, Engineering, Medicine, and Public Policy; Policy and Global Affairs; National Academies of Sciences, Engineering, and Medicine (2019), Reproducibility and Replicability in Science, Washington, D.C.: National Academies Press.
   Google Scholar
• Day, R. (2005), “The Instructional Skills Workshop: The Heart of an Educator Learning Community in British Columbia and Beyond, International Society for the Scholarship of Teaching & Learning.
   Google Scholar
• Ebbinghaus, H. (1913), “Memory: A Contribution to Experimental Psychology,” New York: Teachers College. (Original work published 1885), 39.
   Google Scholar
• Flynn, A. E., and Klein, J. D. (2001), “The Influence of Discussion Groups in a Case-Based Learning Environment,” Educational Technology Research and Development, 49, 71–86.
   Web of Science ®Google Scholar
• Fournier, M., d’Arripe-Longueville, F., Rovere, C., Easthope, C. S., Schwabe, L., El Methni, J., Radel, R. (2017), “Effects of Circadian Cortisol on the Development of a Health Habit,” Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association, 36, 1059–1064. DOI: 10.1037/hea0000510.
   PubMed Web of Science ®Google Scholar
• Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., Wenderoth, M. P. (2014), “Active Learning Increases Student Performance in Science, Engineering, and Mathematics,” Proceedings of the National Academy of Sciences, 111, 8410–8415.
   PubMed Web of Science ®Google Scholar
• Gardner, B., and Rebar, A. L. (2019), “Habit Formation and Behavior Change,” in Oxford Research Encyclopedia of Psychology, Oxford: Oxford University Press.
   Google Scholar
• Garvin, D. A. (2003), Making the Case, Available at https://www.harvardmagazine.com/2003/09/making-the-case-html.
   Google Scholar
• GitHub. (2020), Set the Default Branch for Newly-Created Repositories. Available at https://github.blog/changelog/2020-08-26-set-the-default-branch-for-newly-created-repositories/.
   Google Scholar
• Herndon, T., Ash, M., and Pollin, R. (2013), “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff,” Cambridge Journal of Economics, 38, 257–279.
   Web of Science ®Google Scholar
• Jupyter, P., Blank, D., Bourgin, D., Brown, A., Bussonnier, M., Frederic, J., Granger, B., Griffiths, T. L., Hamrick, J., Kelley, K., Pacer, M., Page, M., Pérez, F., Ragan-Kelley, B., Suchow, J. W., Willing, C. (2019), “Nbgrader: A Tool for Creating and Grading Assignments in the Jupyter Notebook,” Journal of Open Source Education, 2, 32. DOI: 10.21105/jose.00032.
   Google Scholar
• Kelion, L. (2020), Excel: Why using Microsoft’s Tool caused Covid-19 Results to be Lost,” BBC News. Available at https://www.bbc.com/news/technology-54423988.
   Google Scholar
• Kluyver, T., Ragan-Kelley, B., Pérez, F., Granger, B. E., Bussonnier, M., Frederic, J., et al. (2016), Jupyter Notebooks-a Publishing Format for Reproducible Computational Workflows (Vol. 2016), Amsterdam, Netherlands: IOS Press.
   Google Scholar
• Knuth, D. E. (1984), “Literate Programming,” The Computer Journal, 27, 97–111. DOI: 10.1093/comjnl/27.2.97.
   Web of Science ®Google Scholar
• Lally, P., Jaarsveld, C. H. M. van, Potts, H. W. W., and Wardle, J. (2010), “How are Habits Formed: Modelling Habit Formation in the Real World,” European Journal of Social Psychology, 40, 998–1009. DOI: 10.1002/ejsp.674.
   Web of Science ®Google Scholar
• Lowndes, J. S. S., Best, B. D., Scarborough, C., Afflerbach, J. C., Frazier, M. R., O’Hara, C. C., Jiang, N., Halpern, B. S. (2017), Our Path to Better Science in Less Time Using Open Data Science Tools,” Nature Ecology & Evolution, 1, 1–7.
   PubMed Web of Science ®Google Scholar
• Lyman, F. (1981), “The Responsive Classroom Discussion: The Inclusion of All Students,” Mainstreaming Digest, 109–113.
   Google Scholar
• Merkel, D. (2014), “Docker: Lightweight Linux Containers for Consistent Development and Deployment,” Linux Journal, 2014, 2.
   Google Scholar
• National Academies of Sciences, Engineering, and Medicine. (2019), Reproducibility and Replicability in Science, Washington, DC: National Academies Press.
   Google Scholar
• Nederbragt, A., Harris, R. M., Hill, A. P., and Wilson, G. (2020), “Ten Quick Tips for Teaching with Participatory Live Coding,” PLOS Computational Biology, 16, e1008090. DOI: 10.1371/journal.pcbi.1008090.
   PubMed Web of Science ®Google Scholar
• Parker, H. (2017), “Opinionated Analysis Development,” PeerJ Preprints, 5, e3210v1.
   Google Scholar
• Peng, R. D., and Matsui. (2020), Essays on Data Analysis, Victoria, British Columbia, Canada: LeanPub.
   Google Scholar
• Picken, A. (2020), Spreadsheet Error led to Edinburgh Hospital Opening Delay, BBC News.
   Google Scholar
• Pyles, C., and Program, U. B. D. S. E. (2021), Otter-grader: A python and r autograding solution. (If you use this software, please cite it as below.)
   Google Scholar
• Raj, A. G. S., Patel, J. M., Halverson, R., and Halverson, E. R. (2018), “Role of Live-Coding in Learning Introductory Programming,” in Proceedings of the 18th Koli Calling International Conference on Computing Education Research (pp. 1–8).
   Google Scholar
• Ram, K. (2013), “Git Can Facilitate Greater Reproducibility and Increased Transparency in Science,” Source code for Biology and Medicine, 8, 1–8.
   PubMedGoogle Scholar
• RStudio Education. (n.d.), https://education.rstudio.com/.
   Google Scholar
• Sandve, G. K., Nekrutenko, A., Taylor, J., and Hovig, E. (2013), “Ten Simple Rules for Reproducible Computational Research,” PLOS Computational Biology, 9, e1003285. DOI: 10.1371/journal.pcbi.1003285.
   PubMed Web of Science ®Google Scholar
• Timbers, T.-A., Campbell, T., and Lee, M. (2022), Data Science: An Introduction (1st ed.), Boca Raton, FL: CRC Press. Available at https://datasciencebook.ca/.
   Google Scholar
• Ushey, K. (2021), renv: Project environments [Computer software manual]. Available at https://CRAN.R-project.org/package=renv (R package version 0.14.0)
   Google Scholar
• Wallensteen, L., Zimmermann, M., Sandberg, M. T., Gezelius, A., Nordenström, A., Hirvikoski, T., Lajic, S. (2018), “Retraction notice to ‘Evaluation of Behavioral Problems after Prenatal Dexamethasone Treatment in Swedish Adolescents at Risk of cah’ [Hormones and Behavior 85c (2016) 5–11],” Hormones and Behavior, 103, 140.
   PubMed Web of Science ®Google Scholar
• Whitehouse, H., François, P., Savage, P. E., Currie, T. E., Feeney, K. C., Cioni, E., Purcell, R., Ross, R. M. Larson, J., Baines, J., ter Haar, B., Covey, A., Turchin, P. (2021), “Retraction Note: Complex Societies Precede Moralizing Gods Throughout World History,” Nature, 595, 320–320. DOI: 10.1038/s41586-021-03656-3.
   PubMed Web of Science ®Google Scholar
• Wilson, G. (2018), Teaching Tech Together: How to Design and Deliver Lessons That Work and Build a Teaching Community around Them, Leipzig: Amazon Distribution GmbH.
   Google Scholar
• Wilson, G., Bryan, J., Cranston, K., Kitzes, J., Nederbragt, L., and Teal, T. K. (2017), “Good Enough Practices in Scientific Computing,” PLOS Computational Biology, 13, e1005510. DOI: 10.1371/journal.pcbi.1005510.
   PubMed Web of Science ®Google Scholar
• Yadav, A., Lundeberg, M., DeSchryver, M., Dirkin, K., Schiller, N. A., Maier, K., et al. (2007), “Teaching Science with Case Studies: A National Survey of Faculty Perceptions of the Benefits and Challenges of Using Cases,” Journal of College Science Teaching, 37, 34–38.
   Google Scholar
• Zeeberg, B. R., Riss, J., Kane, D. W., Bussey, K. J., Uchio, E., Linehan, W. M., et al. (2004), “Mistaken Identifiers: Gene Name Errors can be Introduced Inadvertently When Using Excel in Bioinformatics,” BMC Bioinformatics, 5, 80.
   PubMed Web of Science ®Google Scholar
• Ziemann, M., Eren, Y., and El-Osta, A. (2016), “Gene Name Errors are Widespread in the Scientific Literature,” Genome biology, 17, 1–3.
   PubMed Web of Science ®Google Scholar