Reproducible research: a minority opinion

References

• AAAS. (2011, February). AAAS annual meeting: Symposium on the digitization of science: Reproducibility and interdisciplinary knowledge transfer. Washington, DC.
   Google Scholar
• AMP. (2011). Applied mathematics perspectives workshop on reproducible research: Tools and strategies for scientific computing applied mathematics perspectives. Vancouver, BC.
   Google Scholar
• Asuncion, A., & Newman, D. J. (2007). UCI machine learning repository (Technical report). Irvine, CA: University of California, School of Information and Computer Science.
   Google Scholar
• Baker, M. (2015, August). Over half of psychology studies fail reproducibility test. Nature. doi:10.1038/nature.2015.18248
   Google Scholar
• Bissell, M. (2013, November). Reproducibility: The risks of the replication drive. Nature, 503, 333–334.
   PubMed Web of Science ®Google Scholar
• Bridgman, P. (1986). Reflections of a physicist. Oxford: Oxford Science Publications, Clarendon Press.
   Google Scholar
• Broad, W., & Wade, N. (1984). Betrayers of the truth. New York: Random House.
   Google Scholar
• Broman, K., Cetinkaya-Rundel, M., Nussbaum, A., Paciorek, C., Peng, R., Turek, D., & Wickham, H. (2017, January). Recommendations to funding agencies for supporting reproducible research. American Statistical Association. Retrieved from http://www.amstat.org/asa/files/pdfs/pol-reproducibleresearchrecommendations.pdf
   Google Scholar
• Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., ... Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433–1436.
   PubMed Web of Science ®Google Scholar
• Crocker, J., & Cooper, M. L. (2011). Editorial: Addressing scientific fraud. Science, 334, 1182.
   PubMed Web of Science ®Google Scholar
• CST. (1993, June). Coffee ‘not factor’ in bladder cancer. Chicago Sun-Times.
   Google Scholar
• Drummond, C. (2008). Finding a balance between anarchy and orthodoxy. In Proceedings of the Twenty-Fifth International Conference on Machine Learning: Workshop on Evaluation Methods for Machine Learning III (4 pages), Helsinki.
   Google Scholar
• Drummond, C. (2009). Replicability is not reproducibility: Nor is it good science. In Proceedings of the Twenty-Sixth International Conference on Machine Learning: Workshop on Evaluation Methods for Machine Learning IV (4 pages), Montreal.
   Google Scholar
• Drummond, C., & Holte, R. C. (2006). Cost curves: An improved method for visualizing classifier performance. Machine Learning, 65(1), 95–130.
   Web of Science ®Google Scholar
• Drummond, C., & Japkowicz, N. (2010). Warning: Statistical benchmarking is addictive kicking the habit in machine learning. Journal of Experimental and Theoretical Artificial Intelligence, 22, 67–80.
   Web of Science ®Google Scholar
• Economist Editorial. (2011, September). An array of errors, investigations into a case of alleged scientific misconduct have revealed numerous holes in the oversight of science and scientific publishing. The Economist. Retrieved from http://www.economist.com/node/21528593.
   Google Scholar
• Economist Editorial. (2013, October). Unreliable research: Trouble at the lab. The Economist. Retrieved from https://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble
   Google Scholar
• ENAR. (2011, March). Research ethics in biostatistics: Invited panel discussion at ENAR 2011 on the biostatistician’s role in reproducible research. Miami, FL.
   Google Scholar
• Feyerabend, P. (1970). Against method: Outline of an anarchistic theory of knowledge. Atlantic Highlands, NJ: Humanities Press.
   Google Scholar
• Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh: Oliver and Boyd.
   Google Scholar
• Fritzson, P., Aronsson, P., Lundvall, H., Nyström, K., Pop, A., Saldamli, L., & Broman, D. (2005). The Open Modelica modeling, simulation, and development environment. Proceedings of the 46th Conference on Simulation and Modelling of the Scandinavian Simulation Society Trondheim, Trondheim, Norway.
   Google Scholar
• Groves, T., & Godlee, F. (2012). Open science and reproducible research. Editorial BMJ, 344, e4383. Retrived from http://www.bmj.com/content/344/bmj.e4383
   Google Scholar
• Guardian. (2011, July). Scientific fraud in the UK: The time has come for regulation. Manchester Guardian.
   Google Scholar
• Hamilton, D. P. (1990). Publishing by -- and for? -- the numbers. Science, 250, 1331–1332.
   PubMed Web of Science ®Google Scholar
• Holton, G. (1986). The advancement of science, and its burdens: The Jefferson lecture and other essays. Cambridge: Cambridge University.
   Google Scholar
• Iorns, E. (2013, October). Reproducibility initiative receives \$1.3m grant to validate 50 landmark cancer studies. Online at Center for Open Science. Retrieved from https://cos.io/about/news/reproducibility-initiative-receives
   Google Scholar
• Jasny, B. R., Chin, G., Chong, L., & Vignieri, S. (2011). Introduction: Special issue on data replication and reproducibility. Science, 334(6060), 1225.
   PubMed Web of Science ®Google Scholar
• Johns Hopkins University. (2015). Reproducible research. Online Course. Retrieved from https://www.coursera.org/course/repdata
   Google Scholar
• Kazai, G., & Hanbury, A. (Eds.). (2015). European Conference on Information Retrieval: Special Track. Vienna, Austria.
   Google Scholar
• Kenett, R. S., & Shmueli, G. (2015). Clarifying the terminology that describes scientific reproducibility. Nature Methods, 12(699).
   PubMedGoogle Scholar
• Kibler, D., & Langley, P. (1988). Machine learning as an experimental science. In Proceedings of the Third European Working Session on Learning, 81–92.
   Google Scholar
• Knuth, D. (1988). Literate programming. The Computer Journal, 27, 97–111.
   Google Scholar
• Kuhn, T. (1962). The structure of scientific revolutions. University of Chicago.
   Google Scholar
• Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141.
   Google Scholar
• Moska, S. (Ed.). (2015). Worshop on Reproducible Research at Computational and Simulation Sciences and eResearch Conference.
   Google Scholar
• Nature. (2015, October). Challenges in irreproducible research. Nature Special Issue.
   Google Scholar
• Nature Editorial. (2016, May). Reality check on reproducibility. Nature, 533(437).
   Google Scholar
• Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society London, 231, 289–337.
   Google Scholar
• NIH. (2016, April). Rigor and reproducibility. Retrieved from https://grants.nih.gov/reproducibility/index.htm
   Google Scholar
• NSF. (2010). National science foundation workshop on changing the conduct of science in the information age summary.
   Google Scholar
• NYT. (1981, March). Study links coffee use to pancreas cancer. New York Times.
   Google Scholar
• NYT. (1989, May). Physicists debunk claim of a new kind of fusion. New York Times.
   Google Scholar
• NYT. (2011, July). How bright promise in cancer testing fell apart. New York Times.
   Google Scholar
• NYT. (2012, January). University suspects fraud by a researcher who studied red wine. New York Times.
   Google Scholar
• Open Society Institute. (2001). Budapest open access initiative.
   Google Scholar
• Oreskes, N. (2004). Beyond the ivory tower: The scientific consensus on climate change. Science, 306(5702), 1686.
   PubMed Web of Science ®Google Scholar
• Peng, R. D. (2011). Reproducible research in computational science. Science, 334(6060), 1226–1227.
   PubMed Web of Science ®Google Scholar
• Polanyi, M. (1958). Personal knowledge: Towards a post-critical philosophy. Routledge.
   Google Scholar
• Popper, K. R. (1968). The logic of scientific discovery. New York: Harper & Row.
   Google Scholar
• R Development Core Team. (2008). R: A language and environment for statistical computing. Retrieved from http://www.R-project.org
   Google Scholar
• Ryan, M. J. (2011). Replication in field biology: The case of the frog-eating bat. Science, 334(6060), 1229–1230.
   PubMed Web of Science ®Google Scholar
• Sanders, J. (Ed.). (2014). Royal society open science. Retrieved from http://rsos.royalsocietypublishing.org/
   Google Scholar
• Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013, October 24). Editorial: Ten simple rules for reproducible computational research. PLOS Computational Biology.
   PubMed Web of Science ®Google Scholar
• Schulte, E., Davison Dye, T., & Dominik, C. (2012). A multi-language computing environment for literate programming and reproducible research. Journal of Statistical Software, 46(3).
   Web of Science ®Google Scholar
• Services, A. W. (2016). AWS public data sets. Retrieved from http://aws.amazon.com/datasets/
   Google Scholar
• SIAM-CSE. (2011). SIAM conference on computational science \ & engineering workshop on verifiable, reproducible computational science.
   Google Scholar
• SIAM-Geo. (2011). SIAM geosciences workshop on reproducible science and open-source software in the geosciences.
   Google Scholar
• Sonnenburg, S., Braun, M. L., Ong, C. S., Bengio, S., Bottou, L., Holmes, G., & Williamson, R. (2007). The need for open source software in machine learning. JMLR, 2443–2466.
   Google Scholar
• ST. (2006). Cancer chemical found in coffee. Sunday Times.
   Google Scholar
• Stallman, R. M. (1981). Emacs: The extensible, customizable, self-documenting display editor (Technical Report AIM-519A, MIT Artificial Intelligence Laboratory), Cambridge Massachusetts
   Google Scholar
• Stodden, V., Leisch, F., & Peng, R. D. (Eds.). (2014). Implementing reproducible research. The R Series. Chapman & Hall/CRC.
   Google Scholar
• Stodden, V. C. (2010). Reproducible research: Addressing the need for data and code sharing in computational science Yale law school roundtable on data and code sharing. Computing in Science & Engineering, 12(5), 8–13.
   Web of Science ®Google Scholar
• Swiss Institute of BioInformatics. (2015, January). Reproducible research. Retrieved from http://edu.ch.embnet.org/course/view.php?id=193
   Google Scholar
• TCL. (2010, August). Duke finds ‘issues of substantial concern’ and sanctions Potti. The Cancer Letter.
   Google Scholar
• USAT. (2010, November). Can coffee, tea lower brain cancer risk? USA Today.
   Google Scholar
• Vanschoren, J., Blockeel, H., Pfahringer, B., & Holmes, G. (2012). Experiment databases. A new way to share, organize and learn from experiments. Machine Learning, 87(2), 127–158.
   Web of Science ®Google Scholar
• Weinberg, S. (1995). The methods of science and those by which we live. Academic Questions, 8(2), 7–13.
   Google Scholar
• Wren, K. (2014, May). As concerns about non-reproducible data mount, some solutions take shape. AAAS News.
   Google Scholar