Teaching for Large-Scale Reproducibility Verification

References

• American Economic Association (2019), “Data and Code Availability Policy.” Available at https://www.aeaweb.org/journals/data/data-code-policy
   Google Scholar
• American Economic Association (2020), “Data and Code Availability Policy,” AEA Papers and Proceedings, 110, 776–78.
   Google Scholar
• Bell, M., and Miller, N. (2013), “How to Persuade Journals to Accept Your Replication Paper.” Available at https://politicalsciencereplication.wordpress.com/2013/09/11/guest-blog-how-to-persuade-journals-to-accept-yourreplication-paper/
   Google Scholar
• Bernanke, B. S. (2004), “Editorial Statement,” The American Economic Review, 94, 404–404.
   Web of Science ®Google Scholar
• Bollen, K., Cacioppo, J. T., Kaplan, R. M., Krosnick, J. A., and Olds, J. L. (2015), “Social, Behavioral, and Economic Sciences Perspectives on Robust and Reliable Science,” Report of the Subcommittee on Replicability in Science Advisory Committee to the National Science Foundation Directorate for Social, Behavioral, and Economic Sciences, National Science Foundation. Available at https://www.nsf.gov/sbe/AC_Materials/SBE_Robust_and_Reliable_Research_Report.pdf
   Google Scholar
• Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., Heikensten, E., Holzmeister, F., Imai, T., Isaksson, S., Nave, G., Pfeiffer, T., Razen, M., and Wu, H. (2016), “Evaluating Replicability of Laboratory Experiments in Economics,” Science, 351, 1433–1436. DOI: 10.1126/science.aaf0918.
   PubMed Web of Science ®Google Scholar
• Chang, A. C., and Li, P. (2017), “A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time,” American Economic Review, 107, 60–64.
   Web of Science ®Google Scholar
• Christian, T.-M., Lafferty-Hess, S., Jacoby, W., and Carsey, T. (2018), “Operationalizing the Replication Standard: A Case Study of the Data Curation and Verification Workflow for Scholarly Journals,” International Journal of Digital Curation, 13, 114–124.
   Google Scholar
• Clemens, M. A. (2017), “The Meaning of Failed Replications: A Review and Proposal,” Journal of Economic Surveys, 31, 326–342. DOI: 10.1111/joes.12139.
   Web of Science ®Google Scholar
• Coffman, L. C., Niederle, M., and Wilson, A. J. (2017), “A Proposal to Organize and Promote Replications,” American Economic Review, 107, 41–45. DOI: 10.1257/aer.p20171122.
   Web of Science ®Google Scholar
• Data Citation Synthesis Group and Martone, M. (2014), “Joint Declaration of Data Citation Principles,” Technical report, Force11. Available at DOI: 10.25490/a97f-egyk.
   Google Scholar
• Dewald, W. G., Thursby, J. G., and Anderson, R. G. (1986), “Replication in Empirical Economics: The Journal of Money, Credit and Banking Project,” The American Economic Review, 76, 587–603.
   Web of Science ®Google Scholar
• Duflo, E., and Hoynes, H. (2018), “Report of the Search Committee to Appoint a Data Editor for the AEA,” AEA Papers and Proceedings, 108, 745.
   Google Scholar
• Editors (2021), “Supporting Computational Reproducibility through Code Review,” Nature Human Behaviour, 5, 965–966.
   PubMed Web of Science ®Google Scholar
• Fanelli, D. (2018), “Opinion: Is Science Really Facing a Reproducibility Crisis, and do We Need it to?” Proceedings of the National Academy of Sciences, 115, 2628–2631.
   PubMed Web of Science ®Google Scholar
• Fineberg, H., Stodden, V., and Meng, X.-L. (2020), “Highlights of the US National Academies Report on ‘Reproducibility and Replicability in Science’,” Harvard Data Science Review, 2.
   Google Scholar
• Gall, T., Ioannidis, J. P. A., and Maniadis, Z. (2017), “The Credibility Crisis in Research: Can Economics Tools help?” PLOS Biology, 15, e2001846.
   PubMed Web of Science ®Google Scholar
• Hamermesh, D. S. (2007), “Viewpoint: Replication in Economics,” Canadian Journal of Economics, 40, 715–733. DOI: 10.1111/j.1365-2966.2007.00428.x.
   Web of Science ®Google Scholar
• Heroux, M. A. (2015), “Editorial: ACM TOMS Replicated Computational Results Initiative,” ACM Transactions on Mathematical Software, 41, 1–5. DOI: 10.1145/2743015.
   Web of Science ®Google Scholar
• Höffler, J. H. (2017a), “Replication and Economics Journal Policies,” American Economic Review, 107, 52–55. DOI: 10.1257/aer.p20171032.
   Web of Science ®Google Scholar
• Höffler, J. H. (2017b), “ReplicationWiki: Improving Transparency in Social Sciences Research,” D-Lib Magazine, 23.
   Google Scholar
• Jacoby, W. G., Lafferty-Hess, S., and Christian, T.-M. (2017), “Should Journals Be Responsible for Reproducibility?” Inside Higher Ed.
   Google Scholar
• Journal of Finance (2021), “Retracted: Risk Management in Financial Institutions,” The Journal of Finance, 76, 2709.
   Web of Science ®Google Scholar
• King, G. (1995), “Replication, Replication,” PS, Political Science & Politics, 28, 443–499.
   Web of Science ®Google Scholar
• Kingi, H., Stanchi, F., Vilhuber, L., and Herbert, S. (2018), “The Reproducibility of Economics Research: A Case Study,” Presentation, Berkeley, CA. Available at https://osf.io/srg57/
   Google Scholar
• McCullough, B. D., and Vinod, H. D. (1999), “The Numerical Reliability of Econometric Software,” Journal of Economic Literature, 37, 633–665. DOI: 10.1257/jel.37.2.633.
   Web of Science ®Google Scholar
• National Academies of Sciences, Engineering, and Medicine (2018), Data Science for Undergraduates: Opportunities and Options, Washington, DC: The National Academic Press.
   Google Scholar
• National Academies of Sciences, Engineering, and Medicine (2019), Reproducibility and Replicability in Science, Washington, DC: National Academies Press.
   Google Scholar
• Olken, B. A. (2015), “Promises and Perils of Pre-analysis Plans,” Journal of Economic Perspectives, 29, 61–80. DOI: 10.1257/jep.29.3.61.
   Web of Science ®Google Scholar
• Pérignon, C., Gadouche, K., Hurlin, C., Silberman, R., and Debonnel, E. (2019), “Certify Reproducibility with Confidential Data,” Science, 365, 127–128. DOI: 10.1126/science.aaw2825.
   PubMed Web of Science ®Google Scholar
• Simonsohn, U., Nelson, L., Simmons, J., and Anonymous. (2021), “[98] Evidence of Fraud in an Influential Field Experiment About Dishonesty.”
   Google Scholar
• Stodden, V., McNutt, M., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. A., and Taufer, M. (2016), “Enhancing Reproducibility for Computational Methods,” Science, 354, 1240–1241. DOI: 10.1126/science.aah6168.
   PubMed Web of Science ®Google Scholar
• Stodden, V., Seiler, J., and Ma, Z. (2018), “An Empirical Analysis of Journal Policy Effectiveness for Computational Reproducibility,” Proceedings of the National Academy of Sciences, 115, 2584–2589.
   PubMed Web of Science ®Google Scholar
• Vilhuber, L. (2021), “Report by the AEA Data Editor,” AEA Papers and Proceedings, 111, 808–817.
   Google Scholar
• Vilhuber, L. (2022), “Report by the AEA Data Editor,” AEA Papers and Proceedings, 112, 813–823.
   Google Scholar
• Vilhuber, L. (2019), “Report by the AEA Data Editor,” AEA Papers and Proceedings, 109, 718–729.
   Google Scholar
• Vilhuber, L. (2020), “Reproducibility and Replicability in Economics,” Harvard Data Science Review, 2. DOI: 10.1162/99608f92.4f6b9e67.
   Google Scholar
• Vilhuber, L., Connolly, M., Koren, M., Llull, J., and Morrow, P. (2020), “A Template README for Social Science Replication Packages,” Zenodo Version Number: v1.0.0.
   Google Scholar
• Vilhuber, L., Turitto, J., and Welch, K. (2020), “Report by the AEA Data Editor,” AEA Papers and Proceedings, 110, 764–775.
   Web of Science ®Google Scholar