Teaching Integrity in Empirical Research: A Protocol for Documenting Data Management and Analysis

Abstract

This article describes a protocol the authors developed for teaching undergraduates to document their statistical analyses for empirical research projects so that their results are completely reproducible and verifiable. The protocol is guided by the principle that the documentation prepared to accompany an empirical research project should be sufficient to allow an independent researcher to replicate easily and exactly every step of the data management and analysis that generated the results reported in a study. The authors hope that requiring students to follow this protocol will not only teach them how to document their research appropriately, but also instill in them the belief that such documentation is an important professional responsibility.

Keywords:

• documentation
• empirical research
• replication

JEL codes:

• A22
• B4
• C8

Acknowledgments

The authors thank participants in the AEA Conference on Teaching Economics and Economic Research, Stanford University, June 1–3, 2011, for valuable discussion and comments, with particular thanks to Vera Brusentsev, the discussant for the paper, for especially insightful and constructive ideas. The article has also benefitted from the thoughtful reports of two referees. This article is based on a paper that was presented at the National Conference on Teaching Economics held at Stanford University on June 1–3, 2011.

Notes

1. For additional discussion of current practice in economics with respect to the documentation and replicability of empirical research, see Anderson et al. (2008); McCullough, McGeary, and Harrison (2006, 2008); and Vinod (2001).

2. Several related books may interest instructors and thesis supervisors. Long's (2009) book is an excellent and detailed guide to organizing data management and analysis in ways that facilitate the construction and assembly of the documentation our protocol requires. (Although it is written expressly for Stata users, its main lessons are easily transferable to any other statistical package.) For an excellent guide to the entire process of writing research papers, aimed at undergraduate economics majors and including several chapters on working with statistical data, see Greenlaw (2005). And for pointers and principles of good writing, see McCloskey (1999).

3. When students turn in their senior theses, they are also asked to sign release forms granting the library permission to post their work in the thesis archive. These releases indicate the level of access to be granted, which the student can choose to be unrestricted open access, access limited to authenticated users of Haverford's network, or access allowed only for archive administrators and Haverford economics faculty. The student also can request to have the thesis embargoed until a specified release date.