![Sample our Humanities journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5939/TOC-Subject-Sample-2021-Humanities.jpg"})
Abstract
The authors of this article describe an empirical research project as a component of an upper-level undergraduate economics writing-in-the-discipline course, thus aiming to reduce the high fixed costs associated with designing an empirical research project assignment and encourage more undergraduate economics research. This project is central to the course structure and has a dual-purpose: to teach students economics writing conventions and reproducible empirical research methods. The authors present a sequenced project design and replication documentation protocol and posit that this promotes student learning and leads to improvements in organization and coherence throughout the entire research and writing process. As an essential element of the course, students learn to do econometrics through effective writing, data management, and empirical analysis.
Acknowledgments
The authors thank Noreen Lape, Gail Hoyt, and Richard Ball for comments, advice, and suggestions throughout the writing process. They also thank attendees of the November 2017 Annual Professors Conference at the Federal Reserve Bank of St. Louis, the November 2017 Annual Economics Teaching Conference, and the May 2018 Conference on Teaching and Research in Economic Education who provided valuable comments. Finally, the authors thank two anonymous reviewers and the editors whose comments and suggestions greatly improved this article.
Appendix A
Just as good writing requires forethought and planning, so does effective data workflow and construction of replication documentation. Thus, this project can be viewed as two interrelated projects happening simultaneously: the writing process and data workflow. This workflow is designed to help keep students organized and enhance their understanding of data processing and analysis. below outlines the project timeline and workflow that is distributed to students at the beginning of the semester:
Notes
1 Of the institutions that do have a formal writing requirement, the most common type is a “writing-in-the-discipline” (WID) course or senior capstone with a writing component. Additionally, only 15 percent of economics departments reported offering courses dedicated to the research process, while fewer still (10%) reported offering a course that was specifically designated as “research methods” (McGoldrick Citation2008b).
2 Project TIER developed the TIER protocol in an effort to advance the goals of research transparency and reproducibility through disseminating and teaching instructional practices. The guiding principle of this protocol is “that the documentation should allow an independent researcher to replicate every step of the data management and analysis and to generate the same results” (Ball and Medeiros Citation2012). For more information, please visit http://www.projecttier.org/.
3 These materials are highly adaptable. Password protected electronic copies of the prompts and rubrics can be downloaded here: http://sites.google.com/site/emilycorinnemarshall/teaching-resources and here: http://blogs.dickinson.edu/underwood/teaching-resources. Please email either Emily C. Marshall ([email protected]) or Anthony Underwood ([email protected]) from your institutional email account to obtain the password.
4 This is a highly adaptable feature of the project. In our experiences, we have utilized both email and Microsoft OneDrive. However, many other platforms could be used such as Dropbox, Google Drive, or Open Science Framework.
5 This includes: (1) original data files (in unmodified form), (2) importable data files (minimally altered in order to be read by the statistical software), (3) base data file (master data file combining data from all original sources), (4) processing command (.do) file (file including all commands used to produce the base data file), and (5) metadata guide with an entry explaining how to obtain and understand each original data file.
6 This entails the submission of (1) an analysis data (.dta) file (used to produce the final estimates in the paper), (2) a construction command (.do) file (file containing all commands used to produce the analysis data from the base data), (3) a data appendix summarizing the analysis data, and (4) a summary command (.do) file (file including all commands used to produce the data appendix). The Data and Methods document describes the conceptual or theoretical framework, data, and model specification.
7 The analysis command (.do) file contains all commands used to produce final results tables included in the paper.
8 We also ask students to include a ReadMe file explaining the contents of the replication folder and providing step-by-step replication instructions.
9 This also could be implemented in a first econometrics course, provided that, it is upper-level.
10 These workshops are designed to facilitate both the completion of problem sets and complement the project components, including topics such as finding sources, finding data, and cleaning data.
11 In 1980, one-third of institutions offered honors in economics (Siegfried and Wilkinson Citation1982) and only about 7 percent of major programs required a senior thesis. As of 2013, these numbers have risen to 46 and 18 percent, respectively.
12 According to Swarthmore College’s (n.d.) National Census of Writing, 39 percent of the sampled 4-year colleges and universities reported having WID and 62 percent require all students to take writing-intensive courses (outside of the English or writing department).
13 As instructors, writing assignment prompts is one of the two most important forms of writing we do for our students (Gottschalk and Hjortshoj Citation2004). The other is written feedback from the instructor, which is discussed in the evaluation and assessment section.
14 At the end of the semester, students ranked their knowledge (1 = least knowledgeable; 5 = most knowledgeable) of college-level writing before and after taking the course. From a sample of 26 students (spring 2017) and two different instructors, students report prior knowledge of 3.5 and after knowledge of 4.3, a statistically significant increase (t = 3.38, p = .002). Students also ranked their ability (1 = least able; 5 = most able) to write at the college-level before and after taking the course. From this same sample, students report prior ability of 3.6 and after ability of 4.2, also a statistically significant increase (t = 2.96; p = .005).
15 From a sample of 55 students (spring 2017, fall 2017, and spring 2018 semesters) and two different instructors, 91 percent of students (agree or strongly agree) that after taking the course they have a better understanding of the subject matter and 96 percent (agree or strongly agree) that they now have a better understanding of the methods used in econometrics.
16 Chang and Li (2015) attempt to replicate 67 empirical papers published in 13 well-regarded economics journals; some of these journals require data and code files and others do not. They were able to replicate the key qualitative results for 49 percent of the papers (Chang and Li Citation2015).
17 This folder structure is an adaptation of the TIER protocol. See https://www.projecttier.org/tier-protocol/ for more details.
18 For partial replication, the folder structure could be simplified. A suggested organization would be three main subfolders: Documents, Data, and Code. Documents would contain all written project components, data would include all data files (most likely in .dta format), and code would consist of .do files to run the data transformation and results.
19 In our course, the project is worth 225 points out of 500 total points available, or 45 percent of the final course grade. At national universities and liberal arts colleges that require a project in econometrics, the projects typically constitute 28 to 29 percent of the course grade (Johnson, Perry, and Petkus Citation2012).
20 Each part of the project is worth 9 to 20 percent of the project grade or 4 to 9 percent of the final course grade.
21 In this context, “practice” refers to the components of the final paper and the final paper itself. Practice is defined as any activity in which students apply their knowledge or skills.
22 For more information on written responses to student writing, see Flanigan and Menendez (Citation1980), Flower (Citation1979), and Sommers (Citation1980, Citation1982).
23 In addition, it is tempting to spend more time on weaker student papers and less time providing feedback to strong student papers. Instructors must be careful to balance feedback between weak and strong student papers.
24 In our experience, we estimate that utilizing a rubric at each phase of the project for an individual student cuts grading time to ∼20 to 30 min per paper. This, of course, can be highly variable and time per paper tends to decrease with experience but converges to a non-zero positive number.
25 From a sample of 55 students (spring 2017, fall 2017, and spring 2018 semesters) and two different instructors, 82 percent of students (agree or strongly agree) that instructor’s comments were an important part of their learning in the course.
26 Note, there are many variations on minimal marking and only one is described here.
27 Originally, this protocol was developed for Stata and R. See http://www.projecttier.org/tier-protocol/ for more details.
28 Stata 15 also now has integration capabilities with Microsoft Word. See https://www.stata.com/new-in-stata/create-word-documents/ for information on this update.