Does Statecraft Improve Student Learning Outcomes? A Controlled Comparison

Abstract

This paper presents results from a comparative analysis of two sections of Introduction to International Politics, one of which used a traditional research paper as a supplemental assignment and one that used the Statecraft online simulation. Both sections were taught during the same semester and used common lecture notes, PowerPoint slides, exam reviews, and exams. The only difference was the nature of the supplemental assignment. The paper finds that the best predictor of student performance on exams is a student’s GPA prior to taking the class. At the same time, the evidence suggests that time dedicated to the online simulation in class may have led to worse performance on at least one exam. Finally, the paper finds that students did enjoy the Statecraft simulation and generally believed that it was preferable to a traditional research paper in spite of its representing additional work relative to the more traditional research paper.

Keywords:

• Active learning
• simulations
• randomized experiment
• online learning

Notes

1 The Statecraft simulation can be found at http://www.statecraftsim.com.

2 For a more detailed review of Statecraft, see Cox 2014.

3 The project was submitted to the Texas Christian University Institutional Review Board and was granted an exemption for pedagogical research.

4 This may have introduced the possibility of cheating if students from the first class informed students in the second class of material on the exam. However, as the classes were only separated by three hours, I determined that the necessity of giving students identical measurements outweighed the risk of cheating.

5 This data was difficult to obtain; full GPA results were not available for more than two years after the initial experiment.

6 As I could not rule out that the treatment led to lower grades on this exam, I did curve the exam scores for the simulation class.

7 The Statecraft website contains several resources to help professors incorporate lessons from the simulation into their classes.

8 Interestingly, Statecraft itself now markets itself at least in part on making life for professors easier; on their website they advertise a reported savings of 13.03% in preparing for class from a 2017 user survey (https://www.statecraftsims.com/international-relations-sim).

Additional information

Notes on contributors

Eric Cox

Eric Cox is an Associate Professor of Political Science at Texas Christian University. He served as president of the Active Learning and International Affairs Section of the International Studies Association from 2010–2011 and as a member of the Board of Directors of National Model United Nations (NMUN) from 2012–2018. He was president of the NMUN Board of Directors from 2014–2018 and is now an ex oficio member of the Board as past president. His research includes the study of enduring rivalries, UN institutions and pedagogical methods. His first book, Why Enduring Rivalries Do—or Don't—End examines the factors that lead rival states to the negotiating table through a comparison of rivalry relationships in the Middle East and Latin America. He has published articles on the origin of the UN Human Rights Council and the effectiveness of the Universal Periodic Review mechanism and coauthored an online Model UN simulation for Statecraft Sims.