Knowledge Surveys as an Assessment Tool of Simulation Course Outcomes

Abstract

One challenge faced by instructors incorporating simulations and games into political science courses is how to assess learning outcomes from non-“traditional,” pedagogical methods. Positive or anticipated simulation outcomes do not necessarily indicate positive learning outcomes for students. And, using more traditional methods of assessment (quizzes, exams, research papers, etc.) may not align well with the intended outcomes and work related to the simulation. This article presents a range of assignments used to assess learning outcomes for GOV 1320: Topics in International Politics, an introductory international relations course at Worcester Polytechnic Institute. The course employs International Relations in Action: A World Politics Simulation in half of the class sessions as a method to engage students with the course material. In order to assess whether participation in the simulation and completion of the related assignments result in positive learning outcomes for students, I have included knowledge surveys as one assessment tool. In this article, I review frameworks for simulation development and assessment, as well as the existing literature on the usage and “success” of using knowledge surveys as an assessment tool. In addition, I present the intended outcomes of GOV 1320 in general and of the simulation specifically, and the assignments and activities used to assess student learning outcomes. Finally, I examine the use of knowledge surveys in the course. Because knowledge surveys rely on self-reported levels of confidence on course material, the article also includes a comparison of students’ understanding of their knowledge to the actual outcomes as measured by course grades.

Keywords:

• Simulations
• assessment
• knowledge survey
• course design
• role-playing

Acknowledgments

The author is extremely grateful to all of the peer-learning assistants who returned to GOV 1320 to assist in implementing the simulation, with special thanks to Ed Burnham for his assistance with this project.

Notes

Notes

1 IRB Exemption #17-004 received for “Knowledge Survey Assessment Implementation for GOV 1320 from WPI IRB# 1 HHS; IRB # 00007374.

2 The university’s academic year is split into 7-week terms (two in the Fall and two in the Spring), instead of the traditional two-semester course calendar. Students usually have four, in-class contact hours per week per course over the 7-week term. Thus, the university’s instructors only have about two-thirds the amount of class contact time with students in one term when compared to the traditional semester.

3 A local public high school runs a joint program with the university that allows advanced high school seniors to take college courses for credit.

4 The reflection question assignment has evolved over the course of the simulation implementation. In the first 3 years of implementation, including Fall 2015, students were required to reflect specifically on the outcome of the simulation round at the close of that session in the Scenario Outcomes Assessments. In the last 2 years (Spring 2017 and Fall 2017), students were required to answer a more general reflection question within 24 hours of the end of each class meeting, including simulation sessions. In both cases, students responded to a prompt and were expected to use IR theories and concepts that they were learning in their responses. See Table 1 for more details.

5 Reflection questions were not used in the other term (B 2015) in which a knowledge survey was implemented; consequently that term’s data has been excluded from this part of the analysis.

Additional information

Notes on contributors

Patricia A. Stapleton

Patricia A. Stapleton is an Assistant Professor of Political Science at Worcester Polytechnic Institute (WPI). She teaches comparative politics, public policy, and international relations, with an emphasis on science, technology, and environmental politics. Her pedagogical practice emphasizes active learning techniques and student reflection, particularly role-playing and simulation activities. Professor Stapleton has received the Romeo L. Moruzzi Young Faculty Award for Innovation in Undergraduate Education from WPI in recognition for the work she has done in the classroom and for her contributions to pedagogical research. She can be reached at [email protected].