Objective Assessment of Pedagogical Effectiveness and the Human Rights Foreign Policy Simulation Game

Abstract

Literature on simulation games stresses both the need for objective assessment of pedagogical effectiveness and the concern that such assessment may not be happening because it is too difficult. This article speaks to both points by presenting an approach to objectively assess the effectiveness of a simulation called the Human Rights Foreign Policy Game (HRFPG). After describing the HRFPG and its learning objectives, the article explains this approach to assessment. This approach applies and extends upon insights from earlier approaches, especially Shaw, by using a pretest/posttest design and open-ended questions. By performing a thematic analysis of students’ posttest descriptions of what they learned, we identify incidents of learning related to the game’s learning objectives and then increase our confidence in this assessment by conducting a student-specific turnover analysis to eliminate any claimed learning that had already been expressed on the pretest. We then demonstrate how the resulting data can be used in basic quantitative analyses to ask further interesting questions. Three main conclusions are reached. First, the HRFPG generates objectively assessable learning. Second, the HRFPG does not systematically disadvantage students according to gender or attitudes toward competitive games or group work. Finally, the assessment methodology, while time-consuming, is practicable.

Keywords:

• Simulations
• pedagogy
• human rights
• foreign policy
• objective assessment

Notes

1 Thanks to Scott Nicholson. We tried to integrate his patient and helpful advice into the game's design. Any major flaws that remain are the authors' responsibility alone.

2 Complete instructions for the HRFPG are available from the corresponding author upon request.

3 This project was approved by the Research Ethics Board of Wilfrid Laurier University, approval number 5408.

4 The data that support the findings of this study are openly available in the Wilfrid Laurier University Library Research Data Repository Dataverse at https://doi.org/10.5683/SP2/L6BYP0

5 Thanks to Dr. Stacey Wilson-Forsberg for suggesting this point.

Additional information

Notes on contributors

Andrew M. Robinson

Andrew M. Robinson, PhD ([email protected]), is Associate Professor at Wilfrid Laurier University, where he is program coordinator for the BA program in Human Rights and Human Diversity. His research primarily focuses on philosophical and practical issues of human rights and cultural diversity in liberal democracies like Canada. He also publishes articles on pedagogy and program development. Twitter handle: @lbhrhd

Michelle Goodridge

Michelle Goodridge, MA, MLIS ([email protected]), is the Liaison Librarian, Game Design and Development at Wilfrid Laurier University. She holds a Master of Arts in History and a Master of Library and Information Science, both from the University of Western Ontario. Michelle’s research interests include outreach and community building both on and off campus, creating unique and nontraditional partnerships (e.g., academic and public libraries), game-based teaching, social media, and active learning. Twitter handle: @migoodridge.