Learning in a game-based virtual environment: a comparative evaluation in higher education

Abstract

The authors define the requirements and a conceptual model for comparative evaluation research of simulation games and serious games (SGs) in a learning context. A first operationalisation of the model was used to comparatively evaluate a suite of 14 SGs on varying topics played between 2004 and 2009 in 13 institutes of higher education in the Netherlands. The questions in this research were: what is the perceived learning effectiveness of the games and what factors explain it? How can we comparatively evaluate games for learning? Data were gathered through pre- and post-game questionnaires among 1000 students, leading to 500 useful datasets and 230 complete datasets for analysis (factor analysis, scaling, t-test and correlation analysis) to give an explorative, structural model. The findings are discussed and a number of propositions for further research are formulated. The conclusion of the analysis is that the students’ motivation and attitudes towards game-based learning before the game, their actual enjoyment, their efforts during the game and the quality of the facilitator/teacher are most strongly correlated with their learning satisfaction. The degree to which the experiences during the game were translated back into the underlying theories significantly determines the students’ learning satisfaction. The quality of the virtual game environment did not matter so much. The authors reflect upon the general methodology used and offer suggestions for further research and development.

Keywords:

• interactive learning environments
• simulations
• authoring tools and methods
• cooperative/collaborative learning
• interdisciplinary projects

Acknowledgements

The authors thank the following for their contributions to and involvement in the Cyberdam projects: Diny Peters, Pieter van der Hijden, Paul Maharg, Emma Nicol, Kees van Haaster, Kees de Vey Mestdagh, Satiesh Bajnath, Rens Philipsen. We also thank the many teachers and students who participated in the games.

Notes

Cyberdam was first built during the KODOS project (December 2004–2006) and continued in the Learning in a Virtual World (LIEVW) project (January 2007–November 2009). Cyberdam was partly funded by the governmental bureau ‘Maatschappelijke Sectoren en ICT’ and the IP was transferred from Stichting Rechten Online (the Law Online Foundation) to an independent Cyberdam User Association. The Cyberdam virtual environment can be found at www.cyberdam.nl. Log in for demos and to use through: games.cyberdam.nl

Focus on K12 games in the classroom and psychology.

A more elaborate model, not discussed here, also distinguishes a second-order learning effect, mediated by the learning context variables.

Important socio-demographics like nationality (culture) were more or less accidently left out: 90% of players in the study were Dutch, and around 10% Scottish. In later studies on other SGs, questions on nationality and culture were included.