A systematic review of serious games in medical education: quality of evidence and pedagogical strategy

Figures & data

Figure 1. Selection process of articles.

Table 1. MERSQI domain and item scores for 21 studies of games used for medical education.

Domain	Item (score)	MERSQI* Average score (SD)	N° Studies (%)
Study design	1. Study design
	Randomized controlled trial (3)	12.6 (1.2)	9 (42.8)
	Nonrandomized 2 group (2)	11 (0.7)	2 (9.52)
	Single group pre-test and post-test (1.5)	9 (4.9)	2 (9.52)
	Single group cross-sectional or single group post-test only (1)	9 (2.6)	8 (38)
Sampling	2. No. of institutions studied
	>2 institutions(1.5)	8 (0)	1 (4.76)
	2 institutions (1)	10.5 (0)	1 (4.76)
	1 institution (0.5)	10.9 (2.8)	19 (90.4)
	3. Response rate %
	>75 % (1.5)	11.5 (2.2)	17 (80.9)
	50-74 % (1)	8 (3.1)	3 (14.2)
	<50% or not reported (0.5)	6 (0)	1 (4.76)
Type of data	4. Type of data
	Assessment by study participant (1)	7 (2.5)	5 (23.8)
	Objective measurement (3)	11.9 (1.4)	16 (76.1)
Validity of evaluation instrument	5. Internal structure
	Not reported (0)	10 (2.8)	15 (71.4)
	Reported (1)	12.5 (1.5)	6 (28.5)
	6. Content
	Not reported (0)	10 (2.6)	16 (76.1)
	Reported (1)	13 (1.6)	5 (23.8)
	7. Relationships to other variables
	Not reported (0)	10.3 (2.7)	18 (85.7)
	Reported (1)	13 (2.2)	3 (14.2)
Data analysis	8. Appropriateness of analysis
	Data analysis appropriate for study design or type of data (1)	11.3 (2.0)	19 (90.4)
	Not statistical analysis (0)	5 (0.7)	2 (9.52)
	9. Complexity of analysis
	Beyond descriptive analysis (2)	12.1 (1.2)	15 (71.4)
	Descriptive analysis only (1)	8.37 (1.8)	4 (19)
	Not statistical analysis (0)	5 (0.7)	2 (9.52)
Outcomes	10. Outcomes
	Satisfaction, attitudes, perceptions, opinions, general facts (1)	7 (2.5)	5 (23.8)
	Knowledge, skills (1.5)	11.9 (1.4)	16 (76.1)
	Behaviours (2)	0 (0)	0 (0)
	Patient/health care outcome (3)	0 (0)	0 (0)

MERSQI* by Reed (2007) medical education research study quality instrument.

SD: Standard deviation

Table 2. Evidence on the effectiveness of games.

Author	Author´s Institution	Study design	Specialty	Assessment	N° of participants	Setting	MERSQI score
Boeder (2013) (28)	U. of Munich, Germany	Single group cross-sectional or single group post-test only	Internal medicine	Knowledge	11	Classroom	4.5
Boeker et al (2013)(29)	Freiburg U., Germany	Randomized controlled trial	Urology	Knowledge and perception	145	Classroom	15.5
Yu et al (2015) (30)	U. of Toronto and St. Michael’s Hospital, Canada	Single group cross-sectional or single group post-test only	Physiology/anatomy	Knowledge	75	Hospital or Clinical setting	11
Cendan et al (2011) (31)	U. of Central Florida, United States	Randomized controlled trial	Physiology/anatomy	Knowledge and perception	40	Hospital or Clinical setting	13.5
Creutzfeldt (2012) (32)	Karolinska Institutet, Sweden	Nonrandomized 2 group	Emergency	Skill	30	Hospital or Clinical setting	10.5
Creutzfeldt et al (2010) (33)	Karolinska Institutet, Sweden	Single group pre-test and post-test	Emergency	Knowledge and perception	12	Classroom	12.5
Gasco et al (2014) (34)	U. of Texas, United States	Randomized controlled trial	Neurology	Skill	26	Not reported	12.5
Kanthan et al (2011) (35)	U. of Saskatchewan, Canada	Nonrandomized 2 group	Pathology	Knowledge	185	Classroom	11.5
Knight et al (2010) (36)	U. of Birmingham and others, United Kingdom	Randomized controlled trial	Emergency	Knowledge, perception and skill	91	Hospital or Clinical setting	12.5
Kreiter et al (2011) (37)	U. of Kansas and U. of Iowa, United States	Single group cross-sectional or single group post-test only	Clinical Lab	Knowledge and perception	Started with 13, ended with 143	Classroom	12.5
Lameris et al (2015) (38)	Radboud U., Netherlands	Single group cross-sectional or single group post-test only	Physiology/anatomy	Knowledge and perception	439	Classroom	9
Longmuir (2014) (39)	U. of California, United States	Single group cross-sectional or single group post-test only	Physiology/anatomy	Knowledge	56	Classroom	6
McGrath et al (2015) (40)	Ohio State U., United States	Randomized controlled trial	Emergency	Knowledge	35	Classroom	12.5
Moreno-Ger et al (2010) (41)	Complutense U. of Madrid, Spain	Randomized controlled trial	Physiology/anatomy	Knowledge	143	Hospital or Clinical setting	11
Nevin et al (2014) (42)	U. of Alabama (UAB) -(UAH), United States	Single group cross-sectional or single group post-test only	Internal medicine	Knowledge and perception	128 at UAB and 24 at UAH	Hospital or Clinical setting	10.5
Rondon et al (2013) (43)	U. of Sao Paulo, Brazil	Randomized controlled trial	Phono audiology	Knowledge	29	Classroom	11.5
Sánchez-Rola et al (2014) (44)	U. of Deusto, Spain	Single group cross-sectional or single group post-test only	Neurology	Knowledge	50	Classroom	8
Schmeling et al (2011) (45)	U. of Müenster, Germany	Single group pre-test and post-test	Forensic	Knowledge and perception. But only perception was reported	29	Hospital or Clinical setting	5.5
Silvennoinen et al (2015) (46)	U. of Jyvaskyla, Finland	Single group cross-sectional or single group post-test only	Surgery	Skill	13	Hospital or Clinical setting	10.5
Stirling et al (2014) (47)	Bond U., Australia	Randomized controlled trial	Physiology/anatomy	Knowledge and perception	85	Not reported	12.5
Holloway et al (2015) (48)	McGill U. and Mount Sinai Medical Center, United States and Canada	Randomized controlled trial	Neurosurgery	Skill	83	Hospital or Clinical setting	12.5

Table 3. Pedagogical description of games.

Author	Game name	Type	Platform	Game genre	Games’ pedagogical role	Pedagogical strategy	Game and pedagogy coherence
Boeder (2013) (28)	Mediman	Game	Web-based	Quiz	Independent	Behaviourism	Coherent
Boeker et al (2013) (29)	Uro Island	Game like simulation	Computer-based	Adventure	Independent	Cognitivism	Coherent
Yu et al (2015) (30)	DKA simulator	Game like simulation	Computer-based	Training simulation	Complementary	Behaviourism	Coherent
Cendan et al (2011) (31)	Physiosim	Game like simulation	Web-based	Training simulation	Independent	Insufficient pedagogical description	Insufficient game description
Creutzfeldt (2012) (32)	SimMan	Game like simulation	Web-based	Training simulation	Complementary	Humanism	Insufficient game description
Creutzfeldt et al (2010) (33)	MMVW-CPR	Game like simulation	Computer-based	Training simulation	Complementary	Cognitivism	Coherent
Gasco et al (2014) (34)	ImmersiveTouch simulator	Game like simulation	Computer-based	Training simulation	Complementary	Behaviourism	Coherent
Kanthan et al (2011) (35)	Path to Success	Gamified App	Web-based	Quiz	Complementary	Constructivism	Incoherent
Knight et al (2010) (36)	Triage Trainer	Game like simulation	Web-based	Training simulation	Complementary	Behaviourism	Coherent
Kreiter et al (2011) (37)	LabCAPS	Game like simulation	Computer-based	Training simulation	Complementary	Behaviourism	Coherent
Lameris et al (2015) (38)	Physiomics to the next level	Gamified App	Mobile app	Quiz	Complementary	Cognitivism	Coherent
Longmuir (2014) (39)	Interactive module based in AppCobra	Gamified App	Mobile app	Quiz	Complementary	Constructivism	Incoherent
McGrath et al (2015) (40)	Virtual examination	Game like simulation	Web-based	Training simulation	Complementary	Behaviourism	Coherent
Moreno-Ger et al (2010) (41)	HCT Game-like simulation	Gamified App	Web-based	Training simulation	Complementary	Behaviourism	Coherent
Nevin et al (2014) (42)	Kaizen- IM (Kaizen-Internal Medicine)	Gamified App	Web-based	Quiz	Complementary	Behaviourism	Coherent
Rondon et al (2013) (43)	Anatese 2.0	Gamified App	Computer-based	Quiz	Independent	Constructivism	Insufficient game description
Sánchez-Rola et al (2014) (44)	Mobile NBM	Gamified App	Mobile app	Quiz	Independent	Behaviourism	Coherent
Schmeling et al (2011) (45)	Inmedea Simulator	Game like simulation	Web-based	Training simulation	Complementary	Cognitivism	Coherent
Silvennoinen et al (2015) (46)	Lap MentorTM VR simulator	Game like simulation	Computer-based	Training simulation	Complementary	Cognitivism	Coherent
Stirling et al (2014) (47)	Multimedia ebook for anatomy	Game like simulation	Web-based	Training simulation	Independent	Cognitivism	Coherent
Holloway et al (2015) (48)	NeuroTouch simulator	Game like simulation	Web-based	Training simulation	Complementary	Behaviourism	Coherent

Boeder N. Mediman – The smartphone as a learning platform? GMS Z Med Ausbild. 2013;30(1):1–5.

 Google Scholar

Boeker M, Andel P, Vach W, et al. Game-based e-learning is more effective than a conventional instructional method: A randomized controlled trial with third-year medical students. PLos One. 2013;8(12): 1–11.

 Web of Science ®Google Scholar

Yu C, Straus S, Brydges R. The ABCs of DKA: Development and Validation of a Computer-Based Simulator and Scoring System. J Gen Intern Med. 2015;30(9):1319–1332.

 Web of Science ®Google Scholar

Cendan J, Johnson T. Enhancing learning through optimal sequencing of web-based and manikin simulators to teach shock physiology in the medical curriculum. Adv Physiol Educ. 2011;35(4):402–407.

 Web of Science ®Google Scholar

Creutzfeldt J, Hedman L, Fellander L. Effects of pre-training using serious game technology on CPR performance – an exploratory quasi-experimental transfer study. Scand J of Trauma Resus. 2012;20(79):1–9.

 Google Scholar

Creutzfeldt J, Hedman L, Medin C, et al. Exploring virtual worlds for scenario-based repeated team training of cardiopulmonary resuscitation in medical students. J Med Internet Res. 2010;12(3):1–22.

 Web of Science ®Google Scholar

Gasco J, Patel A, Ortega-Barnett J, Branch D, Desai S, Kuo Y, et al. Virtual reality spine surgery simulation: an empirical study of its usefulness. Neurol Res. 2014;36(11):968–973.

 PubMed Web of Science ®Google Scholar

Kanthan R, Senger J. The impact of specially designed digital games-based learning in undergraduate pathology and medical education. Arch Pathol Lab Med 2011;135(1):135–142.

 PubMed Web of Science ®Google Scholar

Knight J, Carley S, Tregunna B, et al. Simulation and education: Serious gaming technology in major incident triage training: A pragmatic controlled trial. Resuscitation. 2010;81(9):1175–1179.

 PubMed Web of Science ®Google Scholar

Kreiter C, Haugen T, Leaven T, et al. A report on the piloting of a novel computer-based medical case simulation for teaching and formative assessment of diagnostic laboratory testing. Med Educ Online. 2011;16:1–8.

 Google Scholar

Lameris A, Hoenderop J, Bindels R, et al. The impact of formative testing on study behaviour and study performance of (bio)medical students: A smartphone application intervention study. BMC Med Educ. 2015;15(1):1–8.

 PubMedGoogle Scholar

Longmuir K. Interactive computer-assisted instruction in acid-base physiology for mobile computer platforms. Adv Physiol Educ. 2014; 38(1):34–41.

 Web of Science ®Google Scholar

McGrath J, Kman N, Bahner D, et al. Virtual alternative to the oral examination for emergency medicine residents. West J Emerg Med. 2015;16(2):336–343.

 PubMed Web of Science ®Google Scholar

Moreno-Ger P, Torrente J, Bustamante J, et al. Application of a low-cost web-based simulation to improve students’ practical skills in medical education. Int Journal Med Inform. 2010;79(6):459–467.

 Web of Science ®Google Scholar

Nevin C, Westfall A, Rodriguez J, et al. Gamification as a tool for enhancing graduate medical education. Postgrad Med J. 2014;90(1070):685–693.

 PubMed Web of Science ®Google Scholar

Rondon S, Sassi F, Andrade C. Computer game-based and traditional learning method: a comparison regarding students’ knowledge retention. BMC Med Educ. 2013; 13(30):1–9.

 Google Scholar

Sánchez-Rola I, Zapirain B. Mobile NBM - Android medical mobile application designed to help in learning how to identify the different regions of interest in the brain’s white matter. BMC Med Educ. 2014;14(1):1–22.

 Google Scholar

Schmeling A, Kellinghaus M, Schulz R, et al. A web-based e-learning programme for training external post-mortem examination in curricular medical education. Int J Leg Med. 2011; 125(6):857–861.

 Web of Science ®Google Scholar

Silvennoinen M, Helfenstein S, Ruoranen M, Saariluoma P. Learning basic surgical skills through simulator training. Instr Sci. 2012; 40(5):769–783.

 Web of Science ®Google Scholar

Stirling A, Birt J. An enriched multimedia eBook application to facilitate learning of anatomy. Anat Sci Educ 2014;7(1):19–27.

 PubMed Web of Science ®Google Scholar

Holloway T, Lorsch Z, Chary M, et al. Operator experience determines performance in a simulated computer-based brain tumor resection task. Int J Comput Ass Rad. 2015;10(11):1853–1862.

 Web of Science ®Google Scholar