Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis

Figures & data

Box 1 Definitions of terms*

Participants

Health professions learner: a student, postgraduate trainee, or practitioner in a profession directly related to human or animal health, including physicians, dentists, nurses, veterinarians, physical, occupational, and respiratory therapists, and emergency medical technicians and other first responders.

Instructional design key features

Clinical variation: Variation in the clinical context, for example multiple different patient scenarios (absent if no clinically relevant context was stated).

Cognitive interactivity: Training that promotes learners’ cognitive engagement using strategies such as multiple repetitions, feedback, task variation, or intentional task sequencing.

Curricular integration: Incorporation of the simulation intervention as an integral part (required or formal element) of the curriculum or training program.

Distributed practice: Training spread over a period of time. For this review, we counted this as present for interventions that involved >1 day of simulation training.

Feedback: Information on performance provided to the learner by the instructor, a peer, or a computer, either during or after the simulation activity.

Group (vs independent) practice: Training activities involving two or more learners (as compared with training alone).

Individualized learning: Training responsive to individual learner needs (i.e. tailored or adapted depending on performance).

Mastery learning: Training model in which learners must attain a clearly-defined standard of performance before qualifying or advancing to the next task.

Multiple learning strategies: The number of different instructional strategies used to facilitate learning, such as patient case, worked example, discussion, feedback, intentional sequencing, or task variation.

Range of task difficulty: Variation in the difficulty or complexity of the task (explicitly stated).

Repetitive practice: The opportunity for more than one task performance.

Outcomes

Satisfaction. Learners’ reported satisfaction with the course.

Knowledge: Subjective (e.g. learner self-report) or objective (e.g. multiple-choice question knowledge test) assessments of factual or conceptual understanding.

Skills: Subjective (e.g. learner self-report) or objective (e.g. faculty ratings, or objective tests of clinical skills such as computer-scored technique in a virtual reality surgery simulator, or number of masses detected when examining a breast model) assessments of learners’ ability to demonstrate a procedure or technique in an educational setting (typically a simulation task). We further classified skills as measures of time (how long it takes a learner to complete the task), process (e.g. global rating scales, efficiency, or minor errors), and product (successful completion of the task, evaluation of the finished product, or major errors that would impact a real patient's well-being). For purposes of meta-analysis we combined process and product skills into a single outcome, non-time skills.

Behaviors and patient effects: Subjective (e.g. learner or patient self-report) or objective (e.g. chart audit or faculty ratings) assessments of behaviors in practice (such as test ordering) or effects on patients (such as medical errors). We used a classification system similar to that used for Skills, with time and process measures being counted as behaviors (e.g. procedure time, test ordering, or interviewing technique with real patients) and products being counted as patient effects (e.g. complications, patient discomfort, or procedure completion rates).

Note: *Some of these definitions were published as an online appendix to a previous publication (Cook et al. 2011).

Table 1.  Description of included studies

Study Characteristic	Level	No. studies	No. participants*
All studies		289	18 971
Group allocation	Randomized	208	12 227
Study design	Posttest-only	162	12 167
	Pretest-posttest	127	6804
Location	Simulation center	269	16 785
	Clinical environment	13	1421
	Both simulation center and clinical	7	765
Participants^†	Medical students	111	4894
	Physicians postgraduate training	87	2408
	Physicians in practice	39	1684
	Nurses and nursing students	42	4177
	Emergency medical technicians and students	18	824
	Dentists and dental students	22	1348
	Veterinarians and veterinary students	6	276
	Chiropractors and chiropractic students	2	104
	Other/ambiguous/mixed	43	3256
Clinical topics^‡	Minimally invasive surgery	79	3000
	Resuscitation/trauma training	51	6630
	Other surgery	34	1530
	Dental	21	1416
	Intubation	20	1922
	Physical examination	18	933
	Endoscopy and ureteroscopy	18	827
	Vascular access	15	605
	Communication and team skills	11	485
	Anesthesia	8	260
	Obstetrics	6	641
Key features	Clinical variation^§, present	64	4826
	Cognitive interactivity, high	193	13 339
	Curriculum integration^§, present	52	5395
	Distributed practice, >1 day	98	4745
	Feedback^§, high	75	6245
	Group practice	72	6317
	Individualized learning^§, high	18	1545
	Mastery learning, present^§	23	1050
	Multiple learning strategies^§, high	34	3102
	Range of task difficulty^§, present	40	1593
	Repetitive practice^§, present	233	15 521
Outcomes^†	Satisfaction	56	3042
	Knowledge	34	2687
	Skill: Time	100	4198
	Skill: Process	197	9983
	Skill: Product	56	2976
	Behavior: Time	1	28
	Behavior: Process	9	275
	Patient effects	8	422
Quality	Newcastle-Ottawa ≥4 points	160	10 888
	MERSQI ≥12.5 points	173	11 459

Notes: See Appendix 1 for details on individual studies.

*Numbers reflect the number enrolled, except for Outcomes which reflect the number of participants who provided observations for analysis.

^†The number of studies and learners in some subgroups may sum to more than the number for all studies, and percentages may total more than 100%, because several studies included >1 learner group, fit within >1 clinical topic, or reported multiple outcomes.

^‡Selected listing of the most frequently addressed topics (numerous other topics were addressed, with lower frequency). Some interventions addressed >1 clinical topic.

^§Features identified by Issenberg et al. (Issenberg et al., 2005) Mastery learning is similar to Issenberg's “defined outcomes.”

Figure 1. Trial flow.

Table 2.  Quality of included studies

Scale Item	Subscale (points if present)	No. (%) present; N = 289
Medical Education Research Study Quality Instrument (MERSQI)*
Study design (maximum 3)	Observational 2-group (2)	81 (28)
	Randomized 2-group (3)	208 (72)
Sampling: No. institutions (maximum 1.5)	1 (0.5)	255 (88)
	2 (1)	10 (4)
	>2 (1.5)	24 (8)
Sampling: Follow-up (maximum 1.5)	<50% or not reported (0.5)	81 (28)
	50–74% (1)	9 (3)
	≥75% (1.5)	199 (69)
Type of data: Outcome assessment (maximum 3)	Subjective (1)	38 (13)
	Objective (3)	251 (87)
Validity evidence (maximum 3)	Content (1)	83 (29)
	Internal structure (1)	78 (27)
	Relations to other variables (1)	27 (9)
Data analysis: appropriate (maximum 1)	Appropriate (1)	260 (90)
Data analysis: sophistication (maximum 2)	Descriptive (1)	13 (5)
	Beyond descriptive analysis (2)	276 (95)
Highest outcome type (maximum 3)	Satisfaction (1)	25 (9)
	Knowledge, skills (1.5)	249 (86)
	Behaviors (2)	7 (2)
	Patient/health care outcomes (3)	8 (3)
Newcastle-Ottawa Scale (modified)^†
Representativeness of sample	Present (1)	53 (18)
Comparison group from same community	Present (1)	268 (93)
Comparability of comparison cohort, criterion A^‡	Present (1)	213 (74)
Comparability of comparison cohort, criterion B^‡	Present (1)	106 (37)
Blinded outcome assessment	Present (1)	175 (61)
Follow-up high^‡	Present (1)	208 (72)

Notes: *MERSQI total score (maximum 18): mean 12.3 (SD 1.8), median 12.5 (range 6.5–16).

^†NOS total score (maximum 6): mean 3.5 (SD 1.3), median 4 (range 1–6).

^‡Comparability of cohorts criterion A was present if the study (a) was randomized, or (b) controlled for a baseline learning outcome; criterion B was present if (a) a randomized study concealed allocation, or (b) an observational study controlled for another baseline learner characteristic. Follow-up was high if ≥75% of those enrolled provided outcome data, or if authors described those lost to follow-up.

Table 3.  Research themes (comparisons) addressed by studies

Theme, No. (%)*	Theme definition	Sub-theme	No. studies (% of theme)*
Group composition, N = 2 (1%)	Compare different approaches to grouping learners.	Interdisciplinary vs single-discipline group	1 (50%)
		Independent vs group	1 (50%)
Instructional design, N = 133 (39%)	Compare different design features to enhance instructional effectiveness.	Feedback	47 (35.3%)
		Sequence	24 (18%)
		Teach cognitive or mental imagery techniques	12 (9%)
		Repetition	10 (7.5%)
		Task variability	9 (6.8%)
		Clinical scenario	5 (3.8%)
		Hands on practice	5 (3.8%)
		Timing	4 (3%)
		Instructions	3 (2.3%)
		Stress	3 (2.3%)
		Additional practice	2 (1.5%)
		Blending simulation and non-simulation instruction	2 (1.5%)
		Interactivity	2 (1.5%)
		Mastery	2 (1.5%)
		Testing effect	2 (1.5%)
		Add reminder	1 (0.8%)
Instructor, N = 27 (8%)	Compare different levels of instructor training or presence.	Self-instruction	17 (63%)
		Instructor intensity	5 (18.5%)
		Instructor training/experience	3 (11.1%)
		Distance supervision	2 (7.4%)
Modality added, N = 37 (11%)	Evaluate the addition of one or more other modalities (eg, lecture, CAI, or other simulation) to baseline simulation training	CAI, add or compare	9 (24.3%)
		Simulator, add	9 (24.3%)
		Patient/SP experience, add or compare	7 (18.9%)
		Discussion (excluding debriefing), add or compare	3 (8.1%)
		External support, add	2 (5.4%)
		Robot assistance, add	2 (5.4%)
		Team training, add	2 (5.4%)
		Lecture, add or compare	2 (5.4%)
		Tutor, add	1 (2.7%)
Sensory augmentation, N = 19 (6%)	Evaluate the addition of a feature or effect to enhance sensory experience.	Tactile (includes haptics)	11 (57.9%)
		Visual	7 (36.8%)
		Olfactory	1 (5.3%)
Simulation modalities compared, N = 119 (35%)	Compare two technology-enhanced simulation modalities	VR vs model	40 (33.6%)
		Mannequin vs mannequin	16 (13.4%)
		Model vs model	11 (9.2%)
		Mannequin vs model	9 (7.6%)
		Model vs animal	9 (7.6%)
		VR vs VR	8 (6.7%)
		Mannequin vs cadaver	5 (4.2%)
		Model vs cadaver	4 (3.4%)
		VR vs animal	4 (3.4%)
		Mannequin vs animal	3 (2.5%)
		Synthetic vs human/animal products	3 (2.5%)
		VR vs cadaver	2 (1.7%)
		VR vs mannequin	2 (1.7%)
		Animal living vs dead	1 (0.8%)
		Animal products vs animal products	1 (0.8%)
		Cadaver vs animal	1 (0.8%)

Notes: *The 289 studies addressed a total of 337 research themes (38 studies had 2 or 3 arms). Percentages are for sub-themes within a theme.

Abbreviations: CAI = computer-assisted instruction; SP = standardized patient; VR = virtual reality.

Figure 2. Random effects meta-analysis Comparisons of simulation interventions varying in the specified key feature; positive numbers favor the intervention with more of that feature. If a feature is not present in the analysis for a given outcome, it is because no studies reporting that outcome had interventions that varied for that particular feature. Some feature comparisons had only 1 relevant study; these are included in the figures but we note that the effect size reflects only that 1 study (ie, not pooled).

Table A1.  List of all included studies

	Participants				Research comparisons			Quality
Citation (sorted by year then author)	Trainee	N	RCT	Topic	Features	Theme	Outcomes	MERSQI	NOS
Wittrock JW, et al. Use of different sized models in teaching restorative dentistry. J Dent Educ. 1970; 34:76–82.	D	86		Dentistry		SA:Visual	SPd	10.5	2
Johnson GH, et al. Teaching pelvic examination to second-year medical students using programmed patients. Am J Obstet Gynecol. 1975; 121:714–717.	MS	140		Physical exam	ClinV, CogI, FB, Indiv, MLS	MA:Patient/SP experience, add	Sa	9	2
Salvendy G, et al. A second generation training simulator for acquisition of psychomotor skills in cavity preparation. J Dent Educ. 1975; 39:466–471.	D	30	RCT	Dentistry	CogI, FB, GrpP, Indiv	CM:VR vs model	ST, SPc	13.5	3
Holzman GB, et al. Initial pelvic examination instruction: The effectiveness of three contemporary approaches. Am J Obstet Gynecol. 1977; 129:124–129.	MS	28		Physical exam	Time	MA:Tutor, add	K, SPc	12.5	3
Lefcoe DL, et al. Simulated models: a mode for instruction in root planing procedures. Educ Dir Dent Aux. 1979; 3:20–4.	O	12	RCT	Dental cleaning		SA:Visual	P	13.5	5
Herrin TJ, et al. Modular approach to CPR training. South Med J. 1980; 73:742–744.	MS, RN	303		Resuscitation (BLS,ACLS,ATLS)	FB, Indiv	In:Self-instruction	K	11.5	3
Salvendy G, et al. The development and validation of an analytical training program for medical suturing. Hum Factors. 1980; 22:153–170.	MS	18	RCT	Open surgery/suturing		ID:Teach cognition	ST, SPc	13.5	4
LaTurno SA, et al. An evaluation of a teaching aid in endodontics. J Endod. 1984; 10:507–511.	D	150		Dentistry	MLS	SA:Visual	SPd	10.5	2
Stewart RD, et al. Effect of varied training techniques on field endotracheal intubation success rates. Ann Emerg Med. 1984; 13:1032–1036.	EMT	94		Intubation	ClinV, CogI, Indiv, Mast, MLS, RangeD	MA:Patient/SP experience, add	P	12	4
Weiner S, et al. An evaluation of sequential models in the preclinical laboratory. J Dent Educ. 1985; 49:109–110.	D	142		Dentistry		ID:Sequence	SPd	10.5	2
Champagne MT, et al. Use of a heart sound simulator in teaching cardiac auscultation. Focus Crit Care. 1989; 16:448–456.	RN	37	RCT	Physical exam		SA:Tactile	SPc	15.5	3
Carpenter LG, et al. A comparison of surgical training with live anesthetized dogs and cadavers. Veterinary Surgery. 1991; 20:373–378.	V	12	RCT	Veterinary surgery		CM:Animal living vs dead	ST, SPc, SPd	12.5	6
Stratton SJ, et al. Prospective study of manikin-only versus manikin and human subject endotracheal intubation training of paramedics. Ann Emerg Med. 1991; 20:1314–1318.	EMT	60	RCT	Intubation	ClinV, Mast, RangeD	MA:Simulator, add	P	11	3
Trooskin SZ, et al. Teaching endotracheal intubation using animals and cadavers. Prehospital Disaster Med. 1992; 7:179–182.	EMT	19	RCT	Intubation	CogI, Indiv	CM:Manikin vs animal	SPd, P	14.5	4
Bhat BV, et al. Undergraduate training in neonatal resuscitation – a modified approach. Indian J Matern Child Health. 1993; 4:87–88.	MS	110		Resuscitation (BLS,ACLS,ATLS)	DistP	ID:Timing	K	11.5	4
Holmberg DL, et al. Use of a dog abdominal surrogate for teaching surgery. Journal of Veterinary Medical Education. 1993; 20:61–62.	V	116X		Veterinary surgery	ClinV	CM:Model vs animal	Sa	7.5	2
Mazzuca SA, et al. Improved training of house officers in a rheumatology consult service. Arthritis Care Res. 1993; 6:59–63.	PG	11		Physical exam	MLS	ID:Add reminder	BP	12	2
Campbell HS, et al. Teaching medical students how to perform a clinical breast examination. Acad Med. 1994; 69:993–995.	MS	54	RCT	Physical exam	CogI, FB, Indiv, MLS, Time	In:Instructor training/experience	SPd	11.5	3
From RP, et al. Assessment of an interactive learning system with “sensorized” manikin head for airway management instruction. Anesth Analg. 1994; 79:136–142.	MS	97	RCT	Intubation	MLS	In:Self-instruction	K, BP	13	6
Reader CM, et al. Anatomical artificial teeth for teaching preclinical endodontics. Journal Dental Education. 1994; 58:229–232.	D	29X		Dentistry		CM:Model vs cadaver	Sa, SPd	7.5	2
Greenfield CL, et al. Comparison of surgical skills of veterinary students trained using models or live animals. J Am Vet Med Assoc. 1995; 206:1840–1845.	V	36	RCT	Open surgery/suturing	ClinV	CM:Model vs animal	SPc	11.5	4
van Stralen DW, et al. Retrograde intubation training using a mannequin. Am J Emerg Med. 1995; 13:50–52.	RN, EMT, O	88		Intubation	FB, Indiv	In:Self-instruction	ST	11.5	2
Olsen D, et al. Evaluation of a hemostasis model for teaching basic surgical skills. Vet Surg. 1996; 25:49–58.	V	40	RCT	Open surgery/suturing	RepP	CM:Model vs animal	SPc, SPd	13.5	5
Thomas WE, et al. A preliminary evaluation of an innovative synthetic soft tissue simulation module (‘Skilltray’) for use in basic surgical skills workshops. Ann R Coll Surg Engl. 1996; 78:268–271.	PG	9X		Open surgery/suturing		CM:Synthetic vs human/animal products	Sa	7	2
Long NK, et al. A comparison of two teaching simulations in preclinical operative dentistry. Oper Dent. 1997; 22:133–137.	D	87	RCT	Dentistry		CM:Synthetic vs human/animal products	ST, SPd	14.5	6
Noordergraaf GJ, et al. Learning cardiopulmonary resuscitation skills: does the type of mannequin make a difference? Eur J Emerg Med. 1997; 4:204–209.	MS	104		Resuscitation (BLS,ACLS,ATLS)	FB, GrpP	CM:Manikin vs manikin	SPc, SPd	12.5	4
Roberts I, et al. Airway management training using the laryngeal mask airway: a comparison of two different training programmes. Resuscitation. 1997; 33:211–214.	RN	52	RCT	Intubation	ClinV, CogI, MLS, RangeD	MA:Patient/SP experience, compare	SPd	12.5	3
Ali J, et al. Effect of the Advanced Trauma Life Support program on medical students’ performance in simulated trauma patient management. Journal of Trauma: Injury Infection & Critical Care. 1998; 44:588–591.	MS	44		Resuscitation (BLS,ACLS,ATLS)	CogI, FB, Indiv, MLS, RepP	ID:Hands on	SPc	10.5	2
Christenson J, et al. A comparison of multimedia and standard advanced cardiac life support learning. Acad Emerg Med. 1998; 5:702–708.	MS	113		Resuscitation (BLS,ACLS,ATLS)	ClinV, CogI, FB, Indiv	MA:CAI, compare	K, SPc	13.5	4
Kaczorowski J, et al. Retention of neonatal resuscitation skills and knowledge: a randomized controlled trial. Fam Med. 1998; 30:705–711.	PG	27	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB, Indiv, MLS, Time	In:Self-instruction	K, SPc	12	6
Rogers DA, et al. Computer-assisted learning versus a lecture and feedback seminar for teaching a basic surgical technical skill. Am J Surg. 1998; 175:508–510.	MS	82	RCT	Open surgery/suturing	CogI, FB, MLS	MA:CAI, compare	ST, SPc, SPd	13.5	5
Todd KH, et al. Randomized, controlled trial of video self-instruction versus traditional CPR training. Ann Emerg Med. 1998; 31:364–369.	MS	89	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB, GrpP, Indiv, Time	In:Self-instruction	K, SPc	13.5	6
Ali J, et al. Comparison of performance of interns completing the old (1993) and new interactive (1997) Advanced Trauma Life Support courses. Journal of Trauma-Injury Infection & Critical Care. 1999; 46:80–86.	PG	32	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, MLS	ID:Interactivity	Sa, K, SPc	12.5	4
Anastakis DJ, et al. Assessment of technical skills transfer from the bench training model to the human model. Am J Surg. 1999; 177:167–170.	PG	23X	RCT	Open surgery/suturing		CM:Model vs cadaver	SPc	12.5	5
Burdea G, et al. Virtual reality-based training for the diagnosis of prostate cancer. IEEE Trans Biomed Eng. 1999; 46:1253–1260.	PG	8		Physical exam	CogI, FB	CM:VR vs model	SPd	6.5	1
Gallagher AG, et al. Virtual reality training in laparoscopic surgery: a preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy. 1999; 31:310–313.	O	16	RCT	Min. invasive surg.	CogI, DistP, FB, Indiv, RangeD	CM:VR vs model	SPc	12.5	3
Griffon DJ, et al. Evaluation of a hemostasis model for teaching ovariohysterectomy in veterinary surgery. Vet Surg. 2000; 29:309–316.	V	40	RCT	Open surgery/suturing	RepP	CM:Model vs cadaver	K, ST, SPc	13.5	6
Jordan J-A, et al. A comparison between randomly alternating imaging, normal laparoscopic imaging, and virtual reality training in laparoscopic psychomotor skill acquisition. Am J Surg. 2000; 180:208–211.	O	16	RCT	Min. invasive surg.	CogI, FB, Indiv, RangeD	CM:VR vs model	SPc	10.5	3
Keyser EJ, et al. A simplified simulator for the training and evaluation of laparoscopic skills. Surg Endosc. 2000; 14:149–153.	PG	22X	RCT	Min. invasive surg.		CM:Model vs model	SPc	12.5	4
Kovacs G, et al. A randomized controlled trial on the effect of educational interventions in promoting airway management skill maintenance. Ann Emerg Med. 2000; 36:301–309.	MS, D, O	53	RCT	Intubation	CogI, FB, Indiv	ID:Repetition	SPc	14.5	5
Jordan JA, et al. Virtual reality training leads to faster adaptation to the novel psychomotor restrictions encountered by laparoscopic surgeons. Surg Endosc. 2001; 15:1080–1084.	MS, PG, O	24	RCT	Min. invasive surg.	CogI, FB, Indiv, RangeD	CM:VR vs model	SPc	13.5	4
Pugh CM, et al. The effect of simulator use on learning and self-assessment: The case of Stanford University's E-Pelvis simulator. Stud Health Technol Inform. 2001; 81:396–400.	MS	59	RCT	Physical exam	CogI	ID:Feedback	SPc	11.5	3
Risucci D, et al. The Effects of Practice and Instruction on Speed and Accuracy during Resident Acquisition of Simulated Laparoscopic Skills. Curr Surg. 2001; 58:230–235.	PG	14	RCT	Min. invasive surg.	CogI, Indiv, MLS	In:Instructor intensity	ST, SPc	11.5	3
Torkington J, et al. Skill transfer from virtual reality to a real laparoscopic task. Surg Endosc. 2001; 15:1076–1079.	MS	20	RCT	Min. invasive surg.	CogI, FB, Indiv	CM:VR vs model	ST, SPc	12.5	4
Agazio JB, et al. Evaluation of a virtual reality simulator in sustainment training. Mil Med. 2002; 167:893–897.	MD, RN, EMT	51	RCT	Venous access	ClinV, CogI, FB, Indiv	CM:VR vs model	Sa, ST, SPc	11.5	3
Block EF, et al. Use of a human patient simulator for the advanced trauma life support course. Am Surg. 2002; 68:648–651.	O	14X		Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs animal	Sa	7	2
Byrne AJ, et al. Effect of videotape feedback on anaesthetists’ performance while managing simulated anaesthetic crises: a multicentre study. Anaesthesia. 2002; 57:176–179.	PG	32	RCT	Anesthesia	CogI, Indiv, MLS	ID:Feedback	ST, SPc	11.5	4
Chang KK, et al. Learning intravenous cannulation: a comparison of the conventional method and the CathSim Intravenous Training System. J Clin Nurs. 2002; 11:73–78.	RN, RN	28	RCT	Venous access	FB	CM:VR vs model	BP, P	16	4
Clancy JM, et al. A comparison of student performance in a simulation clinic and a traditional laboratory environment: three-year results. J Dent Educ. 2002; 66:1331–1337.	D	186X		Dentistry		CM:Manikin vs model	SPd	12.5	4
Hamilton EC, et al. Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surg Endosc. 2002; 16:406–411.	PG	19	RCT	Min. invasive surg.	CogI, FB, Indiv, RangeD	CM:VR vs model	Sa, ST, SPc, BP	14	5
Harold KL, et al. Prospective randomized evaluation of surgical resident proficiency with laparoscopic suturing after course instruction. Surg Endosc. 2002; 16:1729–1731.	PG	17	RCT	Min. invasive surg.	CogI, FB, MLS	In:Self-instruction	ST, SPc	11.5	3
Kothari SN, et al. Training in laparoscopic suturing skills using a new computer-based virtual reality simulator (MIST-VR) provides results comparable to those with an established pelvic trainer system. J Laparoendosc Adv Surg Tech. 2002; 12:167–173.	MS	24	RCT	Min. invasive surg.	CogI, FB, Indiv, RangeD	CM:VR vs model	ST	12.5	4
Matsumoto ED, et al. The effect of bench model fidelity on endourological skills: a randomized controlled study. J Urol. 2002; 167:1243–1247.	MS	33	RCT	Endoscopy (GI,Urology,Bronch.)		CM:Manikin vs model	ST, SPc, SPd	14.5	5
Scaringe JG, et al. The effects of augmented sensory feedback precision on the acquisition and retention of a simulated chiropractic task. J Manipulative Physiol Ther. 2002; 25:34–41.	C	71	RCT	Physical exam	FB	ID:Feedback	SPc	11.5	4
Allen J, et al. A teaching tool in spinal anesthesia. AANA J. 2003; 71:29–36.	RN	26X	RCT	Percutaneous proc.	CogI, FB	ID:Feedback	ST, SPd	12.5	3
Ameur S, et al. Learning bronchoscopy in simulator improved dexterity rather than judgement [Swedish]. Lakartidningen. 2003; 100:2694–2699.	MS	20		Endoscopy (GI,Urology,Bronch.)	CogI	ID:Clinical scenario	Sa, SPc	10.5	1
Engum SA, et al. Intravenous catheter training system: computer-based education versus traditional learning methods. Am J Surg. 2003; 186:67–74.	MS, RN	163		Venous access	CogI, FB	CM:VR vs model	Sa, K, SPc	11.5	4
Gerling GJ, et al. Effect of augmented visual performance feedback on the effectiveness of clinical breast examination training with a dynamically configurable breast model. Conf Proc IEEE Int Conf Syst Man Cybern. 2003; 3:2095–2100.	MS	6		Physical exam	CogI, FB	ID:Feedback	SPc	11	2
Gerling GJ, et al. Effectiveness of a dynamic breast examination training model to improve clinical breast examination (CBE) skills. Cancer Detect Prev. 2003; 27:451–456.	MS	48	RCT	Physical exam	ClinV, CogI, MLS, RangeD	ID:Task variability	SPd	12.5	4
Gormley GJ, et al. A randomised study of two training programmes for general practitioners in the techniques of shoulder injection. Ann Rheum Dis. 2003; 62:1006–1009.	MD	38	RCT	Percutaneous proc.	ClinV, CogI, MLS, RangeD, Time	MA:Patient/SP experience, add	SPc, BP	12	4
Katz R, et al. Cadaveric versus porcine models in urological laparoscopic training. Urulogia Internationalis. 2003; 71:310–315.	PG, MD	16		Min. invasive surg.		CM:Cadaver vs animal	Sa	7	1
Lee SK, et al. Trauma assessment training with a patient simulator: a prospective, randomized study. Journal of Trauma: Injury Infection & Critical Care. 2003; 55:651–657.	PG	60	RCT	Resuscitation (BLS,ACLS,ATLS)		MA:Patient/SP experience, compare	SPc	14	3
Quinn F, et al. A pilot study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students. Eur J Dent Educ. 2003; 7:13–19.	D	32	RCT	Dentistry	ClinV, MLS	CM:VR vs model	SPd	14.5	5
Quinn F, et al. A study comparing the effectiveness of conventional training and virtual reality simulation in the skills acquisition of junior dental students. Eur J Dent Educ. 2003; 7:164–9.	D	20	RCT	Dentistry	FB	In:Self-instruction	SPd	13.5	3
Buchanan JA. Experience with virtual reality-based technology in teaching restorative dental procedures. J Dent Educ. 2004; 68:1258–1265.	D	44	RCT	Dentistry	FB, Time	CM:VR vs model	SPd	12.5	5
Grober ED, et al. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg. 2004; 240:374–381.	PG	40	RCT	Microsurgery/Ophthalmology		CM:Synthetic vs human/animal products	ST, SPc, SPd	13.5	5
Hochberger J, et al. The compact Erlangen Active Simulator for Interventional Endoscopy: a prospective comparison in structured team-training courses on ‘endoscopic hemostasis’ for doctors and nurses. Scand J Gastroenterol. 2004; 39:895–902.	PG, MD, RN, O	207		Endoscopy (GI,Urology,Bronch.)		CM:Model vs model	Sa	10	2
Jasinevicius TR, et al. An evaluation of two dental simulation systems: virtual reality versus contemporary non-computer-assisted. J Dent Educ. 2004; 68:1151–1162.	D	28	RCT	Dentistry	ClinV, FB	CM:VR vs model	SPd	13.5	4
Kim HK, et al. Virtual-reality-based laparoscopic surgical training: The role of simulation fidelity in haptic feedback. Comput Aided Surg. 2004; 9:227–234.	O	16		Min. invasive surg.		SA:Tactile	SPc	11.5	2
Martin KM, et al. Effective nonanatomical endoscopy training produces clinical airway endoscopy proficiency. Anesth Analg. 2004; 99:938–44.	PG, MD, O	40	RCT	Endoscopy (GI,Urology,Bronch.)		CM:Model vs model	ST, SPc	14.5	5
Munz Y, et al. Laparoscopic virtual reality and box trainers: is one superior to the other? Surg Endosc. 2004; 18:485–494.	MS	16	RCT	Min. invasive surg.		CM:VR vs model	ST, SPc	12.5	5
Nishida M, et al. Training in tooth preparation utilizing a support system. J Oral Rehabil. 2004; 31:149–154.	D	10	RCT	Dentistry		MA:External support, add	SPd	10.5	4
Rumball C, et al. Endotracheal intubation and esophageal tracheal Combitube insertion by regular ambulance attendants: A comparative trial. Prehosp Emerg Care. 2004; 8:15–22.	EMT	61		Intubation	DistP, Indiv, Time	ID:Repetition	P	14	2
St. Pierre M, et al. Simulator-based modular human factor training in anesthesiology. Concept and results of the module “Communication and Team Cooperation” [German]. Anaesthesist. 2004; 53:144–152.	MD	34	RCT	Critical thinking	CogI, MLS	MA:Discussion, add	Sa, SPc	12.5	3
Backstein D, et al. Effectiveness of repeated video feedback in the acquisition of a surgical technical skill. Can J Surg. 2005; 48:195–200.	PG	26	RCT	Open surgery/suturing	CogI, FB, MLS	ID:Feedback	SPc	15.5	3
Bathalon S, et al. Cognitive skills analysis, kinesiology, and mental imagery in the acquisition of surgical skills. J Otolaryngol. 2005; 34:328–332.	MS	31	RCT	Resuscitation (BLS,ACLS,ATLS),Percutaneous proc.	CogI, FB, MLS, RepP	ID:Teach cognition	SPc	12.5	4
Bowyer CM, et al. Validation of SimPL – a simulator for diagnostic peritoneal lavage training. Stud Health Technol Inform. 2005; 111:64–67.	MS	40	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:VR vs animal	K, SPc	12.5	4
Bowyer MW, et al. Teaching intravenous cannulation to medical students: comparative analysis of two simulators and two traditional educational approaches. Stud Health Technol Inform. 2005; 111:57–63.	MS	14	RCT	Venous access	ClinV, RangeD	CM:VR vs VR	SPc	13.5	4
Donnon T, et al. Impact of cognitive imaging and sex differences on the development of laparoscopic suturing skills. Can J Surg. 2005; 48:387–393.	MS	42	RCT	Min. invasive surg.	CogI, MLS	ID:Teach cognition	ST	12.5	3
Dubrowski A, et al. The influence of practice schedules in the learning of a complex bone-plating surgical task. Am J Surg. 2005; 190:359–363.	MS	19	RCT	Open surgery/suturing		ID:Sequence	SPc, SPd	12.5	3
Enebo B, et al. Experience and practice organization in learning a simulated high-velocity low-amplitude task. J Manipulative Physiol Ther. 2005; 28:33–43.	C	33X		Spinal manipulation		ID:Sequence	SPc	12.5	2
Gerling GJ, et al. Augmented, pulsating tactile feedback facilitates simulator training of clinical breast examinations. Hum Factors. 2005; 47:670–681.	MS	48	RCT	Physical exam	Indiv, MLS, RangeD	ID:Task variability	SPd	13.5	3
Kiesslich R, et al. Combined simulation training: a new concept and workshop is useful for crisis management in gastrointestinal endoscopy [German]. Z Gastroenterol. 2005; 43:1031–1039.	MD	100X		Endoscopy (GI,Urology,Bronch.),Team training	CogI, FB, GrpP, MLS, RepP	CM:VR vs animal	Sa	11	3
Kirlum HJ, et al. Advanced paediatric laparoscopic surgery: repetitive training in a rabbit model provides superior skills for live operations. Eur J Pediatr Surg. 2005; 15:149–152.	PG	12	RCT	Min. invasive surg.		CM:Model vs animal	ST	9.5	2
Lehmann KS, et al. A prospective randomized study to test the transfer of basic psychomotor skills from virtual reality to physical reality in a comparable training setting. Ann Surg. 2005; 241:442–449.	MS, MD	32	RCT	Min. invasive surg.		CM:VR vs model	ST, SPc	12.5	2
Lentz GM, et al. A six-year study of surgical teaching and skills evaluation for obstetric/gynecologic residents in porcine and inanimate surgical models. Am J Obstet Gynecol. 2005; 193:2056–2061.	PG	27		Min. invasive surg.,Open surgery/suturing		ID:Timing	K, SPc	14.5	4
Madan AK, et al. Participants’ opinions of laparoscopic training devices after a basic laparoscopic training course. Am J Surg. 2005; 189:758–61.	MS	18X		Min. invasive surg.		CM:VR vs model	Sa	8	1
Monsieurs KG, et al. Improved basic life support performance by ward nurses using the CAREvent® Public Access Resuscitator (PAR) in a simulated setting. Resuscitation. 2005; 67:45–50.	RN	152	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB	MA:External support, add	SPc	12.5	5
Tani Botticelli A, et al. The effectiveness of video support in the teaching of manual skills related to initial periodontal therapy tested on phantoms. Int J Comput Dent. 2005; 8:117–127.	O	84	RCT	Dentistry	CogI	MA:CAI, compare	SPc	12.5	3
Uchal M, et al. Validation of a six-task simulation model in minimally invasive surgery. Surg Endosc. 2005; 19:109–16.	PG	17	RCT	Min. invasive surg.		ID:Sequence	SPc	15.5	6
Wierinck E, et al. Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training. Eur J Dent Educ. 2005; 9:10–16.	D	24	RCT	Dentistry	FB, Indiv	ID:Feedback	ST, SPd	12.5	4
Youngblood PL, et al. Comparison of training on two laparoscopic simulators and assessment of skills transfer to surgical performance. J Am Coll Surg. 2005; 200:546–551.	MS	33	RCT	Min. invasive surg.	CogI, FB	CM:VR vs model	ST, SPc	13.5	4
Aggarwal R, et al. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg. 2006; 191:128–133.	MS	20	RCT	Min. invasive surg.	RangeD, Time	ID:Sequence	ST	12.5	5
Berkenstadt H, et al. Evaluation of the Trauma-Man® simulator for training in chest drain insertion. European Journal of Trauma. 2006; 32:523–526.	PG	42X	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs animal	Sa	8	4
Bond WF, et al. Cognitive versus technical debriefing after simulation training. Acad Emerg Med. 2006; 13:276–283.	PG	62	RCT	Critical thinking		ID:Teach cognition	Sa	10	4
Chandrasekera SK, et al. Basic laparoscopic surgical training: examination of a low-cost alternative. Eur Urol. 2006; 50:1285–1291.	MS	36	RCT	Min. invasive surg.		CM:Model vs model	ST, SPc, SPd	12.5	5
Chou DS, et al. Comparison of results of virtual-reality simulator and training model for basic ureteroscopy training. J Endourol. 2006; 20:266–271.	MS	16	RCT	Endoscopy (GI,Urology,Bronch.)	RangeD	CM:VR vs model	SPc	12.5	4
Crofts JF, et al. Training for shoulder dystocia: a trial of simulation using low-fidelity and high-fidelity mannequins. Obstet Gynecol. 2006; 108:1477–1485.	PG, MD, O	110	RCT	Obstetrics	CogI, FB, Indiv, MLS	CM:Manikin vs manikin	ST, SPc, SPd	13.5	5
Cummings AJ, et al. Evaluation of a novel animal model for teaching intubation. Teach Learn Med. 2006; 18:316–319.	MS, PG, MD, RN, EMT, O	42X	RCT	Intubation		CM:Model vs animal	Sa, ST	8.5	2
Dubrowski A, et al. Randomised, controlled study investigating the optimal instructor: student ratios for teaching suturing skills. Med Educ. 2006; 40:59–63.	MS	72	RCT	Open surgery/suturing	CogI, FB	In:Instructor intensity	ST, SPc	12.5	4
Halvorsen FH, et al. Virtual reality simulator training equals mechanical robotic training in improving robot-assisted basic suturing skills. Surg Endosc. 2006; 20:1565–1569.	MS	26	RCT	Robotic surg.		CM:VR vs model	Sa, SPc	11.5	2
Heinrich M, et al. Comparison of different training models for laparoscopic surgery in neonates and small infants. Surg Endosc. 2006; 20:641–644.	PG	12	RCT	Min. invasive surg.		CM:Model vs animal	SPc	12.5	2
Jamison RJ, et al. A pilot study assessing simulation using two simulation methods for teaching intravenous cannulation. Clinical Simulation in Nursing Education. 2006; 2:e9–e12.	RN	18	RCT	Venous access		CM:VR vs model	K	14.5	3
Kimura T, et al. Usefulness of a virtual reality simulator or training box for endoscopic surgery training. Surg Endosc. 2006; 20:656–659.	MS	12		Min. invasive surg.		CM:VR vs model	ST, SPc	11.5	2
Mazilu D, et al. Synthetic torso for training in and evaluation of urologic laparoscopic skills. J Endourol. 2006; 20:340–345.	MS, PG, MD, O	25	RCT	Min. invasive surg.		CM:Model vs model	Sa	8	3
Ocel JJ, et al. Formal procedural skills training using a fresh frozen cadaver model: a pilot study. Clin Anat. 2006; 19:142–146.	MS	7X		Venous access,Percutaneous proc.,Natural orifice proc.		CM:Manikin vs cadaver	Sa	7	2
Owen H, et al. Comparison of three simulation-based training methods for management of medical emergencies. Resuscitation. 2006; 71:204–211.	PG	50	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, Indiv, MLS	CM:Manikin vs manikin	SPc	13	5
Panait L, et al. Telementoring versus on-site mentoring in virtual reality-based surgical training. Surg Endosc. 2006; 20:113–118.	MS	20	RCT	Min. invasive surg.		In:Distance supervision	ST, SPc	13	4
Rosenthal ME, et al. Achieving housestaff competence in emergency airway management using scenario based simulation training: comparison of attending vs housestaff trainers. Chest. 2006; 129:1453–1458.	PG	49	RCT	Resuscitation (BLS,ACLS,ATLS)		In:Instructor training/experience	SPc	12.5	4
Rosser JC, Jr., et al. The use of a “hybrid” trainer in an established laparoscopic skills program. J Soc Laparoendosc Surg. 2006; 10:4–10.	PG, MD	817		Min. invasive surg.	FB, Indiv, MLS	ID:Feedback	ST	10.5	1
Savoldelli GL, et al. Value of debriefing during simulated crisis management: oral versus video-assisted oral feedback. Anesthesiology. 2006; 105:279–285.	PG	28	RCT	Anesthesia,Team training	CogI, FB, Indiv, MLS	ID:Feedback	SPc	14.5	5
Scerbo MW, et al. Comparison of a virtual reality simulator and simulated limbs for phlebotomy training. J Infus Nurs. 2006; 29:214–224.	MS	16		Venous access	ClinV, RangeD, Time	CM:VR vs model	SPc, BP	12	2
Scerbo MW, et al. The efficacy of a medical virtual reality simulator for training phlebotomy. Hum Factors. 2006; 48:72–84.	MS	20	RCT	Venous access	ClinV, RangeD	CM:VR vs model	SPc	12.5	3
Stefanidis D, et al. Proficiency maintenance: impact of ongoing simulator training on laparoscopic skill retention. J Am Coll Surg. 2006; 202:599–603.	MS	18	RCT	Min. invasive surg.	CogI, Time	ID:Repetition	SPc	11.5	3
Strom P, et al. Early exposure to haptic feedback enhances performance in surgical simulator training: a prospective randomized crossover study in surgical residents. Surg Endosc. 2006; 20:1383–1388.	PG	19X	RCT	Min. invasive surg.		SA:Tactile	SPc	11.5	3
Ti L-K, et al. The impact of experiential learning on NUS medical students: our experience with task trainers and human-patient simulation. Ann Acad Med Singapore. 2006; 35:619–623.	MS	54		Anesthesia	CogI	ID:Hands on	SPc	11.5	3
Verdaasdonk EGG, et al. Validation of a new basic virtual reality simulator for training of basic endoscopic skills: The SIMENDO. Surg Endosc. 2006; 20:511–518.	MS	16	RCT	Min. invasive surg.		CM:VR vs model	ST, SPc	13.5	4
Wierinck E, et al. Effect of reducing frequency of augmented feedback on manual dexterity training and its retention. J Dent. 2006; 34:641–647.	D	24	RCT	Dentistry	FB, MLS	ID:Feedback	ST, SPd	12.5	4
Wierinck E, et al. Effect of tutorial input in addition to augmented feedback on manual dexterity training and its retention. Eur J Dent Educ. 2006; 10:24–31.	D	24	RCT	Dentistry	FB, MLS	In:Self-instruction	ST, SPd	12.5	4
Adermann J, et al. The impact of force feedback on training of surgical skills in virtual neuroendoscopy. Int J Comput Assist Radiol Surg. 2007; 2(Suppl 1):S198–S200.	MS, MD	60	RCT	Endoscopy (GI,Urology,Bronch.)	FB	SA:Tactile	ST, SPc	11.5	3
Berry M, et al. Porcine transfer study: virtual reality simulator training compared with porcine training in endovascular novices. Cardiovasc Intervent Radiol. 2007; 30:455–461.	MD	12X	RCT	Endovascular proc.		CM:VR vs animal	Sa, SPc	12.5	2
Birch L, et al. Obstetric skills drills: evaluation of teaching methods. Nurse Educ Today. 2007; 27:915–922.	PG, O	4	RCT	Obstetrics	MLS	ID:Blending	SPc	13.5	5
Bruynzeel H, et al. Desktop simulator: key to universal training? Surg Endosc. 2007; 21:1637–1640.	MS	20	RCT	Min. invasive surg.		CM:Model vs model	ST, SPd	12.5	4
Chang J-Y, et al. Effectiveness of two forms of feedback on training of a joint mobilization skill by using a joint translation simulator. Phys Ther. 2007; 87:418–430.	O	24	RCT	Physical therapy	CogI, FB, Indiv, MLS	ID:Feedback	SPc	12.5	5
Cherry RA, et al. The effectiveness of a human patient simulator in the ATLS shock skills station. J Surg Res. 2007; 139:229–235.	PG	44	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs model	K, SPc	13.5	4
Crofts JF, et al. Change in knowledge of midwives and obstetricians following obstetric emergency training: a randomised controlled trial of local hospital, simulation centre and teamwork training. BJOG. 2007; 114:1534–1541.	PG, MD, O	133	RCT	Obstetrics	CogI, FB, Indiv, MLS	CM:Manikin vs manikin	K	14.5	5
Davis DP, et al. The effectiveness of a novel, algorithm-based difficult airway curriculum for air medical crews using human patient simulators. Prehosp Emerg Care. 2007; 11:72–79.	RN, EMT	120X		Intubation	ClinV, MLS, Time	MA:Simulator, add	SPc, P	14	2
Dubrowski A, et al. A comparison of single- and multiple-stage approaches to teaching laparoscopic suturing. Am J Surg. 2007; 193:269–273.	PG	24	RCT	Min. invasive surg.	MLS, RangeD	ID:Task variability	ST, SPc, SPd	13.5	3
Gutierrez F, et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform. 2007; 125:155–160.	MS	25	RCT	Resuscitation (BLS,ACLS,ATLS),Physical exam		CM:VR vs VR	K	13.5	5
Heinrichs WL, et al. Criterion-based training with surgical simulators: proficiency of experienced surgeons. J Soc Laparoendosc Surg. 2007; 11:273–302.	MD	17X	RCT	Min. invasive surg.		CM:VR vs VR	Sa	8	4
Iglesias-Vazquez JA, et al. Cost-efficiency assessment of Advanced Life Support (ALS) courses based on the comparison of advanced simulators with conventional manikins. BMC Emergency Medicine. 2007; 7:18.	MD, RN	250	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs manikin	SPc	12	2
Immenroth M, et al. Mental training in surgical education: a randomized controlled trial. Ann Surg. 2007; 245:385–391.	MD	66	RCT	Min. invasive surg.	CogI, MLS	ID:Teach cognition	SPc	13.5	6
Kahol K, et al. Augmented virtual reality for laparoscopic surgical tool training. Lect Notes Comput Sci. 2007; 4553 LNCS:459–467.	PG	8		Min. invasive surg.	CogI, FB	ID:Feedback	SPc	11.5	2
Lazarski MP, et al. How do feedback and instructions affect the performance of a simulated surgical task? J Oral Maxillofac Surg. 2007; 65:1155–61.	MS, D	61		Open surgery/suturing	MLS	ID:Instructions	SPd	11.5	3
LeFlore JL, et al. Comparison of self-directed learning versus instructor-modeled learning during a simulated clinical experience. Simul Healthc. 2007; 2:170–177.	RN	11		Resuscitation (BLS,ACLS,ATLS),Team training	CogI, FB, Indiv	In:Self-instruction	K, ST, SPc	13.5	5
Lee JC, et al. Randomized controlled trial of an instructional DVD for clinical skills teaching. Emerg Med Australas. 2007; 19:241–245.	MS, PG, MD, RN	36	RCT	Venous access	CogI, FB, Indiv	MA:Lecture, compare	Sa, SPc	12.5	5
Madan AK, et al. Prospective randomized controlled trial of laparoscopic trainers for basic laparoscopic skills acquisition. Surg Endosc. 2007; 21:209–213.	MS	32	RCT	Min. invasive surg.		MA:Simulator, add	ST, SPc	12.5	5
Neequaye SK, et al. Identification of skills common to renal and iliac endovascular procedures performed on a virtual reality simulator. Eur J Vasc Endovasc Surg. 2007; 33:525–532.	PG	20	RCT	Endovascular proc.	CogI	ID:Task variability	ST, SPc	11.5	4
Schlosser K, et al. Training of laparoscopic skills with virtual reality simulator: a critical reappraisal of the learning curve. European Surgery – Acta Chirurgica Austriaca. 2007; 39:180–184.	MS	14	RCT	Min. invasive surg.		ID:Task variability	ST, SPc	12.5	4
Sidhu RS, et al. Laboratory-based vascular anastomosis training: a randomized controlled trial evaluating the effects of bench model fidelity and level of training on skill acquisition. J Vasc Surg. 2007; 45:343–349.	PG	27	RCT	Open surgery/suturing		CM:Model vs cadaver	ST, SPc, SPd	12.5	4
Spooner BB, et al. An evaluation of objective feedback in basic life support (BLS) training. Resuscitation. 2007; 73:417–424.	O	98	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB	ID:Feedback	SPc, SPd	13.5	5
Stefanidis D, et al. Closing the gap in operative performance between novices and experts: does harder mean better for laparoscopic simulator training? J Am Coll Surg. 2007; 205:307–313.	MS	25	RCT	Min. invasive surg.	CogI, Time	ID:Stress	SPc	12.5	4
Stefanidis D, et al. Construct and face validity and task workload for laparoscopic camera navigation: virtual reality versus videotrainer systems at the SAGES Learning Center. Surg Endosc. 2007; 21:1158–1164.	PG, MD, O	90X		Min. invasive surg.		CM:VR vs model	Sa, SPc	12.5	3
Stefanidis D, et al. Limited feedback and video tutorials optimize learning and resource utilization during laparoscopic simulator training. Surgery. 2007; 142:202–206.	MS	22		Min. invasive surg.	FB, Indiv	ID:Feedback	SPc	11.5	1
Thomas EJ, et al. Teaching teamwork during the Neonatal Resuscitation Program: a randomized trial. J Perinatol. 2007; 27:409–414.	PG	32	RCT	Team training	MLS	MA:Team training, add	SPc	13.5	5
Torgerson C, et al. Low fidelity simulation of temporal bone drilling leads to improved but suboptimal outcomes. Stud Health Technol Inform. 2007; 125:470–472.	MS	24		Open surgery/suturing		ID:Task variability	SPd	9.5	1
Van Sickle KR, et al. The effect of escalating feedback on the acquisition of psychomotor skills for laparoscopy. Surg Endosc. 2007; 21:220–224.	MS, O	16	RCT	Min. invasive surg.		ID:Feedback	SPc	13.5	4
Verdaasdonk EGG, et al. The influence of different training schedules on the learning of psychomotor skills for endoscopic surgery. Surg Endosc. 2007; 21:214–219.	MS, O	20	RCT	Min. invasive surg.	DistP	ID:Timing	ST, SPc	11.5	3
Welk A, 21et al. Mental training in dentistry. Quintessence Int. 2007; 38:489–497.	D	41	RCT	Dentistry	MLS	ID:Teach cognition	K, SPc, SPd	13.5	5
Xeroulis GJ, et al. Teaching suturing and knot-tying skills to medical students: a randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery. 2007; 141:442–449.	MS	30	RCT	Open surgery/suturing	CogI, FB, Indiv, MLS	MA:CAI, compare	SPc	13.5	4
Baranauskas MB, et al. Simulation of ultrasound-guided peripheral nerve block: learning curve of CET-SMA/HSL Anesthesiology residents. Rev Bras Anestesiol. 2008; 58:106–111.	PG	6	RCT	Anesthesia,Radiology/other noninvasive dx	Time	ID:Repetition	ST, SPc	9.5	4
Bingener J, et al. Randomized double-blinded trial investigating the impact of a curriculum focused on error recognition on laparoscopic suturing training. Am J Surg. 2008; 195:179–182.	MS	30	RCT	Min. invasive surg.	MLS	ID:Feedback	ST, SPc	15.5	4
Botden SMBI, et al. The importance of haptic feedback in laparoscopic suturing training and the additive value of virtual reality simulation. Surg Endosc. 2008; 22:1214–1222.	PG, MD	45X	RCT	Min. invasive surg.		ID:Sequence	Sa, SPc, SPd	12	4
Chandra DB, et al. Fiberoptic oral intubation: the effect of model fidelity on training for transfer to patient care. Anesthesiology. 2008; 109:1007–1013.	O	28	RCT	Endoscopy (GI,Urology,Bronch.)		CM:VR vs model	BT, BP, P	16	5
Chang S, et al. Verbal communication improves laparoscopic team performance. Surgical Innovation. 2008; 15:143–147.	MS, PG, O	24	RCT	Min. invasive surg.	CogI, FB, Indiv	ID:Feedback	ST, SPc	12.5	2
Chmarra MK, et al. Force feedback and basic laparoscopic skills. Surg Endosc. 2008; 22:2140–2148.	PG	19	RCT	Min. invasive surg.		SA:Tactile	ST, SPc	11.5	4
Cho J, et al. Comparison of manikin versus porcine models in cricothyrotomy procedure training. Emerg Med J. 2008; 25:732–734.	PG, MD, RN, EMT	49X	RCT	Open surgery/suturing,Intubation		CM:Model vs animal	Sa	9	2
Crofts JF, et al. Patient-actor perception of care: a comparison of obstetric emergency training using manikins and patient-actors. Qual Saf Health Care. 2008; 17:20–24.	MD, O	64	RCT	Obstetrics	CogI, FB, Indiv	CM:Manikin vs manikin	SPc	13.5	4
De Regge M, et al. Basic life support refresher training of nurses: individual training and group training are equally effective. Resuscitation. 2008; 79:283–287.	RN	103	RCT	Resuscitation (BLS,ACLS,ATLS)	FB, GrpP, MLS	GC:Solo vs group	SPc	12.5	6
Dine CJ, et al. Improving cardiopulmonary resuscitation quality and resuscitation training by combining audiovisual feedback and debriefing. Crit Care Med. 2008; 36:2817–2822.	RN	65	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB, Indiv, MLS	ID:Feedback	K, SPc	12.5	5
Ellis D, et al. Hospital, simulation center, and teamwork training for eclampsia management: a randomized controlled trial. Obstet Gynecol. 2008; 111:723–731.	MD, O	24	RCT	Obstetrics		CM:Manikin vs manikin	ST	14.5	5
Friedman Z, et al. Teaching lifesaving procedures: the impact of model fidelity on acquisition and transfer of cricothyrotomy skills to performance on cadavers. Anesth Analg. 2008; 107:1663–1669.	PG	22	RCT	Intubation		CM:Manikin vs model	ST, SPc	12.5	4
Grady JL, et al. Learning nursing procedures: the influence of simulator fidelity and student gender on teaching effectiveness. J Nurs Educ. 2008; 47:403–408.	RN	39X		Natural orifice proc.		CM:Manikin vs model	Sa, SPc	13.5	3
Grodin MH, et al. Ophthalmic surgical training: a curriculum to enhance surgical simulation. Retina. 2008; 28:1509–1514.	PG, MD	45	RCT	Microsurgery/Ophthalmology		MA:CAI, compare	SPc	13.5	4
Isbye DL, et al. Voice advisory manikin versus instructor facilitated training in cardiopulmonary resuscitation. Resuscitation. 2008; 79:73–81.	MS	43	RCT	Resuscitation (BLS,ACLS,ATLS)		In:Self-instruction	SPc	13.5	5
Jiang C, et al. A training model for laparoscopic urethrovesical anastomosis. J Endourol. 2008; 22:1541–1545.	PG	40	RCT	Min. invasive surg.,Endoscopy (GI,Urology,Bronch.)		CM:Animal products vs animal products	ST, SPc	12.5	4
Johnson KB, et al. Part Task and variable priority training in first-year anesthesia resident education: a combined didactic and simulation-based approach to improve management of adverse airway and respiratory events. Anesthesiology. 2008; 108:831–840.	PG	21	RCT	Anesthesia	CogI, FB, MLS	ID:Sequence	K, SPc, SPd	14.5	4
Kanumuri P, et al. Virtual reality and computer-enhanced training devices equally improve laparoscopic surgical skill in novices. J Soc Laparoendosc Surg. 2008; 12:219–226.	MS	16	RCT	Min. invasive surg.		CM:VR vs VR	ST, SPc, SPd	13.5	4
Lammers RL. Learning and retention rates after training in posterior epistaxis management. Acad Emerg Med. 2008; 15:1181–9.	PG	28	RCT	Epistaxis management	FB, MLS, Time	ID:Sequence	ST, SPc	14.5	5
LeFlore JL, et al. Effectiveness of 2 methods to teach and evaluate new content to neonatal transport personnel using high-fidelity simulation. J Perinat Neonatal Nurs. 2008; 22:319–328.	RN, EMT, O	24		Resuscitation (BLS,ACLS,ATLS)	CogI, FB, Indiv	ID:Hands on	Sa, SPc	13.5	2
Low D, et al. The use of the BERCI DCI® Video Laryngoscope for teaching novices direct laryngoscopy and tracheal intubation. Anaesthesia. 2008; 63:195–201.	MS, EMT	42	RCT	Intubation	CogI, FB	SA:Visual	ST, SPd	12.5	5
Miotto HC, et al. Advanced Cardiac Life Support Courses: live actors do not improve training results compared with conventional manikins. Resuscitation. 2008; 76:244–248.	MD, RN, O	225	RCT	Resuscitation (BLS,ACLS,ATLS)	MLS	MA:Patient/SP experience, add	K	12	5
Murphy MA, et al. Should we train the trainers? Results of a randomized trial. J Am Coll Surg. 2008; 207:185–190.	MS	30	RCT	Venous access	CogI, Indiv, MLS	In:Instructor training/experience	ST, SPc	14.5	5
Nousiainen M, et al. Comparison of expert instruction and computer-based video training in teaching fundamental surgical skills to medical students. Surgery. 2008; 143:539–544.	MS	16	RCT	Open surgery/suturing	CogI, FB, Indiv, MLS	ID:Interactivity	ST, SPc	13.5	4
O’Connor A, et al. How much feedback is necessary for learning to suture? Surg Endosc. 2008; 22:1614–1619.	MS	6	RCT	Min. invasive surg.	CogI, FB, Indiv, MLS	ID:Feedback	SPd	11.5	2
Pierce J, et al. Comparative usability studies of full vs. partial immersive virtual reality simulation for medical education and training. Stud Health Technol Inform. 2008; 132:372–377.	MS	25	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:VR vs VR	Sa	10	4
Rafiq A, et al. Objective assessment of training surgical skills using simulated tissue interface with real-time feedback. J Surg Educ. 2008; 65:270–274.	MS	12	RCT	Open surgery/suturing	CogI, FB, Indiv, MLS	ID:Feedback	SPc	11.5	2
Reyes SD, et al. Implementation and evaluation of a virtual simulator system: teaching intravenous skills. Clinical Simulation in Nursing Education. 2008; 4:e43–e49.	RN	28	RCT	Venous access	CogI, DistP	CM:VR vs model	Sa, K, SPc	11.5	2
Rissanen MJ, et al. Asynchronous teaching of psychomotor skills through VR annotations: evaluation in digital rectal examination. Stud Health Technol Inform. 2008; 132:411–416.	MS	8		Physical exam	CogI, FB, Indiv, MLS	ID:Feedback	SPc	10.5	2
Roberson DW, et al. Improving wound care simulation with the addition of odor: A descriptive, quasi-experimental study. Ostomy Wound Management. 2008; 54:36–43.	RN	99		Physical exam,wound care		SA:Olfactory	Sa	9	2
Sanders CW, et al. Learning basic surgical skills with mental imagery: using the simulation centre in the mind. Med Educ. 2008; 42:607–612.	MS	63	RCT	Open surgery/suturing	CogI	ID:Teach cognition	SPc	14.5	3
Tanoue K, et al. Effectiveness of endoscopic surgery training for medical students using a virtual reality simulator versus a box trainer: a randomized controlled trial. Surg Endosc. 2008; 22:985–990.	MS	40	RCT	Min. invasive surg.	RangeD	CM:VR vs model	ST	12.5	4
Tzafestas CS, et al. Pilot evaluation study of a virtual paracentesis simulator for skill training and assessment: the beneficial effect of haptic display. Presence: Teleoperators and Virtual Environments. 2008; 17:212–229.	MD, RN	20	RCT	Venous access	CogI, Indiv	SA:Tactile	ST, SPc	12.5	3
Van Herzeele I, et al. Cognitive training improves clinically relevant outcomes during simulated endovascular procedures. J Vasc Surg. 2008; 48:1223–1230.	PG	20		Endovascular proc.	MLS, Time	MA:Lecture, add	ST, SPd	10.5	2
Vankipuram M, et al. Virtual reality based training to resolve visio-motor conflicts in surgical environments. HAVE 2008 – IEEE International Workshop on Haptic Audio Visual Environments and Games Proceedings. 2008; Article number 4685290:7–12.	PG	10		Min. invasive surg.	RangeD	SA:Tactile	ST, SPc, SPd	11.5	2
Wang XP, et al. Effect of emergency care simulator combined with problem-based learning in teaching of cardiopulmonary resuscitation [Chinese]. Chung Hua I Hsueh Tsa Chih. 2008; 88:1651–1653.	MS	42	RCT	Resuscitation (BLS,ACLS,ATLS),Team training	ClinV, CogI	MA:Simulator, add	SPc	12.5	4
Wheeler DW, et al. Retention of drug administration skills after intensive teaching. Anaesthesia. 2008; 63:379–384.	O	72		Physiology: pharmacology	MLS	MA:CAI, add	SPc	11.5	4
Youngquist ST, et al. Paramedic self-efficacy and skill retention in pediatric airway management. Acad Emerg Med. 2008; 15:1295–303.	EMT	135		Intubation	CogI, FB	In:Self-instruction	SPd	13.5	3
de Vries W, et al. Self–training in the use of automated external defibrillators: the same results for less money. Resuscitation. 2008; 76:76–82.	RN	30	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB, GrpP, Indiv, MLS	In:Self-instruction	SPc	10.5	2
Butler KW, et al. Implementation of active learning pedagogy comparing low-fidelity simulation versus high-fidelity simulation in pediatric nursing education. Clinical Simulation in Nursing. 2009; 5:e129–e136.	RN	30	RCT	Physiology:fluid and electrolyte		CM:Manikin vs manikin	Sa	12	3
Campbell DM, et al. High-fidelity simulation in neonatal resuscitation. Paediatrics and Child Health. 2009; 14:19–23.	PG	15	RCT	Resuscitation (BLS,ACLS,ATLS)	Indiv	CM:Manikin vs manikin	Sa, K, ST	13.5	4
Carter YM, et al. Open lobectomy simulator is an effective tool for teaching thoracic surgical skills. Ann Thorac Surg. 2009; 87:1546–1550.	MS	18		Open surgery/suturing	CogI, DistP, FB, Indiv, MLS, RepP, Time	ID:Repetition	K, ST, SPc	11.5	2
Day T, et al. Effect of performance feedback on tracheal suctioning knowledge and skills: randomized controlled trial. J Adv Nurs. 2009; 65:1423–1431.	RN, EMT	38	RCT	Natural orifice proc.,tracheal suctioning		ID:Feedback	K, SPc	15	4
Deladisma AM, et al. A pilot study to integrate an immersive virtual patient with a breast complaint and breast examination simulator into a surgery clerkship. Am J Surg. 2009; 197:102–106.	MS	21	RCT	Physical exam	DistP, MLS, Time	MA:CAI, add	SPc	10	4
Deutsch ES, et al. Multimodality education for airway endoscopy skill development. Ann Otol Rhinol Laryngol. 2009; 118:81–86.	PG	36X		Endoscopy (GI,Urology,Bronch.)		CM:VR vs manikin	SPc	9.5	1
Domuracki KJ, et al. Learning on a simulator does transfer to clinical practice. Resuscitation. 2009; 80:346–349.	MS, RN, RN	101	RCT	Intubation	FB, Indiv, Mast, MLS	ID:Feedback	SPc	13.5	5
Donoghue AJ, et al. Effect of high-fidelity simulation on Pediatric Advanced Life Support training in pediatric house staff: a randomized trial. Pediatr Emerg Care. 2009; 25:139–144.	PG	51	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs manikin	SPc	14.5	4
Friedman Z, et al. Clinical impact of epidural anesthesia simulation on short- and long-term learning curve: High- versus low-fidelity model training. Reg Anesth Pain Med. 2009; 34:229–232.	PG	24	RCT	Anesthesia,Percutaneous proc.,epidural	ClinV	CM:VR vs model	BP	15	5
Gerling GJ, et al. The design and evaluation of a computerized and physical simulator for training clinical prostate exams. IEEE Transactions on Systems, Man, and Cybernetics Part A, Systems and Humans. 2009; 39:388–403.	MS, RN	26	RCT	Physical exam	ClinV, FB, Indiv, RangeD	CM:Model vs model	SPc, SPd	11.5	2
Girzadas DV, Jr., et al. Hybrid simulation combining a high fidelity scenario with a pelvic ultrasound task trainer enhances the training and evaluation of endovaginal ultrasound skills. Acad Emerg Med. 2009; 16:429–435.	PG	45	RCT	Critical thinking,Obstetrics		ID:Sequence	Sa, ST	9	4
Helmy S, et al. Development of laparoscopic skills using a new inexpensive webcam trainer. Journal of Biological Sciences. 2009; 9:766–771.	PG	12X	RCT	Min. invasive surg.		CM:Model vs model	ST	13	3
Hoadley TA. Learning advanced cardiac life support: a comparison study of the effects of low- and high-fidelity simulation. Nurs Educ Perspect. 2009; 30:91–95.	MD, RN, EMT, O	53	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs manikin	Sa, K, SPc	14	2
Jayaraman S, et al. Novel hands-free pointer improves instruction efficiency in laparoscopic surgery. Surgical Innovation. 2009; 16:73–77.	MS, PG, MD	10X		Min. invasive surg.	CogI, FB	ID:Instructions	ST	11.5	2
Jensen AR, et al. Acquiring basic surgical skills: Is a faculty mentor really needed? Am J Surg. 2009; 197:82–88.	PG	44	RCT	Open surgery/suturing	CogI, FB	In:Self-instruction	ST, SPc, SPd	12.5	6
Kahol K, et al. Cognitive simulators for medical education and training. Journal of Biomedical Informatics. 2009; 42:593–604.	PG	10		Min. invasive surg.	CogI, RangeD	ID:Additional practice	SPc	12.5	2
Kardong-Edgren S, et al. VitalSim® versus SimMan®: a comparison of BSN student test scores, knowledge retention, and satisfaction. Clinical Simulation in Nursing. 2009; 5:e105–e111.	RN	76	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs manikin	Sa, K	14.5	4
Kishore TA, et al. Task deconstruction facilitates acquisition of transurethral resection of prostate skills on a virtual reality trainer. J Endourol. 2009; 23:665–668.	MS	18	RCT	Endoscopy (GI,Urology,Bronch.)	RangeD	ID:Sequence	Sa, SPc	11.5	3
Kromann CB, et al. The effect of testing on skills learning. Med Educ. 2009; 43:21–27.	MS	81	RCT	Resuscitation (BLS,ACLS,ATLS)	Indiv	ID:Testing effect	SPc	13.5	5
LeFlore JL, et al. Alternative educational models for interdisciplinary student teams. Simul Healthc. 2009; 4:135–142.	RN, EMT	13	RCT	Resuscitation (BLS,ACLS,ATLS),Team training	CogI, FB, Indiv, RepP	ID:Hands on	Sa, ST, SPc	14.5	4
McDougall EM, et al. Preliminary study of virtual reality and model simulation for learning laparoscopic suturing skills. J Urol. 2009; 182:1018–1025.	MS	20	RCT	Min. invasive surg.		CM:VR vs model	Sa, ST, SPc	15.5	4
Moulton C-A, et al. Teaching communication skills using the integrated procedural performance instrument (IPPI): a randomized controlled trial. Am J Surg. 2009; 197:113–118.	MS, PG	30	RCT	Open surgery/suturing,Communication skill	CogI, FB	ID:Feedback	SPc	13.5	4
Muller MP, et al. Excellence in performance and stress reduction during two different full scale simulator training courses: a pilot study. Resuscitation. 2009; 80:919–924.	MD	29	RCT	Resuscitation (BLS,ACLS,ATLS),Team training		ID:Teach cognition	SPc	14.5	4
Panait L, et al. The role of haptic feedback in laparoscopic simulation training. J Surg Res. 2009; 156:312–316.	MS	10X		Min. invasive surg.	CogI, MLS	SA:Tactile	ST, SPc	11.5	3
Rodgers DL, et al. The effect of high-fidelity simulation on educational outcomes in an Advanced Cardiovascular Life Support course. Simul Healthc. 2009; 4:200–206.	RN	34		Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs manikin	K, SPc	13.5	3
Rodriguez Garcia JI, et al. Does the incorporation of a virtual simulator improve abilities in endoscopic surgery acquired with an inanimate simulator? Cir Esp. 2009; 86:167–70.	PG	17	RCT	Min. invasive surg.	CogI, FB, Time	MA:Simulator, add	SPc	10.5	2
Rosenthal ME, et al. Pretraining on Southwestern stations decreases training time and cost for proficiency-based fundamentals of laparoscopic surgery training. J Am Coll Surg. 2009; 209:626–631.	MS	20		Min. invasive surg.	Time	MA:Simulator, add	ST, SPc	11.5	1
Sullivan-Mann J, et al. The effects of simulation on nursing students’ critical thinking scores: a quantitative study. Newborn and Infant Nursing Reviews. 2009; 9:111–116.	RN	53	RCT	Critical thinking	ClinV	ID:Repetition	K	14.5	4
Szafranski C, et al. Distractions and surgical proficiency: an educational perspective. Am J Surg. 2009; 198:804–810.	PG	14		Min. invasive surg.	Indiv	ID:Stress	ST, SPc	12.5	2
Ti LK, et al. Experiential learning improves the learning and retention of endotracheal intubation. Med Educ. 2009; 43:654–660.	MS	210	RCT	Intubation	CogI, FB, Indiv	In:Instructor intensity	SPc, SPd	12.5	6
Walsh CM, et al. Concurrent versus terminal feedback: it may be better to wait. Acad Med. 2009; 84 (10 Suppl):S54–S57.	MS	30	RCT	Endoscopy (GI,Urology,Bronch.)		ID:Feedback	ST, SPc	13.5	4
Welke TM, et al. Personalized oral debriefing versus standardized multimedia instruction after patient crisis simulation. Anesth Analg. 2009; 109:183–189.	PG	30	RCT	Resuscitation (BLS,ACLS,ATLS),Team training	CogI, FB, Indiv	ID:Feedback	SPc	14.5	5
Yasukawa Y. The effectiveness of cavity preparation training using a virtual reality simulation system with or without feedback. [Japanese]. Kokubyo Gakkai Zasshi. 2009; 76:73–80.	D	39	RCT	Dentistry	CogI, FB, Indiv, MLS	ID:Feedback	ST, SPc	12.5	5
Zausig YA, et al. Inefficacy of simulator-based training on anaesthesiologists’ non-technical skills. Acta Anaesthesiol Scand. 2009; 53:611–619.	MD	42	RCT	Anesthesia	FB, Time	ID:Teach cognition	SPc	15.5	6
Acton RD, et al. Synthesis versus imitation: evaluation of a medical student simulation curriculum via Objective Structured Assessment of Technical Skill. J Surg Educ. 2010; 67:173–178.	MS	189		Open surgery/suturing	CogI, MLS, Time	ID:Sequence	ST, SPc	12.5	1
Ahmad I, et al. Evaluation of Real-time Visio-haptic deformable Bovine Rectal Palpation Simulator. Proceedings 2010 International Symposium on Information Technology – Visual Informatics, ITSim’10 2010; 1:Art. No.: 5561364.	V	20		Physical exam		SA:Visual	SPc	11.5	2
Blum CA, et al. High-fidelity nursing simulation: impact on student self-confidence and clinical competence. International Journal of Nursing Education Scholarship. 2010; 7:Article 18.	RN	53		Communication skill,Critical thinking,Nursing health assessment		CM:Manikin vs model	SPc	14.5	3
Brydges R, et al. Comparing self-guided learning and educator-guided learning formats for simulation-based clinical training. J Adv Nurs. 2010; 66:1832–1844.	RN	30	RCT	Venous access	Indiv, Mast, Time	ID:Sequence	SPc	14.5	5
Brydges R, et al. Coordinating progressive levels of simulation fidelity to maximize educational benefit. Acad Med. 2010; 85:806–812.	MS	30	RCT	Venous access	ClinV, Indiv	ID:Sequence	Sa, SPc	14.5	5
Buzink SN, et al. Do basic psychomotor skills transfer between different image-based procedures? World J Surg. 2010; 34:933–940.	MS	29X	RCT	Min. invasive surg.,Endoscopy (GI,Urology,Bronch.)	ClinV	CM:VR vs VR	ST	11.5	3
Cason CL, et al. Improving learning of airway management with case-based computer microsimulations. Clinical Simulation in Nursing. 2010; 6:e15–e23.	RN	76	RCT	Critical thinking		MA:CAI, compare	K	11.5	5
Chandra V, et al. A comparison of laparoscopic and robotic assisted suturing performance by experts and novices. Surgery. 2010; 147:830–839.	PG	20	RCT	Min. invasive surg.,Robotic surg.		MA:Robot assistance, add	ST, SPc	13.5	4
Da Cruz JAS, et al. Does training laparoscopic skills in a virtual reality simulator improve surgical performance? J Endourol. 2010; 24:1845–1849.	MS	10	RCT	Min. invasive surg.	MLS	ID:Sequence	ST, SPc	11.5	4
Dantas AK, et al. Assessment of preclinical learning on oral surgery using three instructional strategies. J Dent Educ. 2010; 74:1230–1236.	D	20		Dentistry	CogI, FB, Indiv, MLS	MA:Discussion, compare	SPc	10.5	2
Davoudi M, et al. Comparative effectiveness of low- and high-fidelity bronchoscopy simulation for training in conventional transbronchial needle aspiration and user preferences. Respiration. 2010; 80:327–334.	MD	44X	RCT	Endoscopy (GI,Urology,Bronch.)		CM:VR vs model	Sa	9	3
DeMaria Jr S, et al. Adding emotional stressors to training in simulated cardiopulmonary arrest enhances participant performance. Med Educ. 2010; 44:1006–1015.	MS	25	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, MLS	ID:Stress	K, SPc	13.5	4
Debes AJ, et al. A tale of two trainers: virtual reality versus a video trainer for acquisition of basic laparoscopic skills. Am J Surg. 2010; 199:840–845.	MS	38X	RCT	Min. invasive surg.	RangeD	CM:VR vs model	ST, SPc	11.5	5
Donoghue AJ, et al. Perception of realism during mock resuscitations by pediatric housestaff: the impact of simulated physical features. Simul Healthc. 2010; 5:16–20.	PG	51	RCT	Resuscitation (BLS,ACLS,ATLS)		CM:Manikin vs manikin	Sa	11	3
Dunnican WJ, et al. Reverse alignment “Mirror Image” visualization as a laparoscopic training tool improves task performance. Surgical Innovation. 2010; 17:108–113.	MS, PG	21	RCT	Min. invasive surg.		ID:Clinical scenario	SPc	12	4
Gauger PG, et al. Laparoscopic simulation training with proficiency targets improves practice and performance of novice surgeons. Am J Surg. 2010; 199:72–80.	PG	14	RCT	Min. invasive surg.	Mast, MLS	ID:Mastery	SPc, BP	13	3
Grant JS, et al. Using video-facilitated feedback to improve student performance following high-fidelity simulation. Clinical Simulation in Nursing. 2010; 6:e177–e184.	RN	40	RCT	Communication skill,Team training		ID:Feedback	SPc	14.5	5
Hein C, et al. A training program for novice paramedics provides initial laryngeal mask airway insertion skill and improves skill retention at 6 months. Simul Healthc. 2010; 5:33–39.	EMT	50	RCT	Intubation		ID:Repetition	ST, SPc	13.5	4
Hunziker S, et al. Brief leadership instructions improve cardiopulmonary resuscitation in a high-fidelity simulation: a randomized controlled trial.[Erratum appears in Crit Care Med. 2010 Jun;38(6):1510]. Crit Care Med. 2010; 38:1086–1091.	MS	63	RCT	Resuscitation (BLS,ACLS,ATLS)		ID:Teach cognition	ST, SPc	13.5	6
Kardong-Edgren SE, et al. Comparison of two instructional modalities for nursing student CPR skill acquisition. Resuscitation. 2010; 81:1019–1024.	RN	604		Resuscitation (BLS,ACLS,ATLS)	GrpP, Indiv, MLS	In:Self-instruction	SPc	12.5	2
Kent DJ. Effects of a just-in-time educational intervention placed on wound dressing packages: a multicenter randomized controlled trial. Journal of Wound, Ostomy, & Continence Nursing. 2010; 37:609–614.	RN	139	RCT	Dressing change	CogI	ID:Instructions	Sa, SPd	14.5	4
Kromann CB, et al. The testing effect on skills learning might last 6 months. Adv Health Sci Educ Theory Pract. 2010; 15:395–401.	MS	89	RCT	Resuscitation (BLS,ACLS,ATLS)	MLS	ID:Testing effect	SPc	13.5	6
Kruglikova I, et al. The impact of constructive feedback on training in gastrointestinal endoscopy using high-fidelity virtual-reality simulation: a randomised controlled trial. Gut. 2010; 59:181–185.	PG	21	RCT	Endoscopy (GI,Urology,Bronch.)	CogI, FB, Indiv, MLS	ID:Feedback	ST, SPc	12.5	5
Lauscher JC, et al. A new surgical trainer (BOPT) improves skill transfer for anastomotic techniques in gastrointestinal surgery into the operating room: a prospective randomized trial. World J Surg. 2010; 34:2017–2025.	PG	35	RCT	Open surgery/suturing		CM:Manikin vs model	ST, SPc	12.5	4
Leblanc F, et al. A comparison of human cadaver and augmented reality simulator models for straight laparoscopic colorectal skills acquisition training. J Am Coll Surg. 2010; 211:250–255.	MD	31		Min. invasive surg.		CM:VR vs cadaver	Sa, SPc, SPd	12.5	1
Leblanc F, et al. Hand-assisted laparoscopic sigmoid colectomy skills acquisition: augmented reality simulator versus human cadaver training models. J Surg Educ. 2010; 67:200–204.	MD	26		Min. invasive surg.	Indiv	CM:VR vs cadaver	Sa, SPc, SPd	11.5	1
Lee CC, et al. Comparison of traditional advanced cardiac life support (ACLS) course instruction vs. a scenario-based, performance oriented team instruction (SPOTI) method for Korean paramedic students. J Emerg Med. 2010; 38:89–92.	EMT	30	RCT	Resuscitation (BLS,ACLS,ATLS)	FB, Indiv, MLS	ID:Blending	ST, SPc	10.5	4
Lerner MA, et al. Does training on a virtual reality robotic simulator improve performance on the da Vinci® surgical system? J Endourol. 2010; 24:467–472.	MS, PG	22		Robotic surg.	Curr, FB	CM:VR vs model	ST	11.5	2
Martínez AM, et al. Adaptation to a dynamic visual perspective in laparoscopy through training in the cutting task. Surg Endosc. 2010; 24:1341–1346.	PG	26X		Min. invasive surg.		SA:Visual	ST, SPc	10.5	2
McCormick MJ, et al. Case scenarios and simulations as techniques to facilitate asthma education and team building among health care students. Journal of Asthma and Allergy Educators. 2010; 1:18–22.	RN, O	34		Physiology:Asthma		GC:Interdiscplinary vs single disc.	Sa	9	1
Mishra S, et al. Percutaneous renal access training: Content validation comparison between a live porcine and a virtual reality (VR) simulation model. BJU International. 2010; 106:1753–1756.	MD	24X		Endovascular proc.		CM:VR vs animal	Sa	7	2
Mohammadi Y, et al. Comparison of laparoscopy training using the box trainer versus the virtual trainer. J Soc Laparoendosc Surg. 2010; 14:205–212.	MS	43		Min. invasive surg.		CM:VR vs model	ST, SPc	11.5	3
Molinas CR, et al. Defining a structured training program for acquiring basic and advanced laparoscopic psychomotor skills in a simulator. Gynecological Surgery. 2010; 7:427–435.	PG, MD	40	RCT	Min. invasive surg.	RangeD	ID:Sequence	SPc	11.5	3
Morandeira Rivas A, et al. Low cost simulator for acquiring basic laparoscopic skills. Cir Esp. 2010; 87:26–32.	MS	16	RCT	Min. invasive surg.		CM:Model vs model	ST	13.5	4
Muresan C, III, et al. Transfer of training in the development of intracorporeal suturing skill in medical student novices: a prospective randomized trial. Am J Surg. 2010; 200:537–541.	MS	20	RCT	Min. invasive surg.		ID:Additional practice	ST, SPc	12.5	3
Oermann MH, et al. HeartCode BLS with voice assisted manikin for teaching nursing students: preliminary results. Nurs Educ Perspect. 2010; 31:303–308.	RN	603	RCT	Resuscitation (BLS,ACLS,ATLS)	GrpP, Indiv, MLS	ID:Feedback	SPc	12.5	4
Okrainec A, et al. Telesimulation: an effective method for teaching the fundamentals of laparoscopic surgery in resource-restricted countries. Surg Endosc. 2010; 24:417–422.	PG, MD	16		Min. invasive surg.	CogI, FB, Indiv, MLS	In:Distance supervision	SPc	12	1
Orde S, et al. A randomised trial comparing a 4-stage to 2-stage teaching technique for laryngeal mask insertion. Resuscitation. 2010; 81:1687–1691.	MS, RN, RN	120	RCT	Intubation	MLS	ID:Sequence	ST, SPc, SPd	13.5	5
Perkins GD, et al. The effect of pre-course e-learning prior to advanced life support training: A randomised controlled trial. Resuscitation. 2010; 81:877–881.	MD, RN, O	551	RCT	Resuscitation (BLS,ACLS,ATLS)	FB, MLS	MA:CAI, add	K, SPc	15.5	5
Persoon MC, et al. The effect of a low-fidelity model on cystoscopic skill training: a single-blinded randomized controlled trial. Simul Healthc. 2010; 5:213–218.	MS	32	RCT	Endoscopy (GI,Urology,Bronch.)		MA:Simulator, add	Sa, ST, SPc, SPd	13.5	4
Salkini MW, et al. The role of haptic feedback in laparoscopic training using the LapMentor II. J Endourol. 2010; 24:99–102.	MS	20		Min. invasive surg.	FB	SA:Tactile	SPc	10.5	2
Scavone BM, et al. A randomized controlled trial of the impact of simulation-based training on resident performance during a simulated obstetric anesthesia emergency. Simul Healthc. 2010; 5:320–324.	PG	32	RCT	Anesthesia		ID:Clinical scenario	Sa, ST, SPc	14.5	4
Stefanidis D, et al. Initial laparoscopic basic skills training shortens the learning curve of laparoscopic suturing and is cost-effective. J Am Coll Surg. 2010; 210:436–440.	MS	18	RCT	Min. invasive surg.	Time	MA:Simulator, add	ST, SPc	14.5	5
Stefanidis D, et al. Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg Endosc. 2010; 24:377–382.	MS	24X		Min. invasive surg.,Robotic surg.		MA:Robot assistance, add	ST, SPc	11.5	1
Suebnukarn S, et al. Augmented kinematic feedback from haptic virtual reality for dental skill acquisition. J Dent Educ. 2010; 74:1357–1366.	D	16	RCT	Dentistry	CogI, FB, MLS	ID:Feedback	ST, SPc	12.5	4
Thomas EJ, et al. Team training in the neonatal resuscitation program for interns: teamwork and quality of resuscitations. Pediatrics. 2010; 125:539–546.	PG	67	RCT	Resuscitation (BLS,ACLS,ATLS)		MA:Team training, add	ST, SPc	14.5	6
Van Heukelom JN, et al. Comparison of postsimulation debriefing versus in-simulation debriefing in medical simulation. Simul Healthc. 2010; 5:91–97.	MS	161	RCT	Resuscitation (BLS,ACLS,ATLS)		ID:Feedback	Sa	11	5
Wandell HF. Using a virtual reality simulator in phlebotomy training. Laboratory Medicine. 2010; 41:463–466.	O	25		Venous access	ClinV, FB	CM:VR vs model	SPc	11.5	2
Wong W, et al. The effect of cross-training with adjustable airway model anatomies on laryngoscopy skill transfer. Anesth Analg. 2010; Available online October 21, 2010; in press.	MS, EMT	47		Intubation	CogI, MLS, RangeD	ID:Task variability	SPd	10.5	2
Yang JH, et al. Comparison of four manikins and fresh frozen cadaver models for direct laryngoscopic orotracheal intubation training. Emerg Med J. 2010; 27:13–16.	MD, RN	56X	RCT	Intubation		CM:Manikin vs cadaver	Sa	9	3
Zendejas B, et al. Teaching first or teaching last: Does the timing matter in simulation-based surgical scenarios? J Surg Educ. 2010; 67:432–438.	MS, PG	49X		Various surgical topics		ID:Sequence	K	12.5	5
Alfes CM. Evaluating the use of simulation with beginning nursing students. J Nurs Educ. 2011; 50:89–93.	RN	63		Pain management	CogI	ID:Hands on	Sa, SPc	9.5	2
Arora S, et al. Mental practice enhances surgical technical skills: A randomized controlled study. Ann Surg. 2011; 253:265–270.	PG	18	RCT	Min. invasive surg.	CogI, Indiv, MLS	ID:Teach cognition	SPc	14.5	5
Auerbach M, et al. Repetitive pediatric simulation resuscitation training. Pediatr Emerg Care. 2011; 27:29–31.	PG	151		Resuscitation (BLS,ACLS,ATLS)	CogI, MLS, RepP	ID:Repetition	Sa	9	1
Bath J, et al. Standardization is superior to traditional methods of teaching open vascular simulation. J Vasc Surg. 2011; 53:229–223.e2	PG	18	RCT	Open surgery/suturing	MLS	ID:Sequence	Sa, K, SPc	12.5	5
Fraser K, et al. Simulation training improves diagnostic performance on a real patient with similar clinical findings. Chest. 2011; 139:376–381.	MS	57	RCT	Physical exam		ID:Clinical scenario	SPd	11.5	4
Guhde J. Nursing students’ perceptions of the effect on critical thinking, assessment, and learner satisfaction in simple versus complex high-fidelity simulation scenarios. J Nurs Educ. 2011; 50:73–78.	RN	133X		Nursing tasks	FB, GrpP, MLS	ID:Task variability	Sa	8	3
Maggio MP, et al. The effect of magnification loupes on the performance of preclinical dental students. Quintessence Int. 2011; 42:45–55.	D	232		Dentistry	MLS	SA:Visual	ST, SPc	11.5	3
Naughton PA, et al. Skills training after night shift work enables acquisition of endovascular technical skills on a virtual reality simulator. J Vasc Surg. 2011; 53:858–866.	PG	20		Endovascular proc.		ID:Timing	ST, SPc	10.5	2
Oermann MH, et al. Effects of monthly practice on nursing students’ CPR psychomotor skill performance. Resuscitation. 2011; 82:447–453	RN	495	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, DistP, FB, Indiv, MLS, RepP, Time	ID:Repetition	SPc	13.5	5
Snyder CW, et al. Effects of Virtual Reality Simulator Training Method and Observational Learning on Surgical Performance. World J Surg. 2011; 35:245–252.	MS	32	RCT	Min. invasive surg.,Endoscopy (GI,Urology,Bronch.)	FB, Indiv, MLS	ID:Sequence	ST	13.5	4
Sutton RM, et al. “Booster” training: Evaluation of instructor-led bedside cardiopulmonary resuscitation skill training and automated corrective feedback to improve cardiopulmonary resuscitation... Pediatr Crit Care Med. 2011; Online 2010 Jul 9; in press.	PG, RN	46	RCT	Resuscitation (BLS,ACLS,ATLS)	CogI, FB	ID:Feedback	SPc	12.5	4
Swanson EA, et al. Comparison of selected teaching strategies incorporating simulation and student outcomes. Clinical Simulation in Nursing. 2011; 7(3):e81–e90.	RN	96	RCT	Critical thinking	FB	In:Instructor intensity	SPc	14.5	4
Thompson JR, et al. Limited value of haptics in virtual reality laparoscopic cholecystectomy training. Surg Endosc. 2011; 25:1107–1114.	MS, O	8	RCT	Min. invasive surg.		SA:Tactile	ST, SPc	12	3
Uccelli J, et al. The validity of take-home surgical simulators to enhance resident technical skill proficiency. Am J Surg. 2011; 201:315–319.	PG	14	RCT	Min. invasive surg.	CogI, FB, Indiv, MLS, Time	ID:Sequence	ST, SPc	11.5	3
Zhao YC, et al. Can virtual reality simulator be used as a training aid to improve cadaver temporal bone dissection? Results of a randomized blinded control trial. Laryngoscope. 2011; 121:831–837.	PG	20	RCT	Endoscopy (GI,Urology,Bronch.)	FB, GrpP, Indiv, MLS	CM:VR vs model	SPc, SPd	13.5	4

Trainees: MS = medical student, PG = postgraduate physician trainee, MD = practicing physician, RN = nurse or nursing student, EMT = emergency medical technician/paramedic/first responder or EMT student, D = dentist or dental student, V = veterinarian or veterinary student, C = chiropractor or student, O = other/mixed.

N: Number of outcome observations; this is usually the number of trainees, but in some cases reflects the number of teams observed or the number of patient observations. X = Crossover.

RCT = randomized controlled trial.

Feature: Key features that varied between interventions (i.e. included in meta-analyses), see main text for definitions. Features that were coded the same for both groups (eg, both present or both absent) are not listed. ClinV = clinical variation; CogI = cognitive interactivity; Curr = curriculum integration; DistP = distributed practice; FB = feedback; GrpP = group practice; Indiv = individualization; Mast = mastery learning; MLS = multiple learning strategies; RangeD = range of difficulty; RepP = repetitive practice; Time = time learning.

Theme: The overall research theme (research question). Main themes are indicated first, with sub-themes following the colon. MA = modality added; GC = group composition; ID = instructional design; In = Instructor; CM = compare sim modality; SA = sensory augmentation.

Outcomes: Sa = Satisfaction, K = knowledge, ST = skill-time, SPc = skill-process, SPd = skill-product, BT = behavior-time, BP = behavior-process, P = patient effects.

Quality: MERSQI = Medical Education Research Study Quality Instrument (maximum score 18); NOS = modified Newcastle-Ottawa scale (maximum score 6).

Table A2.  Results of sensitivity analyses

Feature	No. studies	Pooled ES (95% CI)	p	I^2	Main analysis pooled ES
Non-time Skills
Clinical variation	16	0.18 (−0.14, 0.51)	0.27	83	0.20
Cog. interactivity	89	0.59 (0.44, 0.75)	0.00	88	0.65
Feedback	80	0.36 (0.21, 0.50)	0.00	87	0.44
Group practice	8	−0.30 (−0.56, −0.05)	0.02	75	−0.22
Individualization	59	0.46 (0.28, 0.64)	0.00	88	0.52
Mult. learning strat.	70	0.54 (0.40, 0.68)	0.00	85	0.62
Range of difficulty	20	0.64 (0.26, 1.03)	0.00	87	0.68
Repetitive practice	7	0.68 (−0.03, 1.38)	0.06	89	0.68
Behaviors and Patient Effects
Feedback	2	0.18 (−0.23, 0.58)	0.39	0	0.32

Notes: Several studies reporting non-time skills had 3 simulation arms. Since we could only compare 2 groups at once, we first included the interventions with the greatest between-group difference and then performed sensitivity analyses substituting the third intervention. The table above summarizes all results influenced by the alternate analyses. The main analysis pooled effect size (ES) is provided to aid in comparison.

Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin P, Hamstra S. Technology-Enhanced simulation for health professions education: A systematic review and meta-analysis. J Am Med Assoc 2011; 306: 978–988

 PubMed Web of Science ®Google Scholar