Applying theory to practice in undergraduate education using high fidelity simulation

Abstract

High-fidelity patient simulation allows students to apply their theoretical knowledge of pharmacology and physiology to practice. The purpose of this study was to determine if experiential education using high-fidelity simulation improves undergraduate performance scores on simulation-based and written examinations. After receiving research ethics board approval, students completed a consent form and then answered a ten question multiple-choice quiz to identify their knowledge regarding the management of cardiac arrhythmias. Four simulation scenarios were presented and students worked through each scenario as a team. Faculty facilitated the sessions and feedback was given using students’ videotaped performances as a template for discussion. Performance evaluation scores using predetermined checklists and global rating scales were completed. Students then reviewed the American Heart Association guidelines for the management of unstable cardiac arrhythmias. The afternoon session involved repetition of the four case scenarios with the same teams involved but different team leaders. Students then repeated the quiz they received in the morning. Descriptive statistics, paired t-test and repeated measures analysis of variance (ANOVA) were used to analyse results. Two hundred and ninety-nine students completed the study. There was a statistically significant improvement in performance on the pharmacology written test. Simulation team performance also statistically improved and a good correlation between checklist and global rating scores were demonstrated in all but one scenario. Student evaluation of the experience was extremely positive. High-fidelity simulation can be used to allow students to apply theoretical knowledge to practice in a safe and realistic environment. Results of this study indicate that simulation is a valuable learning experience and bridges the gap between theory and practice. Simulation technology has the potential to provide an enriching venue to examine the role of communication and dynamics of novice learners in team environments.

Introduction

Bridging the gap between theoretical knowledge and practical application has been problematical for most professions. The seminal work at the Massachusetts Institute of Technology (MIT) School of Business, carried out by Argyris and Schön, revealed that even with extensive intervention it was still difficult for students to apply their knowledge in related practical situations (Argyris & Schön, 1974). Later, Schön, described the gap as one between the ‘high ground’ of academic scientific theory, where problems are resolved by research based theory, and the ‘swamp ground’, where real-world problems are indeterminate and ill defined (Schön, 1987).

Medical education is no exception to the problem of the gap between theory and practice. Traditionally the medical curriculum has been designed to provide a basis of medical science followed by clinical experience in a number of medical specialties. Many recent innovations in medical curricula have focused on the need to integrate practice and theory (Slotnick, 1996; Prince et al., 2000). Nursing has also identified the issues of the gaps between nursing theory and nursing practice (Miller, 1985) and the contrast between the ‘ideal’ as portrayed by theory and the ‘reality’ as experienced in the provision of patient care (Phillips et al., 1998).

In the specialty of anesthesiology it is assumed that application of scientific knowledge at the undergraduate level will take place in the operating room under the supervision of an anesthesiologist. Yet opportunities to actually practice the necessary skills are hampered by such issues as case complexity, the number of learners attending the case, and patient safety. A needs assessment from previous years’ students identified a deficit in the knowledge of the pharmacology of resuscitative drugs used for cardiac arrhythmias and the lack of hands-on experience in managing such cases. At our Institution, Advanced Cardiac Life Support (ACLS) training is not an integral part of the curriculum during the undergraduate years.

Modern technology, such as high-fidelity simulation (HFS) offers unique opportunities to provide this ‘hands-on’ learning. High-fidelity simulation offers the ideal venue to allow ‘practice’ without risk. There are an infinite number of realistic scenarios that can be presented using this technology. As an example, life threatening cardiac arrhythmias can be simulated on a life-like fully computerized mannequin. Monitors, identical to those used in the clinical situation can replicate the arrhythmia and corresponding changes in vital signs. The ‘patient’ can be fully and realistically resuscitated with technical and pharmacological interventions. Viewing of videotaped performances allows personal reflection on the effectiveness of the case management.

Our project was designed to respond to students’ needs to gain experience in the application of their theoretical knowledge to a virtual-reality practice situation. This enabled students to practice their skills and repeat the scenario thereby gaining valuable insights into their performance. We also felt it important to evaluate the program with respect to both improvements in team performance as well as student opinions of the educational session.

Methods

Scenarios

Four scenarios, previously used for another undergraduate simulation study were adapted to facilitate the learning objectives of this proposal. The scenarios involved a clinical situation that ultimately resulted in an unstable cardiac arrhythmia. These included: (1) Transfusion reaction leading to hypotension and Pulseless Electrical Activity (PEA); (2) Intravascular injection of local anesthetic through an indwelling epidural catheter leading to severe bradycardia and hypotension; (3) Hypoxemia leading to paroxysmal supraventricular tachycardia (PSVT) and (4) Anaphylactic reaction to antibiotic leading to pulseless ventricular tachycardia (VT). Details and learning objectives of a sample scenario can be found in Appendix 1.

Checklists and global rating scales

Performance checklists for the selected scenarios had previously shown to have acceptable internal consistency with Cronbach's alpha ≥0.6 and ≤0.9 (Morgan et al., 2004). In addition, these checklists included the ACLS management algorithms for the four presented cardiac arrhythmias1 (Appendix 2).

A global rating scale was used to assess performance using a five-point scale with 1 denoting an unacceptable performance and 5 denoting a superior performance (1= unacceptable; 2 = borderline; 3 = acceptable; 4 = good; 5 = superior). Each point on the scale had descriptors identifying performance criteria.

Facilitators

Anaesthesia faculty involved in undergraduate education and senior residents in anesthesia acted as facilitators and raters for each study session. A workshop outlining the purpose and methodology of the study was held prior to the start of the academic year. All facilitators received an information package including all scenario outlines, learning objectives, global rating descriptors and checklists.

Simulation day

All final year medical students in a two-year period (n = 370) were invited to participate in this research project. Institutional ethics approval was received and informed consent obtained from participants. The simulation sessions were held every 2 weeks and approximately ten students were involved per session. On arrival at the simulation centre and before the educational session began, students were asked to complete a brief demographic questionnaire and to complete a confidentiality form indicating that they would not reveal the nature or content of the test or simulation scenarios. They then answered a ten question multiple choice pharmacology pre-test related to the subsequent educational session. Each question had a brief stem followed by five possible responses with one correct answer. As a part of the education day in the simulator, students were given the opportunity to manage a case as ‘team leader’ or assume the role of a peer who would assist with the case management. Each team consisted, therefore, of two to three students. Student team performance was assessed using the performance checklists and global rating scale. Assessment of this initial team performance was considered the pre-test. All sessions were videotaped. After each of the first four scenarios were completed, faculty gave feedback to all students using the videotaped performance as a template for discussion. Students then received an educational package containing the American Heart Association guidelines specifically addressing the arrhythmias presented in the scenarios. Students were given 30 minutes to review this material.

After a lunch break, students were given the opportunity to repeat the same scenarios in the same team configuration but with a different student assigned the ‘team leader’ role. Both performance checklists and global rating scales of team performance were completed for this repeat session and were considered the post-test. Once the scenarios were finished, students completed the multiple choice pharmacology post-test and an evaluation of the educational experience.

Statistical analysis

SPSS 11.0.1 for Windows was used for data analysis. Descriptive statistics were used to analyse the demographic data and student evaluation of the experience. Pre- and post-test pharmacology answers were compared using a paired t-test. Simulation performance assessments were analysed using repeated measures analysis of variance (ANOVA) with the pre- and post-test used as the within subjects factor and the team used as the between subjects factor. Pearson product moment correlations were used to compare the pre- and post-test checklist scores to the pre- and post-test global rating scores. For all analyses, a p-value of ≤0.05 was considered significant.

Results

Two hundred and ninety-nine students (299/370) completed the study representing 80.81% of the student population. Reasons for non-attendance at the session included conflict with postgraduate interviews and the interruption of the curriculum due to an infectious outbreak. Demographic statistics indicated that 44.7% of respondents were female, and 55.3% male. The majority of students (77.7%) had completed the Basic Cardiac Life Support (BCLS) program. Two percent had completed the Advanced Cardiac Life Support (ACLS) program and 0.3% the Advanced Trauma Life Support (ATLS) course.

Figure 1 indicates the students’ postgraduate program selections. Two hundred and twenty-six of 299 students (75.58%) completed the evaluation of the simulator educational experience and the results are outlined in Figure 2.

Figure 1. Residency program selection.

Figure 2. Student evaluation of simulator experience.

There was no statistically significant difference in pharmacology pre-test scores between the first and second sessions of the academic years (t = 0.283, p = 0.778; t = 1.039, p = 0.302) nor between the two academic years (t = −0.577, p = 0.565). Means and standard deviations of the pharmacology test and the scenario pre- and post-tests are summarized in Table 1. There was a significant improvement between individuals’ pre- and post-test pharmacology test answers, t = −7.650 (p < 0.0001).

Table 1.  Means (%) and standard deviation of pre- and post-test scores

	Pre-test% Mean and SD	Post-test% Mean and SD
Pharmacology Test	48.52 ± 15.71	56.21 ± 16.88
Scenario 1 Team Scores	64.88 ± 14.52	75.86 ± 14.28
Scenario 2 Team Scores	48.91 ± 12.38	61.82 ± 15.73
Scenario 3 Team Scores	36.18 ± 26.14	71.14 ± 9.88
Scenario 4 Team Scores	57.62 ± 10.83	63.74 ± 10.12
All scenarios Team Scores	51.84 ± 20.02	68.18 ± 13.83

One hundred and three teams completed the scenarios. Analyses of data indicated a significant improvement between pre- and post-test simulator team performance scores when all scenarios were considered together (F_1,103 = 101.29, p < 0.0001). Specific scenarios did have an impact on improvement in learning in that some scenarios were more difficult than others (F_3,103 = 15.63, p < 0.0001). There was also significant improvement in checklist and global rating scores between pre- and post-test performance when analyzed by scenario (Table 2).

Table 2.  Comparison of simulator team performance scores and global ratings (repeated measures anova)

Scenario #	Checklist pre-/post-test	p-value	Global pre-/post-test	p-value
1	F1,26 = 18.38	p < 0.0001	F1,26 = 22.75	p < 0.0001
2	F1,26 = 25.99	p < 0.0001	F1,26 = 20.08	p < 0.0001
3	F1,26 = 51.32	p < 0.0001	F1,26 = 34.67	p < 0.0001
4	F1,25 = 7.54	p = 0.011	F1,25 = 8.20	p = 0.008

Pearson product moment correlations comparing pre-test checklist and global ratings and post-test checklist and global ratings are demonstrated in Table 3.

Table 3.  Correlation between checklist and global rating scores

Scenario	Pre-test: Υ	p-value	Post-test: Υ	p-value
1	0.470*	0.013	0.554**	0.003
2	0.257	0.195	0.372	0.056
3	0.497**	0.008	0.398*	0.040
4	0.430*	0.028	0.569**	0.002
ALL	0.449**	0.000	0.477**	0.000

^**Correlation is significant at the 0.01 level (two-tailed). *Correlation is significant at the 0.05 level (two-tailed).

Discussion

Traditionally medical schools have offered the learning of theoretical concepts in the early part of medical education followed by clinical experience in the latter years. The limitation of this clinical experience is well documented with each medical specialty having its own particular difficulties (Van der Vleuten, 1996). Other investigators have suggested that, in some cases, students are not receiving the necessary exposure to the clinical skills that would enable them to become competent physicians (Fincher & Lewis, 1994; Taylor, 1997).

Medical educators are facing new and difficult challenges in the 21st century. Our patient population, once largely based in hospital, has shifted to a predominantly outpatient population in some circumstances, limiting the exposure of undergraduate and postgraduate students to problem solving and clinical skills acquisition. As well, public awareness of the issues regarding patient safety seriously affects educators’ comfort with having ‘novices’ perform procedures on patients. The old adage of ‘see one, do one, teach one’ is no longer a viable educational method in the current medical climate.

Educators now have both the opportunities and means to use new and innovative virtual reality methods for teaching purposes. Advances in computer graphics, accurate anatomical models, the development of haptics (tactile plus spatial orientation) technology have converged to enable the development of virtual reality (VR) simulators (Kaufman & Bell, 1997).

High-fidelity simulation provides a venue to teach and learn in a realistic yet risk free environment. The ‘patient’ is represented by a computer-controlled mannequin who incorporates a variety of physiological functions (e.g. heart and breath sounds, pulse, end-tidal carbon dioxide). An instrumentation computer network can replicate situations likely to be encountered in an operating room, emergency room or critical care environment. A computer operator working in a second room controls the mannequin and the monitors. The simulator mannequin will respond in an accurate way to induced physiologic or pharmacologic interventions. The ‘patient’ will respond according to pre-set physiological characteristics (e.g. a young healthy adult or a geriatric patient with severe emphysema). In addition, the ‘patient’ has the ability to speak, move his arm and open and close his eyes and has pupils that can dilate and constrict. The simulation room can be set up to appropriately reflect the environment, either an emergency room, a recovery room, or a fully equipped operating room. Attached monitors respond to medical interventions. Feedback from participants in the simulated environment has attested to the ‘realism’ of the environment (Morgan & Cleave-Hogg, 2000; Devitt et al., 2001; Garden et al., 2002).

Medical curricula are moving away from didactic lectures and more towards a hands-on experiential environment where students can learn by doing. Education methodology is becoming more focused on knowing how, rather than knowing all (Jones et al., 2001). As well, the importance of communication skills and team work have become part of the core expectations of national bodies such as the Association of Canadian Medical Colleges (ACMC) and the General Medical Council in the United Kingdom.

Innovation in medical education with the ability to provide a learning environment which allows students to apply theory to practice, to fine tune their communication skills and to work in teams was offered in our study. However, as Jones and colleagues have indicated, in order to achieve this goal it is imperative that evaluation of curricular changes and interventions be undertaken (Jones et al., 2001).

It was our purpose, therefore, to investigate the outcomes of high-fidelity simulation education. The students’ lack of training in Advanced Cardiac Life Support (ACLS) and pharmacology courses were the impetus for the content of the simulation based education day. Not unlike the opinion of our medical students, only 52% of undergraduates surveyed in the United Kingdom felt able to provide an effective resuscitation service as a junior house officer (Graham et al., 1994).

As can be seen from our findings, there was a statistically significant improvement in performance of teams managing a patient using videotapes of the simulation education session as a template for discussion. These outcomes parallel an earlier study of individual undergraduate performance (Morgan et al., 2002). Perhaps, even more importantly, students viewed these sessions as a valuable learning experience and found it to be a realistic environment (Morgan & Cleave-Hogg, 2000).

We chose to include both checklist and global rating assessments since previous work has indicated that global ratings may correlate more highly with judgment and technical skills than with knowledge and may in fact be more ‘generalizable’ across disciplines (Regehr et al., 1999; Morgan et al., 2001).

Due to the nature of the study, we were limited to evaluating performance on the same day as the educational session thus we were only able to measure short term learning. Whether the application of their knowledge will continue is unclear although there is some evidence to indicate that training using simulation does endure (Chopra et al., 1994; Holcomb et al., 2002). Further study investigating the long term effects of high fidelity simulation on performance including problem solving, procedural skills and judgment is necessary in order to validate the use of this technology as a permanent part of undergraduate medical education.

As medical educators, we need to continue to search for innovative means to ensure that our medical students receive the education they need to face the challenges of an aging and high risk patient population. As Remmen has said, we need ‘… more and systematic attention to skills training in order to prevent students from becoming ‘a competent doctor’ despite undergraduate training rather than because of it’ (Remmen et al., 1999).

Our study indicates the value of simulation education in the achievement of clinical skills performance objectives and its potential for assessing and improving communication skills and team dynamics. This, together with the overwhelmingly positive student feedback to date, has encouraged us to continue providing experiential learning using simulation technology as part of the anesthesia undergraduate program.

Practice points

Table

•High-fidelity simulation can be used to allow students to apply theoretical knowledge to practice in a safe and realistic environment

•Individual student performance scores on a written test and team performance scores during a simulated scenario improved after videotape reflection of performance and review of advanced cardiac life support protocols

•Further work examining team interactions and dynamics of novice learners using high-fidelity simulation is warranted

Notes

Notes

This study was supported by an Educational Grant from the Dean's Excellence Fund, Faculty of Medicine, University of Toronto.

[1] Advanced Cardiac Life Support, American Heart Association, 2000.

Appendix 1: Sample Scenario

A 68-year-old woman presents for a left total hip replacement due to pain related to long standing osteoarthritis. Her history is positive for the presence of stable angina which she gets once or twice a month. The episodes respond to nitroglycerin. She denies rest or night pain. She is limited in her activity because of her arthritis but a cardiological work-up was negative and left ventricular function was reported as normal. She has never had a myocardial infarction nor is she hypertensive or diabetic. A complete systems review is otherwise negative. She has no allergies and takes nitroglycerin spray prn and Metoprolol 50 b.i.d. Previous general anaesthetics were well tolerated.

The students will give this patient a general anaesthetic. BP 140/85 heart rate 70 bpm, saturation on room air 97%.

Allow students time to induce general anaesthesia. Bring the blood pressure down slightly after induction to about 90/60 mm Hg and heart rate up to 100 beats per minute.

Allow the situation to stabilize. The surgeon will then ask for a gram of cefazolin. Give students time to prepare the cefazolin and administer it. Over the ensuing couple of minutes, the patient's lung fields will become wheezy, the oxygen saturation drops to about 95% and the blood pressure drops to 80/40 mm Hg. The EKG will show signs of ischaemia reflected by ST-T wave depression. Heart rate will rise until the rhythm deteriorates to ventricular tachycardia. There is no output (no carotid pulse detected). Despite treatment (hopefully defibrillation), the patient continues in the same rhythm without an output. Management of pulseless ventricular tachycardia. Allow the scenario to continue as long as the student continues to treat the patient.

Learning objectives

Recognition of anaphylaxis under anesthesia

Management of anaphylaxis

Treatment of pulseless ventricular tachycardia

Appendix 2. Sample team performance scenario checklist

Item
Initiate appropriate management
Increase IV
Increase oxygen/100% oxygen
Management of Pulseless ventricular tachycardia
Review airway and breathing
Begin CPR
Call for crash cart
Defibrillate 200 J
Defibrillate 300 J
Defibrillate 360 J
Epinephrine 1 mg IV push
Epinephrine dose other than above, specify dose
Vasopressin 40 U IV, single dose
Vasopressin dose not known
Repeat defibrillation sequence
State amiodorone as possible IV agent
State lidocaine as possible IV agent
State magnesium (if known to be hypomagnesemic)
State procainamide as possible IV agent
Repeat defibrillation sequence