A simulation-based biodefense and disaster preparedness curriculum for internal medicine residents

Abstract

Aims: Disaster and bioterrorism preparedness is poorly integrated into the curricula of internal medicine residency programs. Given that victims may present to a variety of healthcare venues, including primary care practices, inpatient hospital wards, and intensive care units, we developed a curriculum to address this need.

Methods: The curriculum consisted of four didactic sessions with supplemental readings covering biologic, chemical, and radiologic agents, as well as public health infrastructure. All 30 internal medicine resident participants also underwent a four hour training seminar at a high fidelity human simulation center. Instruction included the use of personal protective equipment (PPE) and participation in simulated scenarios utilizing technologically sophisticated mannequins with monitoring and interactive capability. Sessions were videotaped, reviewed with participants, and followed by self-evaluation and constructive feedback.

Results: Compared to a control group of residents who did not undergo training, the participants’ level of knowledge was significantly better, with mean objective test scores of 66.8% ;± ;11.8% SD vs. 50% ;± ;13.1% SD, p ;< ;0.0001. Although there was a trend toward increasing knowledge with increasing level of training in the control group, this difference was not significant. Subjective preparedness was also significantly better in the intervention group (p ;< ;0.0001). Objective improvements were not maintained after one year.

Conclusions: In this pilot study, a disaster-preparedness curriculum including simulation-based training had a positive effect on residents’ knowledge base and ability to respond to disaster. However, this effect had diminished after one year, indicating the need for reinforcement at regular intervals.

Introduction

Despite heightened awareness regarding the threat of pandemic influenza, bioterrorism, and other natural and man-made disasters, physicians remain poorly trained to recognize and respond appropriately to these events (Pesik et al. 1999; Chen et al. 2002; Chung et al. 2004; Cosgrove et al. 2005; Terndrup et al. 2005). Physicians are likely to be among the first responders to an occult bioweapon attack. Since early recognition of specific disease entities and appropriate reporting to public health authorities may decrease overall morbidity and mortality and improve our ability to contain the spread of transmissible disease, physician education is a vital part of our defense (Artenstein et al. 2002; Chung et al. 2004; Wynia & Gostin 2004; Cosgrove et al. 2005). Unlike basic life support (BLS) and advanced cardiac life Support (ACLS) training, biodefense is not well integrated into the curricula of most internal medicine residency or pulmonary and critical care medicine fellowship programs. Emergency medicine training programs have led the way in this field (Chung et al. 2004; Terndrup et al. 2005; Moye et al. 2007). However, depending on the nature of the event, victims of bioterrorism and naturally-occurring disasters may present not only to the emergency department but also to community-based practices, in-patient hospital wards, and intensive care units. Therefore, it is imperative that educators in other medical specialties devise innovative educational programs to address these deficiencies.

This study represents a collaborative effort between members of the departments of pulmonary and critical care medicine, infectious diseases, and emergency medicine. As natural disasters are rare events, and bioterrorism even rarer, the authors felt that the inclusion of high-fidelity human simulation in the training program represented a unique opportunity to prepare trainees to recognize and respond appropriately to these situations through an active, experiential learning process (Gordon et al. 2004). Although our primary aim was to develop a disaster response and emerging pathogens curriculum for Internal Medicine residents and Pulmonary and Critical Care fellows, our secondary goal was to allow participants to work through realistic critical care scenarios with subsequent provision of constructive feedback and evaluation regarding teamwork and individual performance. The Accreditation Council of Graduate Medical Education (ACGME) and expert medical educators are currently encouraging the implementation of innovative teaching strategies and evaluation methods such as high fidelity simulation to better achieve clinical competence among physicians-in-training (Philibert & Leach 2005).

Table

Practice points

Internists are poorly trained to recognize and respond to bioterrorism and other natural and man-made disasters. A disaster preparedness training curriculum that included an active, experiential learning component significantly improved Internal Medicine residents’ knowledge regarding the epidemiology and clinical manifestations of specific biologic agents and effective utilization of PPE. However, without internal reinforcement, this effect was not long-lasting. Participants also reported improved teamwork ability and increased confidence regarding critical care skills and procedures. In a number of small studies, experiential learning through simulation training has been shown to be more efficacious in the teaching of critical thinking, clinical management, teamwork, and psychomotor skills. As the cost and logistics of simulation based education are important considerations, additional research is needed in order to further define the most effective applications of simulation-based medical education.

Methods

Subjects

All thirty residents enrolled in the Memorial Hospital of Rhode Island/Warren Alpert Medical School of Brown University Internal Medicine residency program took part in this pilot study. There were 10 study participants in each of 3 postgraduate years of training. The initial study took place between July and November 2004. Retesting of this cohort took place one year later. A control group of 30 residents enrolled in the same program, who did not take part in the training, underwent the objective testing between November 2007 and January 2008. Funding for the study was provided by the Rhode Island Department of Health. The study was formally exempted for review by the Institutional Review Board of Memorial Hospital of Rhode Island.

Curriculum development

The disaster preparedness curriculum consisted of a series of four didactic sessions, each one hour in length. In addition, participants were given a manual of selected supplemental readings, and took part in three simulated real-time clinical scenarios. The content of the didactic sessions and manual included the following: (1) overall risk assessment; (2) discussion of specific threats, including naturally-occurring infectious diseases, weaponized biological, chemical and radiologic agents; (3) indications for and appropriate use of PPE; (4) public health infrastructure and reporting procedures; (5) the physician's role in a public health emergency; and (6) a discussion of the psychological consequences of disasters (Table 1).

Table 1.  Instructional and evaluative methodology for biodefense and disaster preparedness curriculum

Topics	Instructional Methods	Time Allotted	Location	Staffing	Evaluation/Feedback
• Introduction to disaster medicine	• Didactic lecture	1 hour	Classroom	1 Physician faculty	Objective testing
• Public health infrastructure and resources	• Supplemental reading materials			1 Public Health Administrator
• Overall risk assessment
• Psychologic consequences
• Biologic agents	• Didactic lecture	1 hour	Classroom	1 Physician faculty	Objective testing
	• Supplemental reading materials
	Scenario 1: Smallpox	30 ;min	High-fidelity simulation center	2 Physician faculty	Videotaped with debriefing and feedback
	• Experiential learning			3 Support staff
	Scenario 2: Tularemia	30 ;min	High-fidelity simulation center	2 Physician faculty	Videotaped with debriefing and feedback
	• Experiential learning			3 Support staff
• Chemical agents	• Didactic lecture	1 hour	Classroom	1 Physician faculty	Objective testing
	• Supplemental reading materials
	Scenario 3: Sarin	30 ;min	High-fidelity simulation center	2 Physician faculty	Videotaped with debriefing and feedback
	• Experiential learning			3 Support staff
• Radiologic agents	• Didactic lecture	1 hour	Classroom	1 Physician faculty
• Bombs
• Blast injuries
• PPE	• Didactic Lecture	30 ;min	Classroom	1 Physician faculty	Debriefing and feedback
	• Practicum	30 ;min	High-fidelity simulation center	2 Physician faculty	Debriefing and feedback
				3 Support staff

The second portion of the curriculum entailed a four-hour training seminar at the Rhode Island Hospital Emergency Department Simulation Center (Table 1). Members of the investigative team developed three scripts for simulated real-time clinical scenarios utilizing technologically sophisticated mannequins (Sim-Man, Laerdal, Wappingers Falls, NY, USA) equipped with hemodynamic monitoring and interactive capability. Biologic agents used in the scenarios were in Centers for Disease Control and Prevention (CDC) risk category A, as these agents are felt to present the greatest potential for adverse public health impact, mortality, dissemination, and civil disruption (Rotz 2002). The first scenario involved a case of smallpox. Residents were expected to make the appropriate diagnosis, understand the mode of transmission, initiate appropriate infection control measures, and invoke appropriate reporting procedures. The second case was inhalational tularemia. Expectations were similar to the first scenario, with the additional requirement of initiation of appropriate antimicrobial treatment. Ancillary goals of this scenario included resuscitation of a critically ill patient and the practice of intubation, ventilator management, and chest tube placement. The final scenario involved exposure to the chemical agent sarin. Residents were expected to recognize the signs and symptoms of nerve agent exposure, recognize the need for decontamination prior to entering the health care facility, utilize PPE effectively, provide appropriate therapy to the patient, and initiate public health reporting. Critical care procedural skills included resuscitation, intubation, and mechanical ventilation. Teamwork and communication skills were practiced and evaluated in all scenarios (Table 2).

Table 2.  High fidelity simulation scenarios

Scenario	Case presentation	Set-up	Educational Goals/Interventions
1: Biologic agent (Smallpox)	• 25 year-old woman following syncopal episode while shopping	• Emergency room patient bay	• Recognition of clinical presentation of smallpox
	• Notes two day history of malaise, fever, sore throat, lightheadedness, and nausea	• Mannequin with maquillage, interactive capability	• Know treatment and prevention guidelines
	• On physical exam numerous discrete erythematous papular lesions on posterior pharynx, face, proximal upper and lower extremities		• Understand transmission routes
			• Initiation of appropriate infection control measures
			• Recognize need for and initiate public health reporting
2: Aerosolized biologic agent (Tularemia)	• 55 year old male journalist	• ICU patient room	• Recognition of the possibility of a weaponized biologic agent
	• Acute onset fever, chills, headache, sore throat, non-productive cough, shortness of breath and pleuritic chest pain	• Interactive capability	• Know treatment guidelines
	• Chest radiograph shows bilateral hilar adenopathy and infiltrates	• Hemodynamic/ECG monitoring	• Understand transmission routes
	• Condition deteriorates; subsequent reporting of two similar cases		• Initiation of appropriate infection control measures
			• Recognize need for and initiate public health reporting
			• Practice skills:
			• Intubation, ventilator management
			• Resuscitation, hemodynamic management
			• Chest tube, central line placement
3: Inhalational agent (Sarin)	• Explosion in public shopping area	• Emergency room	• Recognize signs and symptoms of nerve agent exposure
	• Five victims, one dies at scene	• Decontamination chamber	• Decontamination outside of the ER
	• Four arrive via EMS with rapid development of combinations of the following: miosis, rhinorrhea, chest tightness, sweating, abdominal pain, vomiting, fasiculations, and seizure activity	• Interactive capability	• Don and demonstrate use of appropriate PPE
		• Hemodynamic/ECG monitoring	• Triage and initiate appropriate specific treatment as well as the ‘ABC's’–airway, breathing, and circulation
			• Recognize the important of teamwork and crew resource management
			• Practice skills:
			• Intubation
			• Resuscitation, hemodynamic management

In addition to taking part in one of the real-time scenarios as part of a team, each of the residents also observed the other two scenarios via a one-way mirror. They also received instruction and hands-on training regarding PPE and decontamination procedures. Each of the sessions was videotaped and followed by a debriefing session facilitated by a faculty member. The tapes were reviewed and all participants actively engaged in the constructive feedback and subjective evaluation process.

Evaluation of the educational program

Following completion of the training program, objective testing was performed. The scores of the participants (academic year 2004–5) were compared to those of a control group of residents, evenly matched for postgraduate year of training (academic year 2007–8). Participants were tested again one year later, including graduates, who were contacted by e-mail, or telephone. The Wilcoxon rank sum test was used to compare the scores of the entire control group to those of the participants as well as the scores of both control group residents and participants based upon postgraduate year (PGY) training level. Scores were expressed as percentages correct. The Wilcoxon matched pairs signed ranks test was used to determine whether the participant's knowledge was retained after one year (SAS, version 9.1; SAS Institute, Cary, NC, USA). A two-sided p value of <0.05 was considered significant.

Each of the participants completed a self-assessment of knowledge pre-and post-training, as well as a pre- and post-training assessment of their ability to appropriately recognize the indications for and effectively use PPE. Subjective responses were scored on an ordinal scale from 1 to 4 to reflect levels that ranged from ‘poor’ to ‘excellent’ (1 ;= ;Poor, 2 ;= ;Fair, 3 ;= ;Good and 4 ;= ;Excellent). Improvement with training was assessed by subtracting pre-training score from post-training score. A difference of ‘0’ signified no improvement, a difference of ‘1’ good improvement, and a difference of ‘2’ excellent improvement. The Wilcoxon matched pairs signed ranks test was used to determine whether differences were significant.

Results

Twenty-two of the participants completed the immediate post-training objective testing (PGY-1 n ;= ;8, PGY-2 n ;= ;4, PGY-3 n ;= ;10). Twenty-five (PGY-2 n ;= ;9, PGY-3 n ;= ;10, PGY-4 n ;= ;7) completed the follow-up testing one year later. Thirty residents, 10 at each PGY level comprised the control group, and all completed the testing. As seen in Figure 1, immediately following the training program, participant's knowledge was significantly better than that of the control group, with mean test scores 66.8% ;± ;11.8% SD vs. 50% ;± ;13.1% SD, p ;< ;0.0001. One year later, test scores significantly diminished (55.7% ;± ;14.6% SD, p ;= ;0.006). When compared to the mean scores of the control group (50% ;± ;13.1% SD), the scores of the participants were higher, but this difference was not significant (p ;= ;0.247). As shown in Figure 2, there was a non-significant trend toward increasing levels of knowledge based upon PGY level in the control group (44.2% ;± ;12.7%, 55.3% ;± ;10.0%, 50.0% ;± ;14.9%, p ;= ;0.118). Following training, there were no differences in test scores based upon level of training (66.8% ;± ;12.7%, 67.1% ;± ;20.3%, 66.5% ;± ;6.3%, p ;= ;0.77).

Figure 1. Biodefense and disaster preparedness knowledge in the participant group immediately following training compared to that of the control group (expressed as mean percentage correct ;+ ;1 SD).

Figure 2. Comparison of knowledge based upon postgraduate year (PGY) training level (expressed as mean percentage correct ;+ ;1 SD).

All participants responded to the immediate post-training subjective questionnaire (n ;= ;30). Nine (30%) residents reported their knowledge of biologic agents as poor before training. The remaining 21 (70%) reported their knowledge as fair. After training, only 1 (3.3%) resident assessed his/her knowledge as poor, 11 (36.7%) as fair, 17 (56.7%) as good and 1 (3.3%) as excellent. Eight (26.7%) residents reported no improvement with training, 17 (56.7%) good improvement, and 5 (16.7%) reported excellent improvement with training.

One (3.3%) resident reported his/her level of understanding and ability to use personal protective equipment (PPE) effectively as poor before training, the majority 26 (86.7%) as fair and the remainder 3 (10%) as good. No resident reported his/her level of understanding and ability to use PPE as excellent before training. After training, no resident reported his/her level of understanding and ability to use PPE as either poor or fair. The majority 26 (86.7%) reported it as good and the remainder 4 (13.3%) as excellent. Two (6.7%) residents reported no improvement with training, 24 (80%) reported good improvement and 4 (13.3%) reported excellent improvement with training. These subjective improvements with training were statistically significant for both knowledge of biologic agents and their clinical manifestations and level of understanding and ability to use PPE effectively (p ;< ;0.0001). In the post-training survey, participants rated the educational experience as excellent, reporting increased confidence regarding performance of critical care skills and procedures as well as improved teamwork abilities.

One year later, participants were surveyed again. Twenty-eight of 30 responded. Self-reported knowledge of biologic agents was described by one resident as poor (3.3%), 13 as fair (43.3%), and 14 as good (46.7%). These assessments were not significantly different from those immediately following the course (p ;= ;0.69). When surveyed regarding understanding and ability to effectively use PPE, no respondents self-assessed their level as poor, 4 (13.3%) reported fair, 22 (73.3%) reported good, and 2 (6.7%) as excellent (p ;= ;0.06). Although this difference did not reach statistical significance, there was a trend toward diminution, consistent with the findings of the objective testing.

Discussion

In this pilot study, a disaster preparedness training program consisting of a combination of didactics and active, experiential learning through high fidelity human simulation significantly improved residents’ knowledge regarding the epidemiology and clinical manifestations of specific biologic agents and appropriate, effective utilization of PPE. In addition, participants reported improved teamwork ability and increased confidence regarding performance of critical care skills and procedures.

In order to address the need for enhanced physician education regarding recognition and management of emerging pathogens, bioterrorism, and other disasters, novel curricula have been developed and reported in the literature by other groups of medical educators. One attractive method of disseminating information in terms of cost and accessibility are web-based programs. Those utilizing a passive learning process have reported mixed success (Chung et al. 2004; Terndrup et al. 2005). An interactive, case-based online module significantly improved Internal Medicine resident and attending physician's short term level of knowledge regarding potential biologic agents (Cosgrove et al. 2005).

As high fidelity human simulation presents a unique opportunity for trainees to experience and prepare for uncommon events, and active, experiential learning may be more effective at the graduate medical education level (Williams et al. 1999; Philibert & Leach 2005), the authors feel that this modality is particularly suited to inclusion in a disaster and biodefense training curriculum. High-fidelity simulation training was first utilized in the medical field with the development of anesthesia crisis resource management training (ACRM) (Howard et al. 1992). It was subsequently adopted and customized by other specialties, most notably Emergency Medicine (McLaughlin et al. 2002; Gordon et al. 2004; Binstadt et al. 2006; Moye et al. 2007). The subjective response of participants has been very enthusiastic (Howard et al. 1992; Morgan & Cleave-Hogg 2000; Gordon et al. 2001; Cleave-Hogg & Morgan 2002; McLaughlin et al. 2002; Binstadt et al. 2006). The objective educational benefits related to learning and retention of medical knowledge have been less well studied and quantified. The most potentially valuable utilization of this methodology is in the teaching of cognitive and psychomotor skills, which are less amenable to traditional didactic methodologies. There are some data to show that simulation improves the acquisition and retention of critical assessment, clinical management, and procedural skills compared to traditional teaching methods (Chopra et al. 1994; Mayo et al. 2004; Steadman et al. 2006; Wayne et al. 2006). In the present study, some of the critical care clinical skills included in the scenario designs included teamwork and crisis resource management; recognition of impending clinical deterioration; resuscitation of shock; practice of the psychomotor skills required for performing procedures; and ventilator management. Following this training, participants reported increased levels of self-confidence in their skills in these areas.

Recently, there have been a number of other disaster preparedness training programs aimed at preparing medical students as well as traditional emergency responders such as emergency physicians, nurses, paramedics, and emergency medical technicians (EMTs) reported in the literature (Parrish et al. 2005; Miller et al. 2006; Scott et al. 2006; Subbarao et al. 2006). The improvements in objective knowledge and subjective preparedness noted in the internal medicine residents involved in this study were mirrored by the findings of investigators who developed a similar course including high fidelity human simulation training for medical students at Texas A&M (Parrish et al. 2005). Subbarao et al. (2006) found that a course based upon high-fidelity human simulation and video clinical vignette instruction significantly improved the scores of first responders and emergency department personnel on video clinical vignette-based objective testing. Similarly, a 16-hour, two-day interactive terrorism emergency response curriculum including high-fidelity simulation significantly improved objective testing, performance levels, and self-confidence ratings of participants (Miller et al. 2006; Scott et al. 2006).

Our data shows that a pilot biodefense and disaster curriculum designed for Internal Medicine residents utilizing high fidelity simulation training significantly improved both objective and subjective ability to respond to disaster. The financial cost of this benefit was not insignificant, as the program cost approximately $20,000. This included equipment, technical support, and the simulation center-based faculty member's time. It did not include the preparation and active participation time of the other faculty members. Of note, the initial improvement in knowledge and subjective level of preparedness did not appear to be maintained one year following training. Thus, it appears that re-exposure to the didactic material and/or experiential portion of the training is necessary to maintain this increased level of expertise over time.

The response of the participants in this study was exceptionally positive. The authors believe that the experiential nature of the high-fidelity simulation portion of this Biodefense and Disaster Preparedness training course was instrumental in engaging the trainees. The realism of the exercises to the participants was readily apparent in their performance during the scenarios. The videotape review and feedback sessions following each session effectively reinforced key teaching points, and participants were able not only to learn through their own experiences, but by critiquing others’ as well. Finally, the training program was set up to embrace an ‘all disasters preparedness’ approach. Thus, while learners might feel that they were unlikely to encounter some of the disaster scenarios, they were able to recognize that the knowledge set, skills, and techniques learned could be put to use in a situation that they considered more likely to occur.

In conclusion, experiential learning through simulation training has been shown to be more efficacious in the teaching of critical thinking, clinical management, teamwork, and psychomotor skills in a small number of studies. As the cost and logistics of simulation-based education are important considerations, further research is needed in order to determine the situations in which simulation based education is most useful (Bond et al. 2007). Plans for further development and assessment of this innovative Bioterrorism and Disaster Response curriculum therefore include the utilization of more objective means of knowledge and skills assessment, a comparison of efficacy to other educational modalities, and an assessment of construct validity.

Acknowledgements

The authors would like to acknowledge the Rhode Island Department of Health, through whom this work was funded, the contributions of Rhode Island Hospital Emergency Department Simulation Center staff members, and Mary Roberts, MS, for assistance with the statistical analysis.

Additional information

Notes on contributors

Eleanor M. Summerhill

ELEANOR M. SUMMERHILL, MD is the Director of the Internal Medicine Residency Program and an Assistant Professor of Medicine in the Division of Pulmonary and Critical Care Medicine at Memorial Hospital of Rhode Island and the Warren Alpert Medical School at Brown University.

Milan C. Mathew

MILAN C. MATHEW MD, MPH is a Senior Resident in the Internal Medicine Residency Program at Memorial Hospital of Rhode Island and the Warren Alpert Medical School of Medicine at Brown University. He obtained his MPH in Epidemiology from the University of Massachusetts Amherst and has prior experience with the Cochrane Collaboration.

Sally Stipho

SALLY STIPHO, MD is currently a Gastroenterology fellow at the Carl T. Hayden VA Medical Center in Phoenix, Arizona. She is a former Chief Medical Resident and Assistant Instructor in Medicine at the Memorial Hospital of Rhode Island and the Warren Alpert Medical School at Brown University.

Andrew W. Artenstein

ANDREW W. ARTENSTEIN, MD is the Physician-in-Chief of the Department of Medicine and Director of the Center for Biodefense and Emerging Pathogens (CBEP) at Memorial Hospital of Rhode Island. He is an Associate Professor of Medicine and Community Health at the Warren Alpert Medical School at Brown University.

Liudvikas Jagminas

LIUDVIKAS JAGMINAS, MD is the Physician-in-Chief of the Department of Emergency Medicine at Memorial Hospital of Rhode Island, and an Assistant Professor of Emergency Medicine at the Warren Alpert Medical School at Brown University. He is the past Assistant Director of the Disaster Medical Training Institute of Rhode Island.

Patricia M. Russo-Magno

PATRICIA M. RUSSO-MAGNO, MD is a Clinical Intensivist and Pulmonologist at Memorial Hospital of Rhode Island. She is a Clinical Assistant Professor at Warren Alpert Medical School of Medicine at Brown University. She served as Instructor of Medical Readiness Training in Biological and Chemical Warfare in The United States Air Force.

Susan Potter

SUSAN POTTER, RN, MA is the Educational Coordinator for the Internal Medicine Residency Program, Memorial Hospital of Rhode Island. She is a Clinical Teaching Associate at the Warren Alpert Medical School at Brown University. She is actively involved in curricular development as well as medical education research.

Marc J. Shapiro

MARC J SHAPIRO, MD designed and founded the Rhode Island Hospital Medical Simulation Center in 2002, acted as center director for the first three years of operation, and currently serves in a consultant role. He is an Associate Professor of Emergency Medicine at the Warren Alpert Medical School at Brown University.