Abstract
Background
Procedural skills have historically been taught at the bedside. In this study, we aimed to increase resident knowledge of uncommon emergency medical procedures to increase residents’ procedural skills in common and uncommon emergency medical procedures and to integrate cognitive training with hands-on procedural instruction using high- and low-fidelity simulation.
Methods
We developed 13 anatomically/physiologically-based procedure modules focusing on uncommon clinical procedures and/or those requiring higher levels of technical skills. A departmental expert directed each session with collaboration from colleagues in related subspecialties. Sessions were developed based on Manthey and Fitch’s stages of procedural competency including 1) knowledge acquisition, 2) experience/technical skill development, and 3) competency evaluation. We then distributed a brief, 10-question, online survey to our residents in order to solicit feedback regarding their perceptions of increased knowledge and ability in uncommon and common emergency medical procedures, and their perception of the effectiveness of integrated cognitive training with hands-on instruction through high- and low-fidelity simulation.
Results
Fifty percent of our residents (11/22) responded to our survey. Responses indicated the procedure series helped with understanding of both uncommon (65% strongly agreed [SA], 35% agreed [A]) and common (55% SA, 45% A) emergency medicine procedures and increased residents’ ability to perform uncommon (55% SA, 45% A) and common (45% SA, 55% A) emergency medical procedures. In addition, survey results indicated that the residents were able to reach their goal numbers.
Conclusion
Based on survey results, the procedure series improved our residents’ perceived understanding of and perceived ability to perform uncommon and more technically challenging procedures. Further, results suggest that the use of a cognitive curriculum model as developed by Manthey and Fitch is adaptable and could be modified to fit the needs of other medical specialties.
Introduction
Procedural skills have historically been taught at the bedside in medicine. Many residents are still taught by the “see one, do one, teach one” adage, where experience with procedures is not standardized and is largely informal.Citation1 Most procedural skills are taught during patient-based encounters as the clinical need arises.Citation2 Some procedures are commonly encountered in the clinical environment, such as central venous lines, endotracheal intubation, and laceration repair; while others appear to be more rarely encountered, including needle thoracentesis, cricothyroidotomy, and urgent thoracotomy for trauma.Citation3 This may be because the indications for such procedures require a very specific clinical scenario, which may be rare, or the procedure itself may be a last ditch effort to save a patient’s life. For example, “correctly performed cricothyroidotomy may be life-saving in a cannot-ventilate, cannot-intubate situation. However, emergent cricothyroidotomy is performed infrequently and can be difficult because of the lack of training and skill retention.”Citation4
A review of our graduating residents’ procedure logs revealed several procedures consistently requiring supplementation in the lab. One example – cricothyrotomy – was logged 60% of the time in the lab versus 40% on actual patients. On the other hand, dislocations were consistently performed in the clinical environment, with 0% performed in a procedure lab ().
Figure 1 Percent of reported procedures performed in lab.
Exposure to less common procedures is inconsistent, unpredictable, and inadequate for learning. To control for these variables, the Residency Review Committee (RRC) for each specialty established minimum requirements for procedural competency. Each resident must complete a set number of a given procedure to be considered competent in that procedure; however, these numbers are often not reflective of actual experience and ability and while these thresholds are determined by expert consensus, they may not indicate individual competency levels.
High- and low-fidelity simulation provides a way to teach the less commonly encountered procedures during clinical practice and has the potential to improve both patient care and safety through practice of procedural skills prior to clinical need.Citation5 In addition, “high fidelity simulations have a number of potential advantages over actual patient experiences: they can provide standardized and graded experiences, they can reduce the use of faculty staff instructor time; and they can provide opportunities for contact with rare, life-threatening situations in a low-risk environment”.Citation5 It is this last piece that plays a critical role in emergency medicine education.
The effectiveness of high fidelity simulations, simulated patients, and anatomic simulators are well-documented.Citation3–Citation7 The most common high fidelity simulations use manikins with realistic features responding physiologically to interventions and performance of procedures. In addition, “simulation has been shown to be effective in teaching complex emergency procedural skills”,Citation3 and these skills can be retained for at least 1 year after a single session.Citation3
In our emergency medicine (EM) residency program, procedural skills are typically taught at the bedside as clinical need and opportunity present. However, given the recognized limitations of this bedside teaching method, the unavailability of a high-fidelity simulation center at our academic site, and our residents’ requests for more formal instruction prior to actual clinical experience, we designed an innovative 1-year, longitudinal, curriculum in emergency medicine procedures to address this need. Our goals were to increase resident exposure to uncommon procedures enabling residents to meet RRC required minimum or “goal” numbers, and to integrate cognitive training with hands-on procedural instruction using high- and low-fidelity simulation.
Methods
Curriculum development
We obtained retrospective University of Arizona Institutional Review Board (IRB) approval to survey our residents to assess our curriculum. Procedure modules were developed based upon the Accreditation Council of Graduate Medical Education (ACGME) core competencies. We organized the modules into 13 anatomical/physiological topics and focused the procedure modules on uncommon clinical procedures and/or those requiring higher levels of technical skill to complete (). Each individual session was designed to be 120 minutes long, including a brief introduction, three to four 20-minute stations, a summary, and questions and answers at the end. A departmental expert in the related field directed each session with collaboration from colleagues in other departments such as Orthopedics/Sports Medicine, Critical Care Medicine, Ophthalmology, EM Ultrasound, and Family Medicine/Maternal–Fetal Medicine. The sessions, based on Manthey and Fitch’s stages of procedural competency,Citation2 were as follows:
Table 1 Selected procedure series topics
Stage 1: knowledge acquisition: the module begins with an initial 20-minute introduction by the director of the session or the expert in the related clinical field.
Stages 2 and 3: exposure/technical skill development: the introduction is followed by an 80-minute hands-on procedure lab comprising of four stations (20 minutes at each station). These stations include one to two key procedures, one case-based/oral board preparation station, and one imaging station with hands-on ultrasonography and/or review of radiographic images. A clinical case-based or oral board station is included to demonstrate indications and contraindications for each procedure, and a station with relevant radiological images is presented to assist with our more visual learners to solidify other clinical and anatomical material. Stations are led by EM faculty, colleagues from other specialties and departments, or upper-level residents with previously demonstrated superior proficiency.
Stage 4: competency evaluation: competency is assessed at each procedure station by the expert leading the station. Corrective action is taken until participants are able to correctly demonstrate performance of the procedure. In addition, the final component of each module includes a 20-minute question/answer session, which serves as a self-reflective session to assess residents’ perceived competency with each procedure. At this time, the experts are able to reinforce knowledge acquisition and facilitate retention of learned skills.Citation2
Curriculum evaluation
At the conclusion of the curriculum, we distributed a brief online survey to our postgraduate year (PGY)-1, -2, and -3 EM residents, in order to solicit additional feedback regarding the curriculum (see for sample survey questions). Since a review of the literature failed to identify an existing validated assessment tool, our department developed this survey. One of our investigators with graduate training in health professions education developed the survey questions, which were then reviewed by a second investigator with a professional degree in education. Finally, our study group reviewed and evaluated the questions prior to distribution to our residents.
Table 2 Survey Questions
Results
Fifty percent of our residents (11/22) responded to our 10-question online survey. The survey responses consisted of four categories, “strongly agree” (SA), “agree” (A), “ disagree” (D), and “strongly disagree” (SD). These are reported as the percentage of responses in –.
Figure 2 Responses to main questions.
Figure 3 Helped me reach goal numbers of procedures.
Figure 4 What aspects of the procedure curriculum were beneficial?
Figure 5 Radiological adjuncts.
Residents responded that the procedure series helped with their understanding of both uncommon (65% SA, 35% A) and common (55% SA, 45% A) EM procedures as well as with their ability to perform uncommon (55% SA, 45% A) and common (45% SA, 55% A) procedures. There was less agreement about their mastery of these procedures, with 45% of respondents responding “D” ().
Residents uniformly agreed that the procedure series helped them reach goal numbers of cricothyroidotomy (100% SA or A), cardiac pacing (90% SA or A), and slightly more than half with chest tubes (60% SA or A), splints (60% SA or A), and emergency deliveries (60% SA or A). Half (50% SA or A) of residents reported that the procedure series helped them reach goal numbers of pediatric medical resuscitations, while only 30% of residents reported that the procedure series helped them reach goal numbers of central line placement and adult medical resuscitations ().
In addition, residents responded that practicing difficult (85% SA or A) and uncommon (100% SA or A) procedures was an important aspect of the procedure series curriculum ().
Residents also reported the radiological adjuncts to be less helpful. While 60% (SA or A) reported improvement in their interpretation of plain films, only a few responded with “SA” (0%) or “A” (40%) with improvement in their CT scan interpretation ().
Discussion
We used a cognitive curriculum model developed by Manthey and Fitch and applied it to uncommon and technically challenging EM procedures in an environment without convenient access to a high-fidelity simulation center.Citation2 This allowed for a deliberately designed curriculum built on solid educational theory with our residents’ needs in mind. This model could easily be modified for training residents in other medical specialties by creating directed curricula to specialty specific procedures.
In our survey, residents responded that the procedure series helped with both their understanding and performance of uncommon and common emergency procedures, but were less in agreement about the implications of the curriculum on their mastery of these procedures. This may be because the curriculum was not designed for mastery, but rather to introduce procedures less commonly encountered in the clinical environment that would later be further refined in the clinical environment. This was not an unexpected result.
For the majority of our respondents, the procedure series helped them meet their goal numbers for some procedures, including cricothyroidotomy and cardiac pacing. These are relatively rarely encountered in the clinical environment, but are procedures that need to be accurately performed in a timely manner when indicated. Roughly half of our respondents reported the procedure series helped them achieve goal numbers with chest tubes, splints, emergency deliveries, and pediatric medical resuscitations. We expected this number to be higher; however, this may be because we have dedicated rotations that commonly encounter indications for these procedures, including Trauma, Sports Medicine (Orthopedics), Obstetrics, and Pediatric Intensive Care Unit. Only 30% of our respondents reported the procedure series helped them reach goal numbers of central line placement and adult medical resuscitations. This is likely because these procedures are more common in our clinical environment, and therefore practice during the workshop time did not significantly add to their accumulated numbers.
Typically, learners and instructors express great satisfaction with the use of simulation as an educational tool.Citation4 Our results support this – both formal, via our survey, and informal feedback from our residents is extremely positive, with many residents asking for further procedure sessions, and many of our volunteer instructors have returned to teach subsequent sessions. In fact, this is the reason the didactic time devoted to procedure series was extended from 13 hours the previous year to 26 hours this academic year.
The generalizability of our curriculum is limited by the fact we still need a mechanism to assess learner skill competency beyond the date of the procedure workshop and in clinical settings. At this time, our only mechanism to obtain feedback on the curriculum is to review informal feedback from the residents and attending physicians, as well as our online survey results. It would also be interesting to note if there are differences in PGY year for perceived usefulness of the curriculum, but unfortunately we did not collect demographic data from our survey respondents. In the future, we plan to add this to our survey in order to make further modifications and developments to our procedure curriculum. In addition, further study is needed to assess the transfer of learned procedural skills in high-fidelity simulation to the clinical environment.
Our curriculum design is also limited by the fact that although we had experts at each station making modifications to residents’ procedural skills development, we did not objectively measure procedural competency. In future modules of our curriculum, we plan to objectively assess the procedural skills of our participants, both pre- and post-module to demonstrate improvement in their actual skills from our curriculum. For example, we could include a pre-assessment of residents’ perceptions regarding their knowledge and ability of procedure skills, include a checklist for the experts to complete on each resident, and videotape the residents’ actual procedure for later self-reflection.
While our response rate to our survey was only 50%, “response rates between 40% and 70% do not hamper validity”;Citation8 however, we acknowledge some might question the validity of our survey due to the relatively small sample size.Citation8 With only 22 residents currently in our program, we were fortunate to obtain 11 responses (50%). Also, a small residency program may skew the generalizability of a similar procedure curriculum for another, potentially larger, program.
Our future plans for the curriculum, based on these findings, along with feedback from the participants, include continuation of the 13 modules. These modules represent what we considered and survey results confirmed to be uncommonly encountered procedures and those requiring higher level of technical skills. Our residents were able to count the procedures done in the lab toward their procedure credentialing requirement goals, which is especially helpful for uncommon procedures. We plan to continue to track these procedures to ensure our residents are gaining the experience they need for eventual independent practice. It would be interesting to repeat a survey on our residents to see if their comfort level was perceived to be improved when they eventually performed these procedures on a patient.
The procedure series curriculum was well-received by our residents. Initial feedback suggests it helped improve their perception of their competency and understanding of uncommon and more technically challenging procedures. This model is adaptable and could be modified to fit the needs of other medical specialties.
Disclosure
The views expressed in this manuscript are those of the authors and not an official position of the institution. The authors report no conflicts of interest in this work.
References
- MasonWTStrikePWSee one, do one, teach one – is this still how it works? A comparison of the medical and nursing professions in the teaching of practical proceduresMed Teach200325666466615369918
- MantheyDFitchMStages of competency for medical proceduresClin Teach20129531731922994471
- BoetSBorgesBCRNalkVNComplex procedural skills are retained for a minimum of 1 yr after a single high-fidelity simulation training sessionBr J Anaesth2011107453353921659406
- WongDTPrabhuAJColomaMImasogleNChungFFWhat is the minimum training required for successful cricothyroidotomy? A study in mannequinsAnesthesiology20039834935312552192
- NestelDGroomJEikeland-HuseboSO’DonnellJMSimulation for learning and teaching procedural skills: the state of the scienceSimul Healthc2011671013
- de GiovanniDRobertsTNormanGRelative effectiveness of high-versus low-fidelity simulation in learning heart soundsMed Educ200943766166819573189
- IsenbergBSMcGaghieWCPetrusaERGordonDLScaleseRJFeatures and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic reviewMed Teach2005271102816147767
- DavesRStandard definitionsLavrakasPJEnclyclopedia of Survey Research MethodsThousand Oaks, CASage Publications, Inc2008835838