Analysis of anesthesia chief resident competencies in anesthesia crisis management simulation

Abstract

Background

Anesthesia crisis management (ACM) simulation would expose anesthesiology residents and trainees to dynamic emergencies that need quick yet effective responses to gain their core strength and capabilities as anesthesiologists. Our department had already employed ACM simulation into modules and practiced in examinations. The aim is to enhance professional working ethics and human responsibility in residents, which would be accomplished in semi-realistic simulation. This would be a useful tool for lecturers to evaluate current teaching methods and measure protégés in daily emergencies, which should be done better in continuous pattern.

Methods

We analyzed the ACM simulation scores from resident examination using Strata SE programs. The examination consisted of four topics in which points counted for each topic ranged from 0 to 15 and then would be counted into percentage for passing grade. The required minimum passing grade for each topic was 75%, in which the average score for one topic was 11.25.

Discussion

Overall average score for all subjects was 12.07, which already achieved the 75% passing grade with a cutoff point of 11.25. Postsurgical bleeding and cannot intubate–cannot oxygenate topics were the two bottom topics that could not achieve the 75% passing grade. Preoperative evaluation and decision making were two lowest non-technical skill scores in the examination.

Conclusion

Weakest core competencies were found in preoperative evaluation and decision making even though all subjects had already passed the minimum requirements of 75% passing grade. Postsurgical bleeding and cannot intubate–cannot oxygenate were also topics with lowest scores in core competencies. These findings would suggest for innovation and reevaluation of current teaching.

Keywords:

• anesthesia
• resident competencies
• crisis management simulation

Background

Anesthesiology and intensive care residency training are characterized with mentoring and bedside teaching method involving center-based teaching hospital and its recuperating staffs. Education and training are done in a stepwise manner, starting from basic science continued with advanced practice and clinical skills with repetitive and modified techniques.^1–4

Chief resident is a final stage considered with highest and optimum capabilities of anesthesia resident in accordance to a newly established anesthesiologist. Years of experience would polish the dexterity, but standard requirements and capabilities to work standard procedures are already built.

However, vigilance and emergency intuition should be owned and internalized in every daily working progress. Near misses and sentinel events still hold around 2% of all perioperative events and would ask for more than just aftermath management. Prevention started from education, and regular training is needed to prepare anesthesiologists for the worst case scenarios.^1,2

Anesthesia crisis management (ACM) had already been suggested since 1990s, but still could not be implemented in all anesthesiology-based teaching institutions because of the high cost of its equipment and technology. Based on aviation training, anesthesiology residents and trainees would be exposed to dynamic emergencies that need quick yet effective responses to build intuition and deep conscience as highly skilled medical personnel.^1–4

Anesthesiology and Intensive Care Department in Udayana University, Sanglah Hospital, Denpasar, Bali, had already employed ACM simulation since 2 years ago and adapted into modules and innovative study guides that are practiced in examinations for chief residents. This advanced simulation is considered to fit perfectly for chief residents in order to gain their core strength and capabilities as soon-to-be anesthesiologists.

The unique and sophisticated ACM simulation method would be a useful tool for teachers and lecturers to evaluate current teaching and training methods and also to measure their protégés in daily emergency and vigilance exposures. Continuing and regular evaluation could also be analyzed to collect insight data and perspective of current advances in developing teaching department.

Our aim is to enhance professional working ethics and human responsibility in anesthesiology residents, which would be accomplished in semi-realistic simulations such as ACM. In these cases of simulation, residents vigilance would be trained without compromising real patients because they would interact with mannequins and scenarios.

Introduction

In the year of 1987–1990s, Gaba et al¹ were doing a simulation research on several anesthesiologists from different years of background. They exposed the subjects into emergency situations, and the responses were evaluated and measured. They found that decision making in the operating theater was characterized with naturalistic behavior, and lots of problem occurred relatively related to inconsistent and dynamic diseases, environment, and also objectives. Short and precise timings were also included and were complicated with many different personalities in the operation theater competing to get their work done. High-risk procedures, normative and organized work ethics, and lack of positive feedback were also found.^1–4

In order to bridge and enhance interpersonal communication to minimize sentinel and near miss events, Gaba et al collaborated a simulation training method based on aviation industry, which is called Anesthesia Crisis Resource Management (ACRM), consisting of comprehensive training compiled with technical and non-technical skills related to cognitive and attitude in crisis and emergency situations, which were considered relevant for daily practice. Gaba et al expected that the participants would gain several skills such as:^1–4

1. Specific technical skills in various work domains according to clinical situations (for example, cardiology, orthopedics, obstetrics, surgery, and intensive care)

2. Dynamic decision-making skill, resource management skill, leadership, and applicable teamwork skills in every emergency situation.

3. Effective teamwork with different personalities and attitude.

4. Structured learning method with debriefing system and individual or team analysis of unwanted incidents.

Based on an in-depth analysis of the performance of humans exposed to emergency situations, specifically in aviation industry, human factors have been found to still contribute more than equipment failure in serious accidents. Suboptimum human performance was caused not by lack of knowledge but by lack of communication, leadership, vigilance, decision making, and teamwork. These components are called the non-technical skills that could be defined also as cognitive, social, and personal skills that act as adjunctive in technical skills to contribute in safer and much more efficient work performance. These non-technical skills were getting much more attention and also included in the ACRM simulation method developed by Gaba et al. Many non-technical skills were found, but only four of them were included and considered important in anesthesia practice, which were task management, teamwork, situation awareness, and decision making. The non-technical skills are also naturally related to ACM based on its observational technique to evaluate their valuable components, which obviously could be accomplished by the simulation method as seen in ACM.^5,6

Simulation method, in spite of its high cost, is still considered as an integral education method embedded in a specific medical course for those who work in acute and urgent situations. Simulation would help participants to be exposed to real-life emergencies, develop skills, without risking others, and also gain positive feedback from experienced instructors. Frequent exposures are proven to effectively enhance confidence and performance in real-life situations and also an exhilarating experience. Teamwork would also be built, and interpersonal skills are also developed gradually.^1,7,8

Methods

Our eligible ten chief residents were examined through ACM simulation-based Objective Structured Clinical Examination (OSCE).¹⁰ The examination consisted of four simulation topics that are septic shock, postsurgical bleeding, foreign body aspiration, and cannot intubate–cannot oxygenate (CICO) topic. Points counted for each topic ranged from 0 to 15 (0: lowest score; 15: maximum score) and then would be counted into percentage for passing grade. Required minimum passing grade for each topic was 75%, in which the average score for one topic was 11.25.

ACM simulation scores from residents’ examination were collected and analyzed using Strata SE programs. All the eligible subjects were counted and taken as subjects in this retrospective observational analysis. The Anesthesia Non-Technical Skill (ANTS) components that would be defined as core competencies were calculated and also analyzed among the subjects.

Discussion

Average ACM simulation scores

Based on the analysis results, the overall average score for all subjects was 12.07 (minimum score was 0, and maximum score was 15), which already achieved the 75% passing grade with a cutoff point of 11.25. Of the ten subjects, seven of them had already passed and only for three subjects the ACM average scores were lower than 11.25. As shown in Table 1, the average ACM simulation score per topic was 13.29 for septic shock, 10.65 for postsurgical bleeding, 13.62 for foreign body aspiration, and 10.71 for CICO topic. Postsurgical bleeding and CICO topics were the two bottom topics that could not achieve the 75% passing grade, with only three subjects surpassed.

Table 1 Average ACM simulation scores per topic

Subjects (n=10)	Average per subject	Septic shock topic	Postsurgical bleeding topic	Foreign body aspiration topic	CICO topic
DE	13.925	15	12.7	15	13
DA	13.7	14.6	12.7	15	12.5
SW	10.875	12.7	7.2	12.6	11
DO	11.575	12.7	10.1	11.8	11.7
AL	12.675	12.7	13	14.5	10.5
AR	12.1	14.2	8.4	15	10.8
RU	10.8	12.8	8.3	13.3	8.8
WL	11.95	11.8	10.8	14	11.2
KO	10.19	11.8	9.7	10.96	8.3
AG	12.875	14.6	13.6	14	9.3
Average in total	12.07	13.29	10.65	13.62	10.71

Abbreviations: ACM, anesthesia crisis management; CICO, cannot intubate–cannot oxygenate.

Chief residents’ core competencies evaluation

Besides technical skills, our ACM simulation also includes non-technical skill evaluation, in which the following were considered as core competencies: preoperative evaluation, decision-making, management, and complication awareness. Our subjects were examined and analyzed on their performances in each simulation topic.

Based on the core competencies score diagram in Figure 1, preoperative evaluation and decision-making competencies were the two topics with lowest scores in the ACM simulation examination. Technical skills yielded in management were not the lowest core to be concerned of, yet the non-technical skills in preoperative evaluation and decision making need to be reminded and continuously considered for subjects. Further analysis in each core competency would be explained in Tables 2–6.

Figure 1 Core competencies score in ACM simulation.

Abbreviation: ACM, anesthesia crisis management.

Table 2 The Anesthesia Non-Technical Skill system categories

Categories	Elements
Task management	Planning and preparing Prioritizing Providing and maintaining standards Identifying and utilizing resources
Teamwork	Coordinating activities with team members Exchanging information Using authority and assertiveness Assessing capabilities Supporting others
Situation awareness	Gathering information Recognizing and understanding Anticipating
Decision making	Identifying options Balancing risks and selecting options Reevaluating

Table 6 The ACM simulation scores based on complication awareness competency

Subjects (n=10)	Average per subject	Septic shock topic	CICO topic
DE	4	6	2
DA	3.5	5.6	1.5
SW	2.95	4.9	1
DO	2.95	4.9	1
AL	3	4.5	1.5
AR	3.75	6	1.5
RU	3.15	5.3	1
WL	2.65	4.3	1
KO	3.5	6	1
AG	3.75	6	1.5
Average in total	3.32 (max: 4)	5.35 (max: 6)	1.3 (max: 2)

Note: max: maximum points could be achieved.

Abbreviations: ACM, anesthesia crisis management; CICO, cannot intubate–cannot oxygenate.

A 2006 research by Savoldelli et al⁵ had employed the ANTS behavioral marker system to assess anesthesiologists’ non-technical skills. This scoring system is hierarchical and consists of four main categories of task management, teamwork, situation awareness, and decision making. A four-point scale is used to rate the performance of the subjects (Table 2).

This study had embedded the ANTS components, categories, and assessments in the evaluation and outlined them in the following four competencies that are similar to those ANTS categories.

ACM simulation scores based on preoperative evaluation competency

Results in Table 3 show that lowest ACM simulation scores based on preoperative evaluation core are in postsurgical bleeding (2.4 [60% compared with maximum points]) and CICO (2.7 [67.5% compared with maximum points]) topics in which deep analytical process and thorough history-taking ability should take place before proper resuscitation and airway management would be taken.

Table 3 The ACM simulation scores based on preoperative evaluation competency

Subjects (n=10)	Average per subject	Shock septic topic	Postsurgical bleeding topic	Foreign body aspiration topic	CICO topic
DE	3.45	2	2.8	6	3
DA	3.75	2	4	6	3
SW	2	1.6	0.8	3.6	2
DO	3.15	2	2.8	4.8	3
AL	3.55	2	3.2	6	3
AR	3.125	2	2	6	2.5
RU	2.525	2	0.8	4.8	2.5
WL	3.675	2	3.2	6	3.5
KO	2	1.2	1.2	3.6	2
AG	3.42	2	3.2	6	2.5
Average in total	3.06 (max:4)	1.88 (max: 2)	2.4 (max: 4)	5.28 (max: 6)	2.7 (max: 4)

Note: max: maximum points could be achieved.

Abbreviations: ACM, anesthesia crisis management; CICO, cannot intubate–cannot oxygenate.

ACM simulation scores based on decision-making competency

Table 4 shows that based on decision-making competency, the lowest average score is in CICO topic (1.91 [63.67% compared with maximum points]), ironically the most emergency situation in which quick, effective, and accurate decision making is required. The average score did not pass the 75% passing grade that would need direct attention to enhance and upgrade.

Table 4 The ACM simulation scores based on decision-making competency

Subjects (n=10)	Average per subject	Septic shock topic	Postsurgical bleeding topic	Foreign body aspiration topic	CICO topic
DE	2.5	2	3	3	2
DA	2.5	2	3	3	2
SW	2.35	2	2.4	3	2
DO	2.95	2	2.7	1.5	2
AL	2.35	2	2.4	3	2
AR	2.32	2	1.8	3	2.5
RU	2	1.3	2.4	3	1.3
WL	2.08	1.3	3	2	2
KO	2.32	2	3	3	1.3
AG	2.25	2	3	2	2
Average in total	2.27 (max: 2.75)	1.86 (max: 2)	2.67 (max: 3)	2.65 (max: 3)	1.91 (max: 3)

Note: max: maximum points could be achieved.

Abbreviations: ACM, anesthesia crisis management; CICO, cannot intubate–cannot oxygenate.

ACM simulation scores based on management competency

In the management competency, results in Table 5 show that postsurgical bleeding topic holds the lowest score in management competency (5.57 [69.6% compared with maximum points]). Surgery and consecutive bleeding are daily practice for an anesthesiologist; however, the results were contradictive in which subjects could not pass the 75% passing grade of postsurgical bleeding topic management. Further analysis is required to enlighten this problem and discuss ways to recuperate.

Table 5 The ACM simulation scores based on management competency

Subjects (n=10)	Average per subject	Septic shock topic	Postsurgical bleeding topic	Foreign body aspiration topic	CICO topic
DE	5.97	5	6.9	6	6
DA	5.67	5	5.7	6	6
SW	5.05	4.2	4	6	6
DO	4.9	3.8	4.6	5.5	5.7
AL	5.275	4.2	7.4	5.5	4
AR	4.775	4.2	4.6	6	4.3
RU	4.7	4.2	5.1	5.5	4
WL	4.875	4.2	4.6	6	4.7
KO	4.59	4.6	5.4	4.36	4
AG	5.325	4.6	7.4	6	3.3
Average in total	5.11 (max: 6.25)	4.4 (max: 5)	5.57 (max: 8)	5.69 (max: 6)	4.8 (max: 6)

Note: max: maximum points could be achieved.

Abbreviations: ACM, anesthesia crisis management; CICO, cannot intubate–cannot oxygenate.

ACM simulation scores based on complication awareness competency

Different than other competencies, the complication awareness competency was specifically evaluated for septic shock and CICO topics only. Table 6 shows that the lowest average score in complication awareness is in CICO topic (1.3 [65% compared with maximum points]). This topic demanded vigilance to be prepared for any possibilities and worst case scenarios. Score below 75% passing grade would be a sign for more assistance, discussion, and debriefing.

Previous studies conducted in aviation show that a higher fidelity of simulation does not always lead to better performance. But in anesthesiology, the effects of simulator training on job performance are more difficult to demonstrate because of the diversity of the situations, and the best use of different types of simulators for training has not been established. Nyssen et al⁸ have found that simulators can contribute significantly to the improvement of performance. On the other hand, it also raised questions of whether simulators could be used to assess performance because of simulated situations always a simplification of naturalistic situations of diagnosis, retention of guidelines, and technical skills, which would be different than anesthesiologist experiences in handling crisis with a wider perspective.

Another study by Yee et al⁹ also concluded that single simulation session would improve the non-technical skills of residents. Even the study admits several limitations such as lack of control group, less familiar technology orientation, and also difficulties of creating scenarios of challenging cases, and experts suggested further improvement in non technical skills because the subjects did not achieve the maximum score.

As mentioned earlier, Gaba et al¹¹ as the pioneer of simulation in anesthesiology had suggested to evaluate anesthesiologist performance during crisis using both technical and behavioral aspects such as communication, leadership, and interpersonal conflict, which are already reported as the most critical factors associated with human error incidents. This study had tried to take the behavioral aspects into account and evaluate the core competencies related to anesthesiologist technical skills.

Limitations

Our research limitations are sample size and simplification of simulated evaluation, but we believe that much more appropriate measures and collecting more data on human performance from our field in other teaching departments in Indonesia would help to develop better understanding and finally would be better simulations to meet specific requirements for anesthesiologist training.

Conclusion

Interesting results were found in the retrospective analysis of anesthesia chief resident competencies in 2016 ACM simulation. Weakest core competencies related to ANTS were preoperative evaluation and decision making described in the abovementioned tables and figure, even though all subjects had already passed the minimum requirements of 75% passing grade. Postsurgical bleeding and CICO were also topics with lowest scores in core competencies. These findings were perplexed because those cases were regular cases that would be found in daily practice.

These unique findings would help researchers and lecturers especially in our department to innovate and reevaluate current teaching and training methods adjusted with latest evidence-based technologies.

The ACM simulation method that has already been employed by our department is a giant step to build a highly respective education project in Bali and also support availability and distribution of skilled anesthesiologists in all around Indonesia. Further development and socialization would be needed in many adjoined teaching centers.

Finally, ANTS was considered to be highly significant and also discovered in this analysis of anesthesia chief resident competencies. Technical skills based on knowledge only are not sufficient to ensure patient safety. Evaluation and decision making were also very important to be built and trained to prepare residents in handling real-life emergency situations.

Disclosure

The authors report no conflicts of interest in this work.