Development of a leadership skills workshop in paediatric advanced resuscitation

Abstract

Background: Paediatric residency programs rarely prepare trainees to assume resuscitation team leadership roles despite the recognized need for these skills by specialty accreditation organizations. We conducted a needs-assessment survey of all residents in the McGill Pediatric Residency Program, which demonstrated that most residents had minimal or no experience at leading resuscitation events and felt unprepared to assume this role in the future.

Aims: We developed an educational intervention (workshop) and evaluated immediate and long term learning outcomes in order to determine whether residents could acquire and retain team leadership skills in pediatric advanced resuscitation.

Methods: Fifteen paediatric residents participated in a workshop that we developed to fulfill the learning needs highlighted with the needs assessment, as well as the Objectives of Training in Pediatrics from the Royal College of Physicians and Surgeons of Canada. It consisted of a plenary session followed by 2 simulated resuscitation scenarios. Team performance was evaluated by checklist. Residents were evaluated again 6 months later without prior interactive lecture. Learning was also assessed by self-reported retrospective pre/post questionnaire.

Results: Checklist score (assigning roles, limitations of team, communication, overall team atmosphere) expressed as % correct: initial workshop scenario 1 vs. scenario 2 (63 vs. 82 p < 0.05); 6-month scenario with prior workshop exposure vs. control (74 vs. 50 p < 0.01); initial workshop scenario 2 vs. 6-month scenario control (82 vs. 50 p < 0.001). Retrospective pre/post survey (5 point Likert scale) revealed self-reported learning in knowledge of tasks, impact and components of communication, avoidance of fixation errors and overall leadership performance (p < 0.001).

Conclusions: Residents acquired resuscitation team leadership skills following an educational intervention as shown by both observational checklist scores and self-reported survey. The six-month follow-up evaluation demonstrated skill retention beyond the initial intervention. A control group suggested that these results were due to completion of the first workshop.

View correction statement:

Errata

Introduction

Team leadership has been recognized as very important for many years, in areas such as business, the military, and aviation. It not only improves the team's functioning, it improves the quality of the end-product of the team's work (Helmreich 1997; Hamman 2004). Only recently has medicine started to recognize this fact, and started teaching physicians to be effective team leaders (Howard et al. 1992; Reznek et al. 2003; Marsch et al. 2004). It has been shown that teams who function with an effective team leader adhere more closely to established protocols, sustain fewer medical errors, and as a result, have a more favourable outcome for their patients, both simulated and real (Hoff et al. 1997; Cooper & Wakelam 1999; Morey et al. 2002; Devita et al. 2004, 2005; Marsch et al. 2004).

Specialty training organizations, such as the Royal College of Physicians & Surgeons of Canada, have recognized the importance of leadership skills in the day to day functioning of a physician, and as such, have mandated its’ inclusion in the curriculum of all training programs (Frank 2005). Specifically, the Royal College has listed leadership as a key competency under the “Manager” role of the CanMEDS framework (Frank 2005).

Table

Practice points

Paediatric residents do not get sufficient training in leadership skills, which is something they perceive they need. Specialty training organizations, such as the Royal College of Physicians & Surgeons of Canada, have mandated that residents learn leadership skills during their residency training. Paediatric residents perceive leadership skills training as highly valuable. It would be useful to evaluate if participation in a leadership skills workshop changes residents’ performance in real-life scenarios. This would be a very difficult analysis to perform, due to the rarity of resuscitation events in children.

Leadership skills are valuable to possess in any medical situation, but this is most apparent during a crisis or resuscitation effort. Therefore,we elected to focus our efforts on the education of leadership skills specific to resuscitation. The majority of research in this area has been conducted in the field of crisis resource management (CRM) (Howard et al. 1992; Gaba et al. 1994). This was a curriculum designed to teach anaesthetists how to be effective team leaders in the operating room. This curriculum is based on the training that pilots receive to manage critical situations. Three key areas of focus are: tasks required of a leader, effective communication skills within a team, and avoidance of fixation errors. Fixation errors are defined as “persistent failure to revise a diagnosis or plan in the face of readily available evidence that suggests a revision is necessary”(Gaba et al. 1994).

During their training program, our paediatric residents had no explicit education of the leadership skills required during a resuscitation effort. In addition, their passive learning of these skills by experience was mainly limited to observation alone, since the role of team leader naturally fell to the most senior physicians in most situations, which was usually the consultant physician or the fellow, and rarely was the resident.

With our study, we sought to assess the needs in paediatrics and to apply the principles of CRM to develop an educational intervention for paediatric residents in resuscitation. Learning outcomes following the intervention (consisting of an interactive workshop on leadership skills in paediatric resuscitation) were evaluated both immediately and after 6 months.

Methods

Twenty paediatric residents (at training levels PGY1 to PGY4) at Montreal Children's Hospital, McGill University Health Centre, voluntarily participated in our needs assessment. Not all residents were available to participate in the workshops. Fifteen participated in the first workshop, and fourteen in the second.

For the needs assessment, the residents were given a questionnaire, in which they were asked about the resuscitation situations they had been involved with, how often they got to perform the various tasks involved with a resuscitation, and how comfortable they were accomplishing these same tasks. One of the tasks asked about was functioning as the team leader. We also collected relevant background information, such as level of training, when they had last taken any resuscitation courses (PALS, APLS, ATLS, NRP), and what their career aspirations were.

Based on the needs assessment performed as described above, as well as the “Objectives of Training and Specialty Training Requirements in Paediatrics” (Frank 2005), learning objectives were developed (Appendix 1).

From the learning objectives, a curriculum was designed and a half-day workshop was organized. A brief plenary session introduced the important concepts of leadership and team functioning. The residents were then divided into small groups, after which they performed 2 different mock code scenarios. The team members were each given scripts to follow, which included different pieces of information. This was done to attempt to simulate real-life situations, where not all information is known by all team members.

The functioning of the team and the leader was evaluated by checklist, where tasks were checked as performed (2 points), not performed (0 points) or some had an option of borderline performance (1 point), depending on the task (Appendix 2). The total points were added and a percentage score was generated for the entire team. A debriefing occurred after each scenario, as well as at the end of the workshop, in a large group setting.

Trainee perception of change in knowledge and skills was evaluated using learner retrospective pre/post testing (5-item questionnaire), based on the methodology of Skeff et al. (1984). In this type of evaluation, participants were asked to rate their level of knowledge both before and after an educational intervention, with the test administered following the intervention. This approach has been shown to be a valid way of identifying participants' perceptions of learning outcomes following educational interventions. Moreover, it can mitigate the effect of participants beginning to appreciate how much they do not know after an intervention, which may falsely hide a real change in knowledge and skills.

Six months after the workshop was conducted, we ran a second workshop to assess whether learning was retained. Since some of the residents were not available to participate in both workshops, we divided them into groups based on whether they had or had not previously participated. The workshop consisted of performing 1 resuscitation scenario in teams, which was one of the same scenarios as was used during the first workshop. The residents were evaluated on their performance by the same checklist that was used previously. The plenary session followed the mock scenarios and was modified to function as a summary of the principles of CRM. Finally, the residents again completed a retrospective pre/post questionnaire in order to assess perceived learning from this workshop, from the previous workshop, as well as loss of acquired skills and knowledge in the interval time period.

A 1-tailed Student's T test was used to compare checklist scores between the first and second scenarios during workshop 1, as well as the checklist score generated from the scenario from workshop 2. We also used a 1-tailed Student's T test to compare results from the retrospective pre/post questionnaires given at the end of both workshops.

Results

The results of the needs assessment questionnaire are shown in Figure 1. Twenty residents completed our questionnaire. Mean values were generated for each procedure, for both the number of times the residents got the opportunity to perform the procedure, and the level of comfort the residents expressed with performing each procedure.

Figure 1. Result of needs assessment questionnaire. Number means how many times the resident has had the opportunity to perform the various tasks. Comfort score is based on a 5 point Likert scale, where 1 = very uncomfortable and 5 = very comfortable. A: Setup of airway equipment; B: Select appropriate airway equipment; C: Intubation D: Defibrillation; E: Insert intraosseous needle; F: Insert central venous catheter; G: Perform chest compressions; H: Be the code-team leader.

During the first workshop, the residents’ mean checklist scores improved from scenario 1 to scenario 2 (63% vs. 82%, p < 0.05) (Figure 2).

Figure 2. Performance on mock leadership scenarios. “6-month workshop control” refers to residents who did not participate in the initial workshop. “6-month workshop repeaters” refers to residents who had participated in initial workshop 6 months earlier.

Since the residents were divided into groups during the second workshop, based on whether or not they had previously participated, we were able to compare the scores of residents who participated once versus twice. Figure 2 reveals that the residents’ scores were better during the first scenario of the initial workshop than those during the 6-month workshop who had never previously participated (control). (63% vs. 50%, p < 0.05).

The checklist scores of the residents at the end of the initial workshop were better than the scores of the control group of residents who, during the 6-month workshop, ran through the scenario for the first time (82% vs. 50%, p < 0.001) (Figure 2). In addition, the scores of the residents during the 6-month workshop who had prior workshop exposure were better than the scores of the control group of residents (74% vs. 50%, p < 0.01) (Figure 2).

There was no difference between residents’ scores at the end of the initial workshop and those with prior workshop exposure during the 6-month workshop (82% vs. 74%, NS) (Figure 2).

Retrospective pre/post questionnaire results are shown in Figure 3. There was a significant improvement in their reported values on the Likert scale at the end of both workshops, when compared to values before the workshop (p < 0.001 for all items).

Figure 3. Retrospective pre/post questionnaire results from workshops 1 & 2. Score based on a 5 point Likert scale, where 1 = low level of knowledge/understanding and 5 = high level.

Discussion

It has been previously shown in anaesthesia that leadership training greatly enhances the comprehensiveness of education received as well as improves patient safety (Holzman et al. 1995; Flin & Maran 2004). Since general paediatricians often function as team leaders in the resuscitation of a critically ill child, we sought to determine whether leadership training would enhance the development of leadership skills in paediatric residents as well.

Our needs assessment revealed that the residents reported that they had very little opportunity to perform the various skills required during a resuscitation event. The exception to this was the selection of airway equipment, which was learned in a mandatory anaesthesia rotation in the PGY1 year. However, their level of comfort performing these various tasks was quite variable (Figure 1). We found no correlation between amount of experience with doing certain resuscitation skills and the level of comfort with these same skills. Other research in the acquision of skills by residents has reached the same conclusion (Nadel et al. 2000; Barnsley et al. 2004; Davis et al. 2006; Wayne et al. 2006). It was of particular interest to us to find that the residents had very little opportunity to be the leader of a code, but their level of comfort was higher than expected, in the range of somewhat uncomfortable on the Likert scale (Figure 1). It is concerning that, despite little experience, the residents felt their leadership skills were somewhat adequate.

Therefore, we developed a brief curriculum designed to teach leadership skills to our paediatric residents, in a workshop format, which would give them opportunity to learn to be an effective leader, as well as to practice these skills.

During the initial workshop, the residents’ scores improved from the first scenario to the second. From this, we concluded that learning occurred while participating in the first scenario, which carried forward to the second scenario. It is also possible, however, that the second scenario was simply easier to perform than the first. Our checklist was not formally validated before it was used. However, we derived our checklists from a previously validated Crisis Resource Management curriculum (Gaba et al. 1998), as well as a well-recognized standard for education of resuscitation skills in paediatrics, the Pediatric Advanced Life Support (PALS) course (American Heart Association 2001). Further work needs to be done to thoroughly validate our checklist.

There was a group of residents who did not participate in the first workshop, but did participate in the second (control group). Their scores from the scenario during the 6-month workshop represent baseline knowledge of residents on the concepts of leadership, since we did not introduce these concepts to them before asking them to perform the scenario. When we compared these residents’ scores with the group from the second scenario of the initial workshop, we found that they were lower. In addition, the residents’ scores in the 6-month workshop were better if they had previously participated in the initial workshop. We believe, therefore, that participation in the initial workshop did result in learning, above what they learn passively during their residency.

The residents who participated in both workshops did not see an improvement of their scores during the 6-month workshop. It is possible that no further learning occurred with repeated participation, with no decay of knowledge in between. Or, if there was decay in their knowledge in the six months that passed between the two workshops, then participation in the 6-month workshop reinforced their learning and brought their knowledge back up to where it was at the end of the initial workshop. If, in fact, this was the case, then the residents must have refreshed their knowledge simply by performing the scenario, since the plenary session was not given at the beginning of the 6-month workshop. We would have to use another method of evaluating the residents outside of the workshop to assess whether worsening of their level of knowledge did occur during the interval 6 months. In addition, we cannot rule out the possibility that the high scores seen during the 6-month workshop among the residents with prior workshop exposure represent a “training effect”, i.e. the residents recalled the specifics of the scenario and didn’t learn the concepts of leadership per se. The reason why we chose to use the exact same scenario during the 6-month workshop as was used during the initial workshop was to rule out the possibility that differences in scores were not explained by differences in the specifics of the scenario. We believe that 6 months is a long time to retain knowledge of specifics of a simulated resuscitation scenario. Therefore, we feel that a “training effect” cannot fully explain the high scores seen during the 6-month workshop, although the setup of our experiment does not allow us to completely rule out this possibility.

It came as a surprise to us that the scores of the residents who participated in the 6-month workshop for the first time (the control group) were worse than the scores of the residents who performed the first scenario during the initial workshop. We would have expected that passive learning during 6 months of residency would have resulted in better scores during the 6-month workshop, not worse. However, the second workshop was set up differently, in that the plenary session was not given before the scenario was run. Therefore, it is possible that learning occurred during the first workshop simply as a result of participating in the plenary session.

After both workshops, we gave the residents an evaluation form to fill in, which was structured in the form of a retrospective pre/post questionnaire (Skeff et al. 1984). This was done to assess whether subjective learning occurred. Traditional questionnaires ask the learners to evaluate their own level of knowledge before they participate in an educational activity and again at the end. In contrast, a retrospective pre/post questionnaire asks about level of knowledge/understanding before and after participation but at the end of the exercise. The problem with the traditional way is that sometimes the learners do not appreciate how little they know about the subject before they participate in the exercise. Therefore, when compared to traditional questionnaires, retrospective pre/post questionnaires have been shown to more accurately reflect learning during an educational activity (Skeff et al. 1984).

The results of the retrospective pre/post questionnaire reveal that subjective learning did occur as a result of participation in this educational opportunity. Taken with our objective evaluation, we feel that this confirms a greater understanding of the concepts of effective leadership and team-functioning as a result of attendance in our workshop.

We did not evaluate the residents’ performance during real-life resuscitation scenarios to determine whether the learning that occurred translated into better performance with real patients. This is logistically very difficult to do, however, since the resuscitation of children is fortunately a rare event, even at a children's hospital. Because of this, each resident only participates in a small number of resuscitations during their training. However, this piece of information would be very helpful to confirm the necessity of including a similar educational exercise in all general paediatric training programs.

Finally, it is our hope that learning leadership skills specific to resuscitation will carry over to other areas of professional practice of paediatric medicine, although we did not specifically address this with our study. Further work is needed to address this important issue.

Conclusion

We have developed a formal educational intervention for paediatric residents in the area of leadership skill acquisition, based on the principles of Crisis Resource Management. There was a perceived need for this within our resident group, as well as a normative need established by the Royal College of Physicians and Surgeons of Canada. The intervention appeared to fill that need. Further work needs to be done to validate our evaluation tool used in this educational exercise. In addition, it would be helpful to assess if the learning that occurred as a result of participation in this educational exercise translated into improved performance in real-life resuscitation events, as well as other leadership opportunities, not related to resuscitation.

Additional information

Notes on contributors

Elaine Gilfoyle

ELAINE GILFOYLE, BSc (Hons), MD, FRCPC, is a Clinical Assistant Professor at the University of Calgary. She is a member of the division of Paediatric Critical Care, Department of Paediatrics, University of Calgary. Her academic interests include medical education, specifically paediatric residency education.

Ronald Gottesman

RONALD GOTTESMAN, MD, FRCPC, FAAP, FCCM, is an Associate Professor of Paediatrics at McGill University. He is Division Head and Fellowship Program Director for Paediatric Critical Care Medicine and affiliate member of McGill's Teaching and Learning Services. He also serves on the Executive Committee of the McGill Medical Simulation Centre.

Saleem Razack

SALEEM RAZACK, MD, FRCPC, is a paediatric intensivist and Associate Professor of Paediatrics at McGill University. He completed medical school training at the University of Toronto, and residency and fellowship training at McGill. His academic interests include medical education, and is currently Director of the paediatric residency program at McGill.

Appendix 1: Learning objectives for leadership skills workshop

1. Assign roles to team members:

   • Declare yourself to be in charge of the group

   • Assign PALS algorithm to patient's current condition based on gathered information so far

   • Divide algorithm into distinct steps/actions

   • Recognize skill set of each team member

   • Match members skill set with tasks that need to be done

   • Announce role of each team member to whole team

2. Assess limitations of team members:

   • Recognize skill level of each team member

   • Anticipate difficulty of specific task

   • Compare skill level with difficulty of task to conclude if they are equal

3. Based on limitation of team members (from 2 above), formulate a plan to add skill to team, if required.

4. Continuously reassess and re-evaluate progress of resuscitation using all available information:

   • Acknowledge response or lack of desired response to intervention

   • Avoid fixation errors

   • Generate list of reasons why desired result isn’t seen

   • Examine patient to choose likely reason from list, or delegate team member to examine and report findings back to you

   • Create solution(s) to problem(s) identified

Two subtypes of importance of reassessment: physiological derangement of patient and inadequate performance of team member:

1. Demonstrate use of another algorithm or approach when expected result to an intervention is not happening

2. Critically evaluate each team member's performance and redirect him or her as needed:

   • Observe team member performing assigned task

   • Assess effects of actions of team member

   • If performance is inadequate, causing lack of desired response, then redirect team member to improve skill

1. Display effective communication during performance of resuscitation:

   • Use calm, clear voice when talking and giving orders

   • State commands clearly and precisely

   • Avoid making statements into “thin air”. Direct your orders to a team member by name.

   • Use closed communication loop: Repeat what has just been said to you and verify meaning of ambiguous messages

   • Encourage open exchange of ideas among team members by listening to all ideas and determining what is important to know or act upon

   • Defer dealing with interpersonal conflicts until after the resuscitation is finished, unless it's interfering with the performance of the team

   • Quickly manages disruptive behaviour if it is affecting overall team performance

Appendix 2: Evaluation form used during leadership training workshop

Scenario#1

Table

Assigning Roles	Yes	Borderline	No
▪ Leader declared	□		□
▪ Formulates list of tasks required	□	□	□
▪ Assigns tasks to team members	□		□
▪ Announces roles to whole team	□		□
Limitations of Team
▪ Anticipates difficulty of intubation	□	□	□
▪ Considers ability of team to intubate	□	□	□
▪ Recognizes team not capable of intubating	□		□
▪ Formulates plan to add to team: call anaesthesia	□		□
Communication
▪ Uses calm voice	□	□	□
▪ Orders stated clearly	□	□	□
▪ Orders directed to specific individuals	□	□	□
▪ Team members are made aware of important plans	□	□
Overall Team Atmosphere
▪ Team members notify each other of necessary info	□	□
▪ Team members seek info from others	□	□	□
▪ Team members speak up with persistence	□	□	□
▪ Team members acknowledge orders	□	□	□
▪ Open atmosphere among team members	□	□	□
Scenario #2
Assigning Roles	Yes	Borderline	No
▪ Leader declared	□		□
▪ Formulates list of tasks required	□	□	□
▪ Assigns tasks to team members	□	□
▪ Announces roles to whole team	□		□
Reassessment of Progress: Bagging
▪ Acknowledges lack of desired response to bagging	□		□
▪ Generates list of reasons why	□	□	□
▪ Examines patient (or delegates)	□		□
▪ Concludes ineffective bagging	□		□
▪ Redirects RT to proper technique	□
▪ Watches for response after redirection	□
Reassessment of Progress: Intubation
▪ Acknowledges lack of desired response to intubation	□
▪ Generates list of reasons why (DOPE)	□	□	□
▪ Examines patient (or delegates)	□		□
▪ Concludes pneumothorax	□		□
▪ Orders needle decompression	□		□
▪ Watches for response after intervention	□		□
Management of Distraction
▪ Leader acknowledges anxiety of RT	□	□	□
▪ Leader acknowledges impact on grp performance	□	□	□
▪ Leader attempts to redirect RT	□		□
▪ Leader replaces RT if continues distraction	□		□
▪ Leader reassesses group performance	□	□	□
Communication
Uses calm voice	□	□	□
Orders stated clearly		□	□
Orders directed to specific individuals	□	□	□
Team members are made aware of important plans	□	□
Overall Team Atmosphere
▪ Team members notify each other of necessary info	□	□
▪ Team members seek info from others	□	□	□
▪ Team members speak up with persistence	□	□	□
▪ Team members acknowledge orders	□	□	□
▪ Open atmosphere among team members	□	□	□