Simulation in bronchoscopy: current and future perspectives

Abstract

Objective

To provide an overview of current literature that informs how to approach simulation practice of bronchoscopy and discuss how findings from other simulation research can help inform the use of simulation in bronchoscopy training.

Summary

We conducted a literature search on simulation training of bronchoscopy and divided relevant studies in three categories: 1) structuring simulation training in bronchoscopy, 2) assessment of competence in bronchoscopy training, and 3) development of cheap alternatives for bronchoscopy simulation.

Conclusion

Bronchoscopy simulation is effective, and the training should be structured as distributed practice with mastery learning criteria (ie, training until a certain level of competence is achieved). Dyad practice (training in pairs) is possible and may increase utility of available simulators. Trainee performance should be assessed with assessment tools with established validity. Three-dimensional printing is a promising new technology opening possibilities for developing cheap simulators with innovative features.

Keywords:

• simulation
• bronchoscopy
• training
• assessment

Introduction

Bronchoscopy is a central and important clinical procedure used in a variety of specialties such as anesthesiology, critical care medicine, pulmonology, and thoracic surgery. The learning curve for new bronchoscopists is steep and highly individual.¹ Acquisition of bronchoscopy skills should therefore start in a simulated setting, instead of training on patients right away.

The use of bronchoscopy simulator training has been found superior to traditional apprenticeship training when comparing novices’ performance on patients.² Furthermore, simulator-trained novices were superior to nontrained novices in terms of performance on patients.³ This is in accordance with a meta-analysis that found simulator training to be superior compared to no simulator training, across many simulators and medical specialties.⁴

Simulators have been validated as useful for bronchoscopy simulation training in several studies.^2,5–8 Thus, it is the aim of this study to give an overview of current literature that informs how to approach simulation practice of bronchoscopy and discuss how findings from other simulation research can help inform the use of simulation in bronchoscopy training.

Methods

Our group conducted an extensive systematic review on simulation in airway endoscopy with a database search in July 2016.⁹ For this review, we updated the search (July 2017) to include the latest research. Articles’ relevance for simulation training of bronchoscopy was assessed independently by two of the authors.

Article inclusion in the review was based on their relevance for:

1. Structuring simulation training in bronchoscopy

2. Assessment of competence in bronchoscopy training

3. Development of cheap alternatives for bronchoscopy simulation

Results

Structuring simulation training

In 2001, the effect of a short simulation training course for novice bronchoscopists was investigated.¹⁰ Five pulmonary fellows received an eight-hour course, with introduction, supervised simulator training, and unsupervised simulator training. A pre–post test in the simulator showed improvement in maneuvering the bronchoscope and more thorough examination. The novices achieved comparable performance with four experienced bronchoscopists (>200 procedures) and even missed fewer bronchial segments.

A cohort study with pulmonary fellows from seven different medical schools studied the effect on introducing simulation training during the standard apprenticeship training offered at each institution.¹ All participants had their 5th, 10th, 15th, 20th, 30th, 50th, 75th, and 100th clinical bronchoscopy assessed in order to construct learning curves for both cohorts. In the second cohort, participants performed 20 bronchoscopies on a virtual simulator after their fifth clinical bronchoscopy. Both groups showed a steep learning curve for their first 30 clinical bronchoscopies, but the simulator-trained cohort demonstrated a faster increase in performance of clinical bronchoscopy on patients.

A randomized study compared dyad practice (training in pairs of two) to individual practice.¹¹ Both groups of medical students received the same amount of training, but the dyad group had to alternate between training and observing. Based on simulator metrics, there was no difference in performance between dyad and individual practice.

The same group investigated the difference between one-day practice and distributed practice of bronchoscopy simulation training.¹² Training time was equal between groups, but spaced over 3 weeks, for the distributed practice group. Using simulator metrics to assess effect of training, no difference in performance was found between the two groups and the authors conclude that there is no added benefit of distributing practice for bronchoscopy training. Table 1 provides an overview of studies on structuring of simulation training.

Table 1 Study description and key findings of studies investigating how to structure bronchoscopy training

Study	Year	Type of study	Participants’ experience level (number)	Intervention	Assessment	Key findings
Colt et al^Citation10	2001	Pre-posttest of novices and comparison to experienced bronchoscopists	-Trainees in pulmonology, bronchoscopy novices (5) -Experienced bronchoscopists (4)	2 h introduction, 2 h supervised and 4 h unsupervised training on virtual reality simulator	Simulator metrics (wall contact, red-out, missed segments, time) Manikin (time, missed segments) by nonblinded rater	-Pre–post: less wall contact, fewer missed segments, no difference in time or red-out. -Expert comparison: Trainees missed fewer segments than experienced, otherwise comparable performance
Wahidi et al^Citation1	2010	Cohort study	Trainees in pulmonology, bronchoscopy novices (22 in first cohort, 25 in second cohort)	20 bronchoscopies on virtual simulator (second cohort)	Performance on patients, nonblinded rating	-Steep learning curve for first 30 bronchoscopies. -Faster performance improvement after intervention in second cohort -Large variation in skills after 50 bronchoscopies
Bjerrum et al^Citation11	2014	Randomized study	Medical students, bronchoscopy novices (36)	Dyad practice vs individual practice on virtual simulator	Simulator metrics (wall contact, red-out, missed segments, time)	No difference in performance between dyad and individual practice
Bjerrum et al^Citation12	2016	Randomized study	Trainees in pulmonology, bronchoscopy novices (20)	1 day practice vs distributed practice over 3 weeks on virtual simulator	Simulator metrics (wall contact, red-out, missed segments, time)	No difference in performance between 1 day practice and weekly distributed practice

Assessment of competence in bronchoscopy simulation

Davoudi et al¹³ developed two assessment tools, for bronchoscopy training, the Bronchoscopy Skills and Tasks Assessment Tool (BSTAT) and Bronchoscopy Step-by-Step Evaluation Tool. The BSTAT assesses the trainee on bronchial anatomy and mucosal abnormality knowledge, scope maneuvering, equipment handling, and diagnostic maneuvers, scored on a checklist. It is designed to be used both with virtual simulation, on manikins, and during clinical bronchoscopy. The Bronchoscopy Step-by-Step Evaluation Tool is a tool developed to aid during the training of standardized training modules to gradually teach bronchoscopy skills. Both tools were assessed for reliability and validity, and both were found reliable when administered by more assessors and valid by their ability to discriminate between novices and more experienced bronchoscopists. However, they were not able to significantly discriminate the performance of intermediates and experts in bronchoscopy.

The same group applied the BSTAT in a study of various measures on learning gain.¹⁴ In a pre–post test study of novices at a one-day bronchoscopy course, performances were used to calculate a “class-average normalized gain.” In short, it is the difference in pre- and posttest scores seen in comparison to the maximum attainable posttest score, thus informing how much was gained from the course relative to the maximally achievable score. The authors concluded that it was possible to measure an improvement in technical skills after the course using this calculation.

Since the BSTAT was not able to discriminate between intermediates and experts, an alternative assessment approach was suggested. In a study by Konge et al,¹⁵ an assessment procedure with six simulated bronchoscopies with increasing difficulty was investigated. With a checklist administered by two blinded raters, it was possible to reliably discriminate among novices, intermediates, and experts in bronchoscopy. Video recording of the procedures made blinding of raters possible in this study, but removed the possibility to assess scope handling. This important aspect of bronchoscopy was investigated with an innovative new method: Using a low-cost motion-sensor from a video game console, it was possible to objectively distinguish bronchoscope handling by novices, intermediates, and experts.¹⁶ The authors commented that this approach opens the opportunity for trainees in bronchoscopy to receive automated feedback on their scope handling during training, without the need of an instructor being present.

Developing cheap alternatives for simulation

Various cheaper alternatives to the virtual simulators have been developed, starting with simple nonanatomical labyrinths to train the hand–eye coordination of scopic movements.¹⁷ A homemade model of the bronchial tree for anatomy training has also been developed using newspaper and vinilic glue.¹⁸

Recently, three-dimensional (3D) printing has been used to increase the realism of homemade bronchoscopy simulators. Two studies describe the development of 3D-printed airways for bronchoscopy training.^19,20 Both studies use open source software to process a patient’s computed tomography scan and print anatomically realistic models, and bronchoscopists agreed that these had high anatomical fidelity. An added feature of the model developed by Osswald et al²⁰ is a screw-cap in each bronchial segment so that artificial mucus or blood can be injected during training. The same research group compared their 3D-printed model with two commercially available simulators and found the 3D-printed model to be the most anatomically realistic of the three.²¹

Discussion

Structuring training beyond bronchoscopy simulation

As it has been thoroughly established, the use of simulator training in general improves performance of practical procedures.⁴ Thus, in this review we chose to focus on how to structure training efficiently. The concept of spaced repetition (distributing training volume over time) was investigated, and no difference between massed (one day) and distributed simulation training of bronchoscopy was found.¹² This is in contrast to a variety of other studies in simulation education that find spaced repetition to be superior to massed practice.^22–24 The study by Bjerrum et al¹² is however limited by the use of simulator metrics, which have been shown to be a crude measure of performance in bronchoscopy training.^2,6

Also using simulator metrics for assessment, Bjerrum et al¹¹ found no difference in performance between dyad and individual practice of simulated bronchoscopy. Two other studies of medical simulation of lumbar puncture and ultrasound training confirm the possibility of having two trainees practicing at the same time with results as good as individual practice.^25,26 These findings open the opportunity for more effective use of simulators, as more trainees can train at the same time, thus doubling the utility of simulators.

A relatively short amount of practice was able to improve bronchoscopy performance of complete novices,¹⁰ but regarding how much training is needed, the study by Wahidi et al¹ showed large variability in performance at 50th bronchoscopy also in their simulator-trained cohort. This finding suggests that training needs are variable among trainees and should not be limited to a set amount of training for all. Instead, trainees should practice until reaching an established level of competence before moving on to performance on patients, a concept known as mastery learning, which has recently been found highly beneficial in a review in the journal Advances in Medical Education and Practice.²⁷

The above findings are further emphasized by a comprehensive review and meta-analysis identifying the key elements to improve simulation training outcomes.²⁸ Besides the already mentioned concepts of spaced repetition and mastery learning, Cook et al²⁸ recommend the use of feedback during training, a varying range of difficulty during training, and individualized learning.

Assessment of competence beyond bronchoscopy simulation

The studies investigating assessment tools demonstrate ways to thoroughly assess the performance of simulated bronchoscopy in order to ready trainees for clinical bronchoscopy. The tools can assist instructors in pointing out areas of training that need improvement (allowing targeted feedback) before the trainee moves on to the next level of training (mastery learning), which ensures the established level of competence is reached. Established pass/fail criteria should direct when the trainee is ready to move on to supervised performance on patients. Konge et al²⁹ established pass/fail criteria both for the previously mentioned simulator-based test¹⁵ and a test for use in the clinical setting.³⁰ These criteria can be used for certification purposes, an important part of competence-based medical education.³¹

The assessment tool devised by Konge et al²⁹ is a global rating scale, as opposed to the checklist assessment tool BSTAT.¹³ Global rating scales have higher reliability and may be able to assess finer nuances of the trainees’ performance than a checklist with yes/no answers is capable of.³²

The class-average normalized gain is not for individual assessment of competence, but may be used for evaluating effectiveness of entire courses or new educational intervention.¹⁴

Developing cheap alternatives for simulation beyond bronchoscopy

The studies of 3D-printed bronchoscopy simulators demonstrate cheap alternatives to the commercially available simulators. Simulation of other procedures has been explored, such as 3D-printed simulators for surgical airway management, spinal needling, and endovascular procedures.^33,34 In surgery, 3D-printed models have been used in preoperative planning.³⁵ This could be used in bronchoscopy training as well, where the bronchoscopist could train on a 3D model of a specific patient case before performing the procedure.

Cheaper 3D printers are increasingly made available and will allow a wider usage of highly customizable manufacturing of simulators in the future, also for smaller departments. An added advantage could be to pair a 3D-printed bronchial tree with a single-use bronchoscope for off-site training outside the simulation center, thus increasing training volume.³⁶

The future of clinical bronchoscopy

This review is focused on simulation training of bronchoscopy as it is practiced today, but the procedure itself is also evolving, and the training of bronchoscopy needs to adapt to these changes.³⁷ As an example, the technology for biopsy taking with endobronchial ultrasound may soon be integrated in a standard bronchoscope, which will necessitate integration of simulated endobronchial ultrasound training in standard bronchoscopy training, including training in recognizing the ultrasound anatomy of mediastinal lymph nodes and vessels.³⁸ Navigated bronchoscopy, where preoperative computed tomography scans are used to guide biopsy taking, is also increasingly used and thus needs incorporation in simulation training as well.³⁹ The currently available simulators are mostly focusing on basic bronchial anatomy and scope maneuvering. Future simulators should incorporate decision-making scenarios to allow for complex skills training beyond basic proficiency.

In our own simulation center, we have taken the current body of literature on simulation training in bronchoscopy into consideration when constructing the simulation based bronchoscopy course: The trainee is introduced by an expert in bronchoscopy, reviews the bronchial anatomy, and gets feedback on initial performance. Afterward, spaced repetition practice program (distributed learning) on our simulators follows, including the possibility of practicing with a colleague (dyad practice). The training sets mastery learning criteria, and ends with a test of performance with validated assessment tools.

Conclusion

Bronchoscopy simulation is effective, and the training should be structured as distributed practice with mastery learning criteria. Dyad practice is possible and may increase utility of available simulators. Trainee performance should be assessed with assessment tools with established validity. 3D printing is a promising new technology opening possibilities for developing cheap simulators with innovative features.

Disclosure

The authors report no conflicts of interest in this work.