Predictors of virtual reality simulation bronchoscopy performance among novice bronchoscopists

Abstract

Introduction

Simulation-based training is gradually replacing the classic “apprenticeship” training model. Predictors of better performance of virtual reality simulation bronchoscopy are not clear.

Objective

We aim to explore the predictors of performance of simulation bronchoscopy among novice bronchoscopists.

Materials and methods

This is a descriptive observational cohort study conducted at King Abdulaziz University Clinical Skills and Simulation Center, Jeddah, Saudi Arabia. All participants filled a demographic questionnaire and attended a pre-simulation orientation about the requested tasks. The Simbionix bronchoscopy simulator was used in this study. First, each resident performed three trails of basic scope manipulation task to test hand–eye coordination skills. Thereafter, each resident performed the guided anatomical navigation task to accurately examine as many lung segments as possible. Results and metrics were retrieved from the simulator, and statistical analysis was performed using t-test to measure statistically significant P-value (<0.05).

Results

Fifty-three internal medicine residents participated in this study. Male residents significantly achieved higher score in the basic scope manipulation task than female residents (65% vs 46%, P<0.001). Furthermore, the percentage of time spent at mid lumen during the scope manipulation was significantly higher for males compared to female residents (48% vs 37%, P=0.003). Residents who were interested in pursuing procedure-based specialty training spent significantly less time in contact with wall (14.6% vs 20.3%, P=0.045). Smokers needed more time to finish the first task (mean 2.5 minutes vs mean 1.1 minutes, P=0.005).

Conclusion

Simulation bronchoscopy performance was different between genders, smoking status and future interest in pursuing a procedure-based career. Overall, male residents performed better than female residents in basic scope manipulation. Gender differences in performing simulation bronchoscopy need to be examined in future studies. Tailored educational programs may be needed to fit gender-specific skills and requirements as well as future career interests.

Keywords:

• bronchoscopy
• virtual reality
• simulation
• gender differences

Introduction

Bronchoscopy is a central and important clinical procedure used in various specialties such as anesthesiology, critical care medicine, pulmonology and thoracic surgery.¹ Increased concerns for patient safety have prompted an ongoing shift from apprenticeship models of medical education to approaches that insulate patients from the initial learning phase in procedural training.² Simulation-based education appears ideally suited to offer effective training in a zero-risk environment. The use of simulator-based training in the field of bronchoscopy has been found to be better than traditional apprenticeship training upon comparing novices’ performance on patients.³ A comprehensive analysis of evidence regarding the efficacy and key features of simulator-based bronchoscopy training would facilitate decision-making among educators. Several recent reviews of bronchoscopy training have offered such summaries.^4–6 We believe that in order to better plan instruction and assess the efficacy of curricular interventions, valid and reliable assessments of technical skills are needed. One such assessment that has not been widely discussed in the literature is toward understanding the predictors of bronchoscopy performance among novices and their effect in the acquisition of basic bronchoscopy skills in a simulator-based training program. A recent study by a Danish group investigated whether male and female medical students performed differently while training in a bronchoscope simulator.⁷ They opined that gender differences should be taken into consideration, when planning and organizing medical education and assessment in simulation training. Along similar lines, we wish to explore the predictors of bronchoscopy performance among novices.

Materials and methods

Sitting and study design

This is a descriptive observational cohort study conducted between January 2018 and April 2018 at King Abdulaziz University Clinical Skills and Simulation Center, Jeddah, Saudi Arabia.

Participants and procedure

All internal medicine residents in training at King Abdulaziz University Hospital were invited to participate in this study. Out of the 57 residents invited, 53 residents consented to participate. All residents were novices’ bronchoscopists without any previous bronchoscopy or simulation experience. Each resident filled a questionnaire about demographic data. Thereafter, all residents received a pre-study orientation about the requested tasks. We used the Simbionix bronchoscopy simulator (Cleveland, OH, USA). Each resident was required to do two tasks. The first task is the basic scope manipulation task that is designed to make users get accustomed to navigating the bronchoscope in a cyber environment to further develop hand–eye coordination. The resident will navigate the scope in a narrowing industrial lumen, following a guiding light and is then asked to keep the insertion tube straight and use the control unit’s roll and lever to keep the scope’s tip in mid-lumen and avoid wall contact. The score will drop with each wall contact based on lumen’s width. Each resident performed this task three times, and the software randomly selects the path each time the resident made an attempt. Metrics and results about the time of the procedure, wall contacts and final scores were retrieved from the simulator. Mean score of the three attempts was analyzed.

The second task was the guided anatomical navigation task. Each resident was asked to perform bronchial navigation maneuver with directional guidance and get performance feedback to accurately examine as many lung segments as possible. The allocated time given for this task was 5 minutes for each resident. The metrics and results in regard to the number of segments examined and the percentage of time with clear visibility were retrieved from the simulator.

This study was conducted by classifying the participatory residents based on seven categories (gender, residency levels, handedness, smoking habit, cumulative grade point average [GPA], interest in a procedure-based career and gaming experience). We included smoking status because different studies have shown that smoking has negative effects on hand coordination, speed, alertness and attention.^8,9 Smokers were defined as those who regularly smoked at least one cigarette per day for at least 1 year. Video game players were defined as those who played video games daily or occasionally, whereas residents who never played video games were considered non-video game players.¹⁰ The outcomes measured were time taken to finish the first basic scope manipulation task, total score for basic scope manipulation task, percentage of time spent at mid lumen, percentage of time spent in contact with the bronchial wall, number of lung segments examined and percentage of time with clear visibility of lung segments. Each resident performed both tasks individually, and no interaction between the researcher and the candidate was allowed during task performance.

Ethical considerations

Informed written consent was obtained from all the residents prior to recruitment. Participants were assured of the confidentiality of their responses and that all findings would be used solely for research purposes. No incentive was provided for their contribution. Permission to conduct this study was granted by the Biomedical Ethics Research Committee of King Abdulaziz University.

Statistical analysis

All the data were collected and entered into the computer. Statistical analysis was done using Statistical Package of Social Science Version 16 (SPSS Inc., Chicago, IL, USA). The quantitative data were presented in the form of mean and SD. Student’s t-test was used for quantitative data of two groups and one-way ANOVA for more than two groups. The qualitative data were presented in the form of number and percentage. A P-value of <0.05 was considered to be statistically significant.

Results

Fifty-three internal medicine residents from different residency levels participated in this study, wherein 31 (58.5%) of them were males (Table 1). Residents were predominately (90.6%) right handed. There were only 7 (13.2%) smokers, and 23 (43.4%) residents had an experience of playing video games. Among all residents, only 14 were interested in pursuing a procedure-based career, whereas 39 residents did not have such interest.

Table 1 Demographic data of the residents in this study, n=53

Variables	Values (% of total)
Gender
Male	31 (58.5)
Female	22 (41.5)
Hand orientation
Left	5 (9.4)
Right	48 (90.6)
Residency level
R1	19 (35.8)
R2	15 (28.3)
R3	12 (22.6)
Smoking status
Smoker	7 (13.2)
Non-smoker	46 (86.8)
Do you play video games?
Yes	23 (43.4)
No	30 (56.6)
Are you interested in a procedure-based career?
Yes	14 (26.4) 39
No	(73.6)

Male residents achieved significantly higher score in the basic scope manipulation task than female residents (65% vs 46%, P<0.001; Table 2). Furthermore, smokers needed more time to finish the first task than non-smokers (mean 2.5 minutes vs mean 1.1 minutes, P=0.005; Table 2). On the other hand, the remaining parameters – residency level, hand orientation, cumulative GPA score, interest in a procedure-based career and video games playing – were found to have no significant effect on the scores and time taken to finish the basic scope manipulation task (Table 2).

Table 2 Results from the basic scope manipulation task

Variables		Time taken to finish the task		Total score of the task
		Mean±SD	P-value	Mean±SD	P-value
Gender	Males	1.27±1.58	0.8	65.14±12.43	<0.001
	Females	1.35±0.55		45.9±9.19
Residency level	R1	1.09±0.65	0.16	56.25±12.5	0.196
	R2	1.88±2.09		61.52±15.46
	R3	1.19±0.40		50.66±13.57
	R4	0.78±0.40		61.009±17.16
Hand orientation	Right	1.31±1.31	0.77	56.19±14.04	0.78
	Left	1.20±0.22		58.8±20.38
Smoking status	Smoker	2.50±2.98	0.005	61.35±14.04	0.39
	Non-smoker	1.12±0.56		55.64±20.38
What is your cumulative GPA?	3–4	1.16±0.62	0.27	56.43±15.61	0.76
	4.1–5	1.44±1.62		57.32±13.62
Are you interested in a procedure-based career?	Yes	1.31±1.43	0.93	58.92±14.33	0.76
	No	1.28±0.52		52.31±14.27
Do you play video games?	Yes	1.02±0.5	0.065	59.61±14.08	0.153
	No	1.66±1.74		53.81±14.7

Abbreviation: GPA, grade point average.

Furthermore, Table 3 shows that the percentage of time spent at mid lumen during the scope manipulation task was significantly higher for males as compared to female residents (48% vs 37%, P=0.003), and they displayed significantly less time in contact with the walls during the procedure (13% vs 20%, P=0.004). Residents who were interested in pursuing procedure-based specialty training spent significantly less time in contact with the wall (14.6% vs 20.3%, P=0.045). On the other hand, the other parameters – residency level, hand orientation, smoking habit, cumulative GPA score, interest in a procedure-based career and video games playing – do not significantly affect the bronchoscopy simulation procedure performances of participatory residents.

Table 3 Time spent at mid lumen or in contact with the wall during the basic scope manipulation task

Variables		% of time spent at mid lumen		% of time in contact with wall
		Mean±SD	P-value	Mean±SD	P-value
Gender	Males	48.14±15.18	0.003	13.28±7.78	0.004
	Females	36.88±8.08		20.41±9.47
Residency level	R1	40.82±13.52	0.68	18.33±11.25	0.31
	R2	49.51±15.32		12.8±8.05
	R3	41.58±12.24		17.7±7.48
	R4	40.71±10.70		14.66±6.84
Hand orientation	Right	43.68±13.48	0.68	15.53±8.08	0.206
	Left	41.06±16.6		21.11±17.18
Smoking status	Smoker	49.41±13.07	0.212	12.95±5.33	0.159
	Non-smoker	42.52±13.63		16.67±9.57
What is your cumulative GPA?	3–4	43.6±12.95	0.92	16.61±9.94	0.72
	4.1–5	43.2±14.52		15.7±8.57
Are you interested in a procedure-based career?	Yes	44.27±12.95	0.66	14.64±8.2	0.045
	No	43.2±14.52		20.3±10.73
Do you play video games?	Yes	46.2±15.77	0.07	15.44±8.62	0.508
	No	39.8±9.39		17.07±10

Abbreviation: GPA, grade point average.

Table 4 shows the number of collisions during scope manipulation across different lumen widths among different residents. Significantly less number of collisions were observed in the wide lumen for the male residents as compared to the female residents (P<0.001). Furthermore, in wide lumen, second-year residents demonstrated significantly less number of collisions than any other residency levels (P=0.018). Meanwhile, it was noted that there was no significant variation in the collision data with respect to different resident categories.

Table 4 Results from collision data derived during the basic scope manipulation task

Variables		Number of collisions in wide lumen		Number of collisions in mid lumen		Number of collisions in narrow lumen
		Mean±SD	P-value	Mean±SD	P-value	Mean±SD	P-value
Gender	Males	1.44±1.31	<0.001	2.31±1.36	0.125	1.14±1.04	0.4
	Females	4.15±2.09		2.9±1.35		0.93±0.5
Residency level	R1	2.561±1.92	0.018	2.42±1.31	0.62	2.42±1.31	0.62
	R2	1.733±1.85		2.31±1.31		2.31±1.31
	R3	4.11±2.37		2.97±1.4		2.97±1.4
	R4	1.761±1.65		2.66±1.74		2.66±1.74
Hand orientation	Right	2.56±2.05	0.97	2.56±1.4	0.88	1.09±0.85	0.37
	Left	2.6±3.13		2.33±1.22		0.73±0.82
Smoking status	Smoker	1.66±1.18	0.23	2.47±1.18	0.101	1.71±1.87	0.027
	Non-smoker	2.7±2.21		2.55±1.41		0.95±0.54
What is your cumulative GPA?	3–4	2.78±2.24	0.48	2.23±1.3	0.101	1.03±0.59	0.88
	4.1–5	2.37±2.04		2.85±1.39		1.07±1.05
Are you interested in a procedure-based career?	Yes	2.46±1.99	0.53	2.38±1.23	0.53	1.017±.518	0.57
	No	2.88±2.64		3.02±1.67		1.16±1.45
Do you play video games?	Yes	2.15±2.04	0.105	2.54±1.49	0.132	0.95±0.50	0.33
	No	3.11±2.16		2.55±1.22		1.18±1.16

Abbreviation: GPA, grade point average.

The second task demonstrated the number of segments examined by each resident and the percentage of time with clear visibility during the 5-minute session. It was observed that there were no significant differences in the number of segments examined and the time with clear visibility across all variables (Table 5).

Table 5 Summary of examination of lung segments performed during the guided anatomical navigation task

Variables		% of time with clear visibility		Number of segments examined
		Mean±SD	P-value	Mean±SD	P-value
Gender	Males	90.17±9.78	0.43	18.67±4.8	0.39
	Females	87.95±10.39		17.5±4.97
Residency level	R1	87.21±13.66	0.60	4.48±1.02	0.86
	R2	89.67±7.16		4.76±1.23
	R3	92.25±6.62		6.27±1.81
	R4	88.86±8.45		3.78±1.42
Hand orientation	Right	56.91±14.04	0.23	18.1±4.75	0.98
	Left	58.8±20.38		18.2±6.22
Smoking status	Smoker	92.57±6.024	0.32	19.14±2.67	0.55
	Non-smoker	88.91±10.42		17.96±5.09
What is your cumulative GPA?	3–4	87.77±11.21	0.24	18.35±4.88	0.73
	4.1–5	90.96±8.56		17.89±4.86
Are you interested in a procedure-based career?	Yes	88.64±10.6	0.36	17.90±4.9	0.59
	No	91.5±7.95		18.71±4.77
Do you play video games?	Yes	90.87±8.3	0.22	17.9±4.93	0.78
	No	87.48±11.74		18.39±4.81

Abbreviation: GPA, grade point average.

Table 6 shows multiple linear regression (Enter model) for the prediction of higher score for basic scope manipulation task. The significant factors were determined by univariate analysis. Enter method was used to determine the most significant factors that predict the higher performance score. The significant factors were determined by univariate analysis. The only significant predictor for higher score was observed in male gender with high β coefficient and P-value of 0.000. Table 7 shows multiple linear regression (Enter method) for the prediction of time needed to finish the basic scope manipulation task. Enter method was used to determine the most significant factors that predict the time needed to finish the task. The significant predictors for shorter time are nonsmoking and playing video games with P-values of 0.003 and 0.03, respectively.

Table 6 Multiple linear regression model (Enter method) for predictors of higher score of basic scope manipulation task

Model	Unstandardized coefficients		Standardized coefficients	t	Sig.
	β	SE	β
Constant	77.078	24.778		3.111	0.003
Gender	−21.497	3.777	−0.733	−5.692	0.000
Residency level	0.143	0.775	0.021	0.185	0.854
Hand orientation	−0.360	5.532	–0.007	−0.065	0.948
Smoking status	4.397	4.985	0.104	0.882	0.383
Interest in a procedure-based career	−6.177	3.777	–0.189	−1.635	0.109
Habit of playing video games	4.642	3.693	0.159	1.257	0.215

Abbreviations: SE, standard error; Sig., significance.

Table 7 Multiple linear regression model (Enter method) for predictors of time needed to finish the basic scope manipulation task

Model	Unstandardized coefficients		Standardized coefficients	t	Sig.
	β	SE	β
Constant	4.084	2.509		1.628	0.111
Gender	0.125	0.382	0.050	0.328	0.745
Residency level	−0.023	0.078	−0.039	−0.293	0.771
Hand orientation	−0.076	0.560	−0.018	−0.135	0.893
Smoking status	−1.587	0.505	−0.439	−3.143	0.003
Interest in a procedure-based career	−0.391	0.383	−0.140	−1.022	0.312
Habit of playing video games	0.836	0.374	0.336	2.237	0.030

Abbreviations: SE, standard error; Sig., significance.

Discussion

Among all potential variables, this study demonstrates the significant influence of gender, smoking habit and interest in a procedure-based career on the bronchoscopy skills of residents. Our findings suggest that simulation bronchoscopy performance in regard to basic scope manipulation task was better among males, non-smokers and residents interested in pursuing a procedure-based career. How-ever, the performance of all residents was similar while undertaking the guided anatomical navigation task. The higher residency levels or cumulative GPA did not give any advantage while performing the bronchoscopy simulation procedure (Table 2). This result can be interpreted to mean that the theoretical knowledge of a procedure and the actual performance are two completely different aspects (Table 2). The duration of time spent at mid lumen during the scope manipulation and the duration of time spent in wall contact signify the effectiveness of the bronchoscopy simulation procedure performed by participatory residents.

Overall, male residents have demonstrated better performance than female residents in basic scope manipulation task, where males took less time to finish the task and also achieved higher score. Furthermore, male residents also performed better with spending more time at mid lumen and lesser time spent in contact with the bronchial wall than that of female residents. These results are in agreement with earlier reports that establish the fact that male and female medical students differ in their visuospatial abilities, interest in surgery and technical experience.^11,12

In order to perform in a given task, an important factor driving force is motivation that is necessary in an acquisition task.¹³ Better performance of the males as compared to the females can possibly be attributed to the higher motivation in males as compared to the females in a competitive setting, which was also observed in a previous study.¹⁴ Even the number of male residents who participated in this study was higher than the females, which also points toward higher motivation among male students. But, parameters such as time taken to complete a task cannot be considered as the only parameter as it usually favors men.¹⁵ Therefore, it is also imperative to design studies in such a way that appeals to both genders.

Smokers were observed to poorly fare in the basic manipulation tasks than non-smokers (Table 2). This result is in line with studies that report on the negative effects of smoking on hand–eye coordination. It has been reported that smoking can affect the speed and coordination of upper limb in a negative way and cause its deterioration.⁹ A previous study also suggest that attention and alertness may be significantly impaired in smokers compared to non-smokers.⁸ However, our observation that non-smokers performed better in the scope manipulation tasks goes against counter-intuitive reports that show performance-enhancing effects of smoking habits.¹⁶

We found that factors such as interest in a procedure-based career and prior video-gaming experience, which would otherwise be considered trivial, have a significant impact on the simulator bronchoscopy performance, as supported by earlier reports.^10,17 Put together, these observations could not stress enough the importance of practice and guidance in acquiring bronchoscopy competence.

Interestingly, no statistically significant differences were noted in the performance of the residents in guided anatomical navigation task (Table 5). All data obtained for all the residents showed similar results in terms of the number of segments visualized and the percentage of the time with clear visibility within the allocated time duration (Table 5). It is possible that resident’s skills improved during the second task as they became more comfortable with the simulation setup during the second task. These results contradict a previous study that revealed that the male students were able to examine more segments than female students.⁷

Despite providing many interesting observations, this study has certain limitations. First, the gender differences that we observed in the simulation bronchoscopy performance may be perceived as biased as a result of unconsciously or purposely helping a specific gender and not the other during the procedure. However, prohibiting any interaction between the candidates and the investigator minimized this limitation. Furthermore, all procedure results were obtained objectively from the simulator. This was an effort to overcome information bias, which is a commonly encountered problem in observational studies. Additionally, we need to consider one of the fundamental limitations of observational research, which is the inability to attribute causation to the corresponding effect.

Although this study sheds light on several interesting aspects related to the predictors of performance of simulation bronchoscopy, systematic reviews such as the recent reviews on laparoscopy performance skills^18,19 will shed more light on the efficient translation of the procedural skills from the training program to the real-time patient-based setting. A recent review has also addressed the financial outcomes of the growing trend of simulation-based medical education.²⁰ All these factors must be considered before any general conclusions can be derived regarding the demographic features that bring about competence in bronchoscopy simulator performance.

Conclusion

In this study, we found that simulation bronchoscopy performance in regard to basic scope manipulation task was better among males, non-smokers and residents interested in pursuing a procedure-based career. When organizing future simulation-based teaching, it is imperative that simulation trainers and medical institutions/departments should be made aware of the various demographic factors that affect the simulation performance in order to make it effective.

Acknowledgments

I would like to thank the staff at the Clinical Skills & Simulation Center at King Abdulaziz University, Jeddah, Saudi Arabia.

Disclosure

The author reports no conflicts of interest in this work.