94
Views
0
CrossRef citations to date
0
Altmetric
Articles

Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal – a validation study

ORCID Icon, , &
Pages 399-403 | Received 17 May 2021, Accepted 20 Jul 2021, Published online: 02 Aug 2021
 

Abstract

Objective

The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning.

Design

This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor™ virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process.

Results

Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach’s alpha = 0.14, p = 0.15) and across the three modules (Cronbach’s alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences.

Conclusion

Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Log in via your institution

Log in to Taylor & Francis Online

PDF download + Online access
  • 48 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
USD 65.00 Add to cart
* Local tax will be added as applicable

Related Research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.