A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery

ABSTRACT

Background: Research proves that the apprenticeship model, which is the gold standard for training surgical residents, is obsolete. For that reason, there is a continuing effort toward the development of high-fidelity surgical simulators to replace the apprenticeship model. Applying Virtual Reality Augmented Reality (AR) and Mixed Reality (MR) in surgical simulators increases the fidelity, level of immersion and overall experience of these simulators.

Areas covered: The objective of this review is to provide a comprehensive overview of the application of VR, AR and MR for distinct surgical disciplines, including maxillofacial surgery and neurosurgery. The current developments in these areas, as well as potential future directions, are discussed.

Expert opinion: The key components for incorporating VR into surgical simulators are visual and haptic rendering. These components ensure that the user is completely immersed in the virtual environment and can interact in the same way as in the physical world. The key components for the application of AR and MR into surgical simulators include the tracking system as well as the visual rendering. The advantages of these surgical simulators are the ability to perform user evaluations and increase the training frequency of surgical residents.

KEYWORDS:

• Virtual reality (VR)
• augmented reality (AR)
• mixed reality (MR)
• surgical simulation
• surgical training

Declaration of interest

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Reviewer Disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Additional information

Funding

This work was supported by grants from National Key R&D Program of China (2017YFB1302900), National Natural Science Foundation of China (81971709; 81828003; M-0019; 82011530141), the Foundation of Science and Technology Commission of Shanghai Municipality (19510712200; 20490740700), Shanghai Jiao Tong University Foundation on Medical and Technological Joint Science Research (ZH2018ZDA15; YG2019ZDA06; ZH2018QNA23), and 2020 Key Research project of Xiamen Municipal Government (3502Z20201030). In addition, it was supported by the Austrian Science Fund (FWF) KLI 678-B31 and by CAMed (COMET K-Project 871132), which is funded by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT), and the Austrian Federal Ministry for Digital and Economic Affairs (BMDW), and the Styrian Business Promotion Agency (SFG). Further, the TU Graz Lead Project ‘Mechanics, Modelling and Simulation of Aortic Dissection’.