An innovative virtual reality training tool for the pre-hospital treatment of cranialmaxillofacial trauma

Abstract

Virtual reality (VR) surgery using the High Technology Computer Corporation Very Immersive Virtual Experience professional 2(HTC VIVE Pro2) suite is a multi-sensory, holistic surgical training experience. A multimedia combination including videos and three-dimensional interaction in VR has been developed to enable trainees to experience a realistic battlefield environment. The innovation allows trainees to interact with the individual components of the cranialmaxillofacial(CMF) anatomy and apply surgical instruments while watching close-up stereoscopic three-dimensional videos of the surgery. In this study, a novel training tool for the pre-hospital treatment of CMF trauma based on immersive virtual reality (iVR) was developed and validated. Twenty-five CMF surgeons evaluated the application for face and content validity. Using a structured assessment process, the surgeons commented on the content of the developed training tool, its realism and usability and the applicability of VR surgery for CMF trauma rescue simulation training. The results confirmed the applicability of VR for delivering training in the pre-hospital treatment of CMF trauma. Modifications were suggested to improve the user experience and interactions with the surgical instruments. This training tool is ready for testing with surgical trainees.

Keywords:

• Medical training
• surgical simulation
• virtual reality
• cranialmaxillofacial trauma

Introduction

Craniomaxillofacial (CMF) trauma is one of the most common types of trauma, which is receiving increasing attention in global public health settings due to its high morbidity and mortality rates [1]. Due to their relative exposure, CMF areas are vulnerable in combat and are prime targets for assault and self-harm. The increasing use of explosive weapons and increased terrorist activities have led to an increase in explosive injuries in CMF areas [2]. The first aid for CMF trauma was difficult and usually required specialists. When large-scale civilian and military casualties occur, many CMF trauma cannot be treated immediately [3]. However, CMF trauma can lead to adverse functional and esthetic outcomes, such as permanent defects and potentially disfiguring scars, which can significantly affect the patient’s quality of life. These injuries are often complicated by traumatic brain injuries and serious facial substance loss [4]. Due to the complexity of CMF anatomy, the difficulty of treatment, as well as the differences between daily life and wartime trauma, the rescuers generally have ‘no way to start’ in the pre-hospital treatment of CMF trauma [5].

As mentioned above, the standardized rescue training for the pre-hospital treatment of CMF trauma is an urgent problem to be solved [6,7]. Traditional methods of surgical training, including residents training using cadavers, animals and synthetic mockups, have some major drawbacks [8]. These approaches are in some cases inefficient due to the considerable number of specimens required to perform a surgical simulation. Moreover, the continuous evolution of medic science forces us to explore and practice safer novel techniques. Virtual reality (VR) based simulation environments can help address these drawbacks while serving as a platform for supplementing these current training approaches [9]. The interest toward training the surgeons with alternative methods such as VR based environments has been increasing over the recent decades [9]. The use of VR aimed at improving surgeons’ knowledge on anatomy and operating skills was first introduced by Satava et al. [10] in 1994. Ever since then, VR based surgery simulation systems have developed rapidly and several systems have been applied to train novices all over the world [11–13]. These systems have been oriented to assist specialists in medical tasks such as diagnosis, planning and simulation of different surgical procedures in a virtual environment [14,15]. Moreover, the sense of touch by means of haptic technologies has also been integrated into many of these systems [16]. Haptics allows the tactile interaction with the patient’s virtual model and the comprehensive planning of surgery by incorporating computer aided and haptic tools into all stages of the planning process [17]. Many surgical planning and simulation systems have demonstrated that enabling the sense of touch and force feedback by means of haptic technologies, gives users the capability to increase their performance by reducing the planning time and increasing manual skills [18–20]. Internet of Medical Training Things allows trainers, trainees and patients coherently over an evidence-based data-oriented simulation environment [21]. The major impending factor in achieving this, however, was none other than the delay, caused by relaying large amounts of data in a wireless manner over large distances [22].

In the area of CMF surgery, a novel training tool for Le Fort I osteotomy based on immersive virtual reality (iVR) was evaluated by seven consultant oral and maxillofacial surgeons [17]. Although the results suggested that iVR is valid to test trainees and to improve their non-surgical expertise and knowledge on orthognathic surgery, the technology was only evaluated by expert surgeons. In addition, the authors mentioned that one technological limitation of the proposed tool is the lack of haptic force feedback. On the other hand, Elledge et al. [23] reported that a virtual learning environment approach in maxillofacial surgery training was well-accepted by trainees and it improved their confidence. A comprehensive review to identify all digital and mannequin maxillofacial simulators available for education and training in maxillofacial surgery was reported by Maliha et al. [24]. A total of 22 simulators were identified: ten virtual reality haptic-based simulators, six physical model simulators and six web-based simulators; however, only nine formalized studies with low level of evidence were recognized, and the rest were descriptive in nature. The significant lack of scientific and validated studies reported in the literature clearly indicates that simulation in CMF surgery education is underused. In fact, Ahmed et al.’s [25] revealed that while other surgical disciplines have adopted simulated clinical teaching, maxillofacial surgery has only seen limited formal use; therefore, the benefits of simulation-based surgical training are apparent.

However, all the above systems mentioned mainly focused on operations procedures, such as cutting, drilling and milling, etc. At present, there is no research on VR simulation training related to the pre-hospital treatment of CMF trauma. Organizers need to be cognoscente of the growth of CMF injuries and continue to be vigilant in finding new strategies and equipment that can prevent and treat these injuries both on the battlefield and in civilian practice. Human-Computer Interaction (HCI) principles and mixed reality (MR) technology can play a pivotal role for the design of complex Extended Reality (XR) environments for surgical training, however, it requires a specific learning environment and there are limited resources available [26–29]. Therefore, our research group plans to use VR technology to build a simulation training system for the pre-hospital treatment of CMF trauma.

Materials and methods

The hardware architecture of the system

The Virtual Training System for CranioMaxillofacial Trauma Treatment (VR-CMTT) is built on the basis of a high-performance graphics workstation (DELL) and the High Technology Computer Corporation Very Immersive Virtual Experience professional 2 (HTC VIVE Pro 2) suite (Figure 1). The workstation model is Dell Precision Tower 7920^®, equipped with 32 Gigabyte (GB) of running memory, a Xeon Silver 4214^® processor and NVIDIA Quadro RTX 8000^® graphics card, running on Windows 10 operating system. HTC VIVE Pro 2 package contains two locators, two control handles and a head-mounted display. HTC VIVE Pro 2 has two 3.5-inch Active Matrix Organic Light Emitting Diode screens, providing monocular 1440 × 1600 binocular 2880 × 1600 resolution, 90 Hertz refresh rate and has G-sensor calibration, proximity distance sensor, gyroscope and other sensors.

Figure 1. VR-CMTT application scenario.

The software architecture of the system

In order to realize the simulation of the surgical process and restore the real surgical scene to the greatest extent, the VR-CMTT system will mainly develop four functional modules as shown in Figure 2. The first part is the construction of virtual scene, including the establishment of scene model and the import of model and the configuration of related attributes. The second part is model loading and rendering, mainly with the help of Unity engine to achieve dynamic loading and real-time rendering scene model. The third part is virtual interactive operation, which mainly includes the functions of button response, collision detection and ray detection. The fourth part is the assessment and evaluation module.

Figure 2. Division of system functional modules.

Scene building and model construction

The scene building and model of the VR-CMTT system mainly include virtual battlefield environment and virtual rescue scenarios. Virtual battlefield environment mainly consists of three parts: geographic environment model, battlefield entity model and battlefield special effect model. Among them, the geographical environment model includes terrain and vegetation of battlefield environment, and the physical model of the battlefield includes battlefield elements such as forts, guard towers and tents, as well as the simulation of battlefield special effects such as flame and smoke. The virtual rescue scenario mainly consists of human body model, surgical environment model and surgical instrument model. The surgical environment model includes the operating table and medical console, and the surgical instrument model includes the models of oropharyngeal airway, syringe, test paper and other models. By sampling the height map, the terrain height at a specific position in the environment can be obtained, and then the corresponding material mapping can be given to restore the real terrain environment. For different terrains in the scene, different vegetation such as grass, shrubs and trees were added using the vegetation drawing tool in the terrain editor to improve the scene details [17,30]. Based on the geographic model, the physical battlefield models such as the turrets, guard towers and tents were modeled and added to the virtual scene through Blender. Appropriate lighting and sky boxes were added to further optimize the virtual battlefield environment. Elements such as fire and smoke cannot be represented and rendered by conventional mesh models due to their random shapes and movements. For these elements, the Unity 3D Particle System is used to achieve a more ideal simulation of flames and smoke [31–33]. Particle system has a significant advantage in modeling and rendering irregular objects. It represents dynamic objects as a collection of a large number of particles and achieves the simulation of dynamic objects such as smoke and flames by giving each particle different properties such as position, velocity, shape, color and mapping. By setting the properties of particle shape, velocity, lifecycle color, lifecycle size and rendering and combining with the corresponding maps and materials, the final flame and smoke simulation is obtained.

Model loading and rendering

Collision detection

In the process of virtual simulation, collision detection can determine whether a collision occurs between objects in the virtual environment and record contact state information such as contact pairs, collision points, collision normal and embedding depth when a collision occurs, which can be used for the subsequent calculation of collision response [34–37]. Collision detection is an important step in the simulation cycle and can easily become a computational bottleneck for simulation systems when it involves intersection testing of complex objects in virtual environments. In order to improve the efficiency of collision detection, Bounding Volume Hierarchy (BVH) is often used as the collision model for complex objects. BVH is a tree-like structure that organizes spatial objects. It can quickly exclude elements pairs that are impossible to collide in the rough detection stage of collision detection, so as to avoid redundant intersection test and accelerate the collision detection process. The leaf nodes of BVH tree store all spatial objects that need to be managed and wrap these objects with simple bounding boxes for fast intersection testing. These leaf nodes are grouped and wrapped into larger bounding boxes according to certain partitioning rules to form the intermediate nodes of BVH tree, which are continuously divided in the form of recursion and finally form the tree structure. The Aixe Align Bounding Box (AABB) layer bounding box of the model is to organize the triangular surface set of the model. The AABB layered bounding boxes of skin and mandible models shows in Figure 3. It can be seen that as the layers of BVH become deeper, the enclosing boxes become smaller and more numerous, the more they can approximate the original model [38–41].

Figure 3. AABBs at different levels in BVH.

Blood simulation algorithm

During the pre-hospital treatment of CMF trauma, hemostasis of the casualty and detection of cerebrospinal fluid leakage are important parts of the treatment. In order to realize the simulation of the above key steps, realistic simulation of wounded bleeding is crucial. Although Unity 3D's built-in particle system is capable of simple fluid simulation, the physical properties of the fluid cannot be well controlled and represented. The system is based on Obi Fluid plug-in to simulate blood, which involves the position dynamics and the fluid simulation process based on the position dynamics is the key of this system [42].

The core of the Position Based Dynamic (PBD) algorithm is the definition of various types of constraints and the calculation of constraint projections. The constraint functions of various constraints usually generate nonlinear equations, and PBD uses the Gauss-Seidel (GS) method to iteratively solve the constraint by restricting the amount of particle position change $\Delta p$ to the direction of the constraint gradient ${\nabla }_{p}C(p)$, so that it maintains momentum conservation while satisfying the constraint conditions. The PBD algorithm flow is shown in Table 1. The core of PBD algorithm is how to define and solve constraints. In order to realize fluid simulation, it is necessary to establish appropriate constraints on fluid particles, so that they can show the effect of liquid flow in the real world. According to Smoothed Particle Hydrodynamics (SPH), the incompressibility of a fluid can be achieved by introducing a density constraint on each fluid particle, whose constraint function is as follows: (1) $$C_i\left(p_1,\mathrm{ }{\dots ,p}_n\right)=\frac{{\rho }_i}{{\rho }_0}-1$$(1)

Table 1. PBD algorithm.

where ${\rho }_0$ is the rest density and ${\rho }_i$ is given by the standard SPH density estimator： (2) $${\rho }_i={\sum }_j{m}_{j}W(p_i-p_j,h)$$(2) where, $m_j$ is the mass of the domain particle $j$ of the current fluid particle $i,$ and $h$ is the support radius of the Kernel smoothing function. Equation (2)(2) $${\rho }_i={\sum }_j{m}_{j}W(p_i-p_j,h)$$(2) based on the Kernel Density Estimation (KDE) theory, the approximate estimation of the current particle density is obtained by using the density of surrounding particles through Kernel smoothing function interpolation.

According to Equations (1)(1) $$C_i\left(p_1,\mathrm{ }{\dots ,p}_n\right)=\frac{{\rho }_i}{{\rho }_0}-1$$(1) and (2)(2) $${\rho }_i={\sum }_j{m}_{j}W(p_i-p_j,h)$$(2) , all fluid particles are considered to have the same mass, so $m_i$ is omitted in the derivation process. The position correction of each fluid particle after density constrained projection can be obtained as follows: (3) $$\Delta p_i=\frac{1}{{\rho }_0}\sum _j({\lambda }_i+{\lambda }_j){\nabla }_{p_i}W\left(p_i-p_j,h\right)$$(3) (4) $${\lambda }_i=-\frac{C_i(p_1,\mathrm{ }\dots ,\mathrm{ }{p}_n)}{{\sum }_k{\left|{\nabla }_{p_k}{C}_i\right|}^2}$$(4)

The denominator in Equation (4)(4) $${\lambda }_i=-\frac{C_i(p_1,\mathrm{ }\dots ,\mathrm{ }{p}_n)}{{\sum }_k{\left|{\nabla }_{p_k}{C}_i\right|}^2}$$(4) is affected by Kernel smoothing function. In this paper, Poly 6 Kernel function is used for density estimation and Spiky Kernel function is used for gradient calculation. When the distance between the particles is near the support radius of the smooth Kernel, namely $\left|\mathit{r}\right|\approx h,$ the denominator in Equation (4)(4) $${\lambda }_i=-\frac{C_i(p_1,\mathrm{ }\dots ,\mathrm{ }{p}_n)}{{\sum }_k{\left|{\nabla }_{p_k}{C}_i\right|}^2}$$(4) approach to zero, lead to system instability, therefore, the relaxation factor $\epsilon$ should be introduced into Equation (4)(4) $${\lambda }_i=-\frac{C_i(p_1,\mathrm{ }\dots ,\mathrm{ }{p}_n)}{{\sum }_k{\left|{\nabla }_{p_k}{C}_i\right|}^2}$$(4) : (5) $${\lambda }_i=-\frac{C_i(p_1,\mathrm{ }\dots ,\mathrm{ }{p}_n)}{{\sum }_k{\left|{\nabla }_{p_k}{C}_i\right|}^2+\epsilon }$$(5)

The problem of tensile instability occurs in simulation due to the use of SPH interpolation to calculate fluid density. When the number of domain particles is insufficient, the particle density calculated in Equation (2)(2) $${\rho }_i={\sum }_j{m}_{j}W(p_i-p_j,h)$$(2) will be lower than the static density, resulting in negative pressure, which makes the pressure between particles become attractive, resulting in particle condensation. In order to prevent the unnatural attraction between particles, it is necessary to introduce the exclusion term $s_{\mathit{corr}}$ into Equation (3)(3) $$\Delta p_i=\frac{1}{{\rho }_0}\sum _j({\lambda }_i+{\lambda }_j){\nabla }_{p_i}W\left(p_i-p_j,h\right)$$(3) : (6) $$\Delta p_i=\frac{1}{{\rho }_0}\sum _j({\lambda }_i+{\lambda }_j+s_{\mathit{cor}\mathrm{r}}){\nabla }_{p_i}W\left(p_i-p_j,h\right)$$(6) (7) $$s_{\mathit{corr}}=-k{\left(\frac{W\left(p_i-p_j,h\right)}{W\left(\Delta q,h\right)}\right)}^n$$(7) where $\Delta q$ represents a fixed distance from particle $i,$ $k$ can be regarded as the surface tension coefficient. Usually take $\Delta q=0.1\sim 0.3h,$ $k=0.1,$ $n=4.$ After introducing the repulsive fluid particles, the particles are more evenly distributed on the fluid surface, which effectively alleviates the problem of particle condensation caused by insufficient field particles.

PBD based simulation methods usually introduce additional damping to the system, resulting in energy dissipation. The vortex that should be there disappears quickly; therefore, energy needs to be added into the system by vorticity confinement. The basic idea of vorticity confinement is to maintain the curl of particles in the system by adding volume force. The calculation equation of volume force can be expressed as follows: (8) $$f_i^{\mathit{vorticity}}=\epsilon (N\times {\omega }_i)$$(8) (9) $${\omega }_i=\nabla \times \mathit{v}=\sum _{\mathit{j}}(v_j-v_i)\times {\nabla }_{p_j}W\left(p_i-p_j,h\right)$$(9) where, ${\omega }_i$ is the curl of the particle, $N=\frac{\eta }{\left|\eta \right|},$ $\eta =\nabla {\left|\omega \right|}_i,$ and $\epsilon$ is the vorticity confinement intensity.

In addition, artificial viscosity needs to be introduced to improve the numerical stability of simulation and ensure the stable convergence of numerical calculation, which is realized by adding damping to the relative motion of particles. The specific velocity correction formula is as follows: (10) $$v_i^{\mathit{new}}=v_i+c\sum _j(v_j-v_i)W\left(p_i-p_j,h\right)$$(10) where, $c$ is the speed correction factor, usually $c=0.01.$

In conclusion, the fluid simulation algorithm based on PBD is shown in Table 2.

Table 2. Fluid simulation algorithm based on PBD.

Based on the above method, using the obi fluid package and setting the density, viscosity, surface tension, color and other properties of fluid particles, and the final simulated blood effect is as shown in Figure 4. Figure 4(A) shows the effect of rendering fluid particles using billboard method; Figure 4(B) shows a rendering of a fluid surface reconstructed using screen-space splatting; Figure 4(C) shows the blood effect in the scene.

Figure 4. Blood simulation effect based on obi fluid plug-in: (A) particle rendering; (B) fluid rendering; (C) blood effect in scene.

Virtual interaction

The hardware basis for VR scene interaction is the HTC VIVE Pro 2 gamepad, which provides a number of buttons for development, including the menu button, Touchpad, Trigger button and Grip button. By adding the Virtual Reality Tool kit (VRTK) Controller Events component to the controller object corresponding to the handle in Unity, other scripts can obtain the component and register the events corresponding to the change of key state and trigger the events bound to it by listening to key state changes during program operation to realize the related business logic.

Assessment and evaluation module

The simulation process of VR-CMTT system is divided into two phases, battlefield environment assessment and CMF trauma rescue. In the battlefield environment assessment phase, trainees find the casualty in the virtual battlefield environment, follows the operational prompts to transfer the casualty to a safe environment and selects the treatment process through the teaching video. Conduct battlefield situation assessment by observing battlefield and determining fire coverage area. If it is found that there is no enemy fire coverage temporarily, the operator confirms that the battlefield treatment environment is safe. If a casualty is found to be exposed to enemy fire, move the casualty to a safe environment, namely the tent area in the system, according to the prompts of the rescue process and then select a different rescue scenario by switching rescue scenarios. The key steps such as ventilation, hemostasis, cerebrospinal fluid leakage diagnosis and CMF fracture fixation simulation are realized in the CMF trauma rescue treatment stage, as shown in Figure 5.

Figure 5. Assessment process of CMF trauma treatment: (A) find the wounded; (B) move the casualty to a safe environment; (C) remove foreign bodies in the respiratory tract; (D) insert ventilation; (E) hemostasis; (F) extract cerebrospinal fluid; (G) judged there is cerebrospinal fluid leakage or not; (H) reduction of fracture; (I) evacuation.

The main process of the medical activities is shown in Figure 6. After switching to the rescue scenario, the first step is to remove foreign bodies in the respiratory tract. Remove foreign bodies with fingers or suction device according to the instructions. The second step is tracheal intubation. The patient is placed in a supine position, and the forehead is pressed and the chin is lifted so that the mouth, pharynx and trachea are basically in the same axis. The operator is positioned on the side of the head, with the patient’s lips open in the right hand and the laryngoscope in the left hand, and the vocal hilum is visible by lifting up the laryngoscope. After exposing the glottis, the catheter is gently inserted into the trachea, the dental pad is placed, and the laryngoscope is withdrawn.

Figure 6. The main process of the pre-hospital treatment of CMF trauma.

The third step is hemostasis. Quickly find the bleeding point and select methods such as acupressure hemostasis, bandaging hemostasis, tamponade hemostasis and vascular clamp hemostasis according to the bleeding site and amount to stop bleeding. Acupressure hemostasis can be used to stop bleeding in the following cases. In the case of facial bleeding, compression of the facial artery, located at the anterior edge of the stop of the occlusal muscle on the mandibular surface. When bleeding from the top of the head, the superficial temporal artery is compressed, located at the base of the zygomatic arch in front of the ipsilateral tragus. In the case of posterior head bleeding, compression of the posterior auricular artery, located slightly posterior to the posterior mastoid process. When the neck is bleeding, compress the common carotid artery on one side, located between the ipsilateral lateral trachea and the midpoint of the anterior border of the sternocleidomastoid muscle and press it firmly to the transverse process of the sixth cervical vertebra. For small bleeding or traumatic bleeding, use a bandage to stop the bleeding. After cleaning the wound, the soft tissue was reduced. The injury site is covered or filled with gelatin sponge, then covered with multiple layers of sterile dressing and finally bandaged with pressure. For open or cavernous wounds, use the tamponade method to stop bleeding. The wound is filled with sterile iodoform gauze or oiled gauze and then bandaged with pressure. When an artery with persistent bleeding within the wound is encountered, it is clamped closed using a vascular clamp and sent along with a vascular clamp with a bandage. When the bleeding artery in the wound is encountered, vascular clamp is used. The vascular clamp is bandaged with the wound.

The fourth step is rapid detection of cerebrospinal fluid. Use a syringe to aspirate bloody secretions at the bleeding site of the cranial and maxillofacial wound and use a cerebrospinal fluid leakage diagnostic test paper to determine whether there is cerebrospinal fluid leakage. According to the concentric circle diameter of the bloody secretions and blood on the cerebrospinal fluid test paper, the patient is judged whether there is cerebrospinal fluid leakage. If cerebrospinal fluid leakage is present, compression bandages are applied for treatment.

The fifth step is fracture fixation. When a maxillary fracture is present, a tongue depressor or chopstick is placed across the maxilla in the position of the bilateral premolar teeth, holding the falling maxillary block upward and securing the ends to a bandage on the head. When a mandibular fracture is present, an elastic jaw cap or a three-dimensional elastic mask is chosen for fixation after manipulation and repositioning. When an orbital fracture is present, determine whether it is a simple closed orbital fracture or an open fracture. A simple closed orbital fracture requires no special management. In the case of open fractures, the option of removing free bone fragments and the option of resetting bone fragments with periosteum is chosen. When an orbital fracture is present, determine whether it is a simple closed orbital fracture or an open fracture. A simple closed orbital rim fracture requires no special management. If the fracture is open, remove the free bone fragments and choose to reset the bone fragments with periosteum. In the presence of nasal bone fracture, nasal bone fracture reduction is selected. In the case of zygomatic fracture, if there is only mild displacement after fracture, no obvious deformity, no restriction of mouth opening, diplopia and other functional disorders, conservative treatment should be chosen. If there is obvious facial collapse, diplopia, restricted mouth opening, optic nerve compression and other symptoms, emergency reduction and fixation should be required. The last step is evacuation. Select the ambulance to evacuate the casualty. After the completion of the above steps, the system will automatically assess and score the training effect of trainees.

Ethics and inclusion criteria

The study protocol was approved by the ethics committee review board of the General Hospital of the Southern Theater Command of the Chinese People’s Liberation Army, and written informed consent was obtained from recipients in accordance with all regulations and confirmation that informed consent was obtained. Three experienced surgeons and 22 novice surgeons from the General Hospital of the Southern Theater Command of the Chinese People’s Liberation Army volunteered to participate in the validation process. Following instructions on safety measures before use of the VR-CMTT system, all participants were asked if they suffered from any psychiatric disorders (including attention deficit hyperactivity disorder or epilepsy) or if they were on any antipsychotic drugs. Any previous history of motion sickness or seizures was considered an exclusion criterion.

Evaluation of the developed VR-CMTT system

Two separate questionnaires were used to check the validity of the VR-CMTT system: a pre-intervention questionnaire to understand the training needs and a post-intervention feedback questionnaire to comment on the efficacy, usability and acceptability of the system. The questions were developed with expert surgeons in CMF surgery. Questions specific to CMF trauma, including technical challenges and the common human errors were asked. Specific questions regarding the types of educational methods currently used to deliver training were included. The user’s expectations regarding how the new technology could influence their satisfaction levels in improving on-surgical skills were considered [43]. The questionnaire also explored the consultants’ previous experience of using head-mounted surgical displays to determine whether they were familiar with the technology. Questions regarding awareness and certification for the non-technical skills for surgeons were asked.

Following a structured session of demonstrating the innovative technology and allowing the participants to experience its facilities, a post-intervention questionnaire on the content, usability and application of the developed tool in training was conducted. A five-point Likert rating scale was used to rate the quality of the videos and the 3D models of the instruments and anatomy. The following scoring elements were used: 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree (neutral), 4 = agree, 5 = strongly agree. Space for additional open comments was provided, and the participants were encouraged to make use of it. Additional suggestions regarding future developments needed in the application were taken from the surgeons. The participants commented on the quality of the videos and 3D models of the instruments and anatomy. The experts rated the comfort of using the headset and the accuracy and appropriateness of hand tracking based on the system usability scale of Bangor et al. [22,44]. The last section of the questionnaire focused on the potential applications of the VR-CMTT system in training CMF trauma rescue trainees. Their opinions on the use of the VR-CMTT system for training, benefits of its use for multiple procedures, and acceptability for inclusion in the curriculum were questioned. In line with current studies, participants were asked if they considered the VR-CMTT system an effective adjunct to current training methods. The effectiveness of the VR-CMTT system with regard to self-confidence and knowledge of trainees was also investigated. A question regarding the inclusion of non-technical skills was added to the feedback. IBM SPSS Statistics for Windows, version 22.0 (IBM Corp., Armonk, NY, USA) was used for the data analysis.

Result

All of the surgeons completed the questionnaires. The mean age of the experienced surgeons was 42.7 years. All of the experienced surgeons were male and they had a mean surgical experience of 15.5 years. The mean age of the novice surgeons was 26.7 years. All of the novice surgeons were also male and they had a mean surgical experience of 3.5 years. None of the participants had previous experience in using a head-mounted display for training. The participants suggested that learning in the battlefield training ground is the best form of training, and 20 of them mentioned that educational videos are currently used as an adjunctive method of training. All of the novice surgeons reported rapid detection of cerebrospinal fluid as the most difficult step while training novices in rescue of CMF injuries in their questionnaire answers. With regard to the validity of the content of the surgical video clips within the application, the mean score was 4.29, showing strong agreement. The responses to the individual questions on the quality of the content are shown in Figure 7. Overall, the mean scores for the content of the application showed agreement with the validity of the developed innovation.

Figure 7. Responses to the questions regarding the content of the VR-CMTT system: (A) the CMF trauma treatment process is shown accurately in this application; (B) all the steps of the CMF trauma treatment are covered in this application; (C) the order of steps in CMF trauma treatment are not shown correctly; (D) I could clearly see the benefit of videos in training; (E) I could see how 3D videos are better than conventional 2D videos; (F) I found the virtual environment of battlefield scene useful; (G) interacting with the pre-surgical data was beneficial; (H) the menu scene showing all steps is useful for trainees to remember the sequence.

The mean score for the appearance, use and realism of the developed training tool was 4.29 out of 5. The questions regarding the anatomy in the application scored 4.68, showing strong agreement with the face and content validity. The responses to the individual questions on the quality of the anatomy are shown in Figure 8. When asked about the applicability of the VR-CMTT system to the current curriculum, the surgeons rated the application with a mean value of 4.29. Figure 9 shows the responses to the individual questions on the applicability of the developed VR-CMTT system for training. The mean score for the various questions regarding the ease of use and hand tracking was 4.14. The responses to the individual questions on the usability of the VR-CMTT system are shown in Figure 10.

Figure 8. Responses to the questions regarding the quality of the anatomy in the application: (A) the anatomy of the skull shown in the VR-CMTT system application is accurate; (B) the blood vessels shown in the VR-CMTT system are accurate; (C) the 3D models resemble the real life anatomy realistically in their appearance; (D) it is necessary to have this aspect of anatomy interaction in the application.

Figure 9. Responses to the questions regarding the applicability of the VR-CMTT system to the curriculum: (A) I found this application interesting; (B) I think this application will be useful for teaching trainees regarding the rescue of CMF injuries; (C) addition of virtual reality application like the VR-CMTT system to the training will be beneficial for surgical trainees; (D) I think this application will enhance the understanding of surgical trainees regarding the rescue of CMF injuries; (E) I think this application will increase the confidence of surgical trainees before they perform a real surgery on the battlefield; (F) I see this application more like an adjunct than a necessary tool for studies; (G) I want to see more first aid surgical procedures developed into the VR-CMTT system.

Figure 10. Responses to the questions regarding the usability of the VR-CMTT system: (A) I found the headset comfortable to wear throughout the application; (B) I found the quality of audio to be excellent; (C) I found the quality of the videos to be excellent; (D) I could interact with the user interface as I expected; (E) I could touch the virtual objects appropriately; (F) I could not track my hands in the application accurately; (G) I could look around the battlefield comfortably without any discomfort; (H) I did not experience nausea, dizziness or headache using the VR-CMTT system; (I) It was comfortable to use the VR-CMTT system.

The data processing capability of this system and the speed of model dynamic loading and scene real-time rendering are good. In accordance with the requirements of the operating environment hardware configuration and full load, the system software takes 5 s to start and initialize. The scene loading and initial scene rendering takes 10 s, while the single model dynamic loading and real-time rendering takes 2 ∼ 3 s on average. The response delay time of the interaction between the handle button and ray detection is controlled in 200 ∼ 250 ms, the operation feedback delay time is controlled in 250 ∼ 500 ms, and the processing time of collision detection is 1 ∼ 3 s. It takes 2 s to update the treatment process prompt, 1 s to update the model status and 5 s to generate the training assessment and evaluation report. Since the entire system is developed based on the HTC VIVE Pro 2 head-mounted display device, only one person is supported during a training session. During the testing, experienced surgeons could easily position the instrument on the designated trajectory and smoothly perform pervasive life-saving operations, including hemostasis, bandaging, ventilation, immobilization, lifting and basic life support, with little warning from the VR-CMTT system. However, it is much more difficult for novices to do the same work in the first place. Operating virtual devices often deviates from certain trajectories and is not familiar with the order of rescue, resulting in false warnings from the system. Nevertheless, after several rounds of training, the novices became experienced and skilled, because young people generally do better in VR simulations.

Discussion

The prevalence of CMF trauma is significant in both the civilian and military setting. The increasing rate of CMF trauma, particularly on the battle field cannot be ignored. The head, face and neck are only 12% of the bodies surface area exposed during combat; however, this area is not well protected partially because any obstruction in this zone can limit senses that are vital in hostile environments. Combine this with the increase use of improvised explosive devices by enemy combatants and it’s no surprise that the CMF region is experiencing disproportionally more injuries leading to death [45]. CMF trauma may involve almost all organ systems and impair many critical functions such as ventilation, cognition and vision. As the CMF region contains major vascular structures and is generally composed of highly vascular tissue, injuries can lead to uncontrolled bleeding, hypotension, shock and death [45].

In this study, immersive visualization simulation of CMF trauma was realized based on virtual reality technology and the head-mounted VR hardware platform was integrated to finally build a simulation training system for the pre-hospital treatment of CMF trauma. Every patient with a CMF trauma should be assessed according to the advanced trauma life support protocol. In the realistic virtual battlefield environment, the process of diagnosis and the pre-hospital treatment of CMF trauma are simulated visually. This system realized the common clinical manifestations of CMF injuries, such as uncontrolled bleeding, hypotension, shock, fracture, injury, cerebrospinal fluid leakage, etc. Injuries to this area are potentially devastating. Respiratory, vascular and neurological structures may all be involved, either in isolation or combination. These injuries are particularly difficult to manage in the resource poor, often austere and/or remote, pre-hospital environment. However, even in gross disruption of the airway, most facial injuries can be orally intubated. The vast majority of facial hemorrhage can be managed with direct pressure and packing of wound cavities, including hemostat dressings. Based on PBD method, using the obi fluid package and setting the density, viscosity, surface tension, color and other properties of fluid particles, bleeding and hemostasis process was simulated vividly in the CMTT system. Airway management in CMF trauma is potentially challenging. The airway may be complicated by the injury itself, the presence of blood or secretions or may be secondarily distorted by soft tissue swelling or hematoma formation. The bleeding upper airway poses many challenges including preoxygenation, denitrogenation being less efficient and hypovolemia. As such, decisions on airway control should be taken on a patient-by-patient basis. Indications for intubation include airway compromise/injury, apnea or respiratory failure, hypoxia and a reduced level of consciousness. Early recognition of any of these indications is crucial in the pre-hospital setting, especially when these two factors are in combination. Based on virtual reality technology, intubation process was simulated in the CMTT system. Severe CMF trauma is often connected with fractures of the frontal skull base or nasoethmoido-orbital complex and cerebrospinal fluid leakage. A spinal fluid leak from the intracranial space to the nasal respiratory tract is potentially very serious because of the risk of an ascending infection which could produce fulminant meningitis. The CMTT system successfully simulates the detection of cerebrospinal fluid leakage.

The results showed agreement among the novices and experienced surgeons regarding the face and content validity of the VR-CMTT system. The surgeons found the VR-CMTT system easy to use, following a short learning curve of about an hour before getting used to the sensitivity of head tracking and interaction. To further shorten the learning curve, a tutorial was introduced, which gives a hands-on demonstration before use of the application. The surgeons felt that the advantage of the VR-CMTT system lies in its interactivity. They suggested that the addition of haptic force feedback and realistic interaction with 3D models of instruments would enhance the experience. The ability to rescue and take part in it virtually was also recommended. The interactive 3D anatomy and instruments were the most appreciated features in the application. The addition of multiple levels of complexity for basic, intermediate and advanced levels of training was also suggested. A key strength of this study is the combining of VR and motion detection technology and pre-hospital CMF trauma rescue treatment knowledge to create an evidence-based immersive rescue training experience. The use of natural hand gestures in combination with VR experience to learn a complex rescue is the core functionality. The validation studies of this research add more value to the work. According to the content, instrumentation, anatomy, usability and applicability of the VR-CMTT system, the surgeons ranked it as a valid training tool. The VR-CMTT system can be used for remote or distant learning. This may increase the impact of projects results as the training simulator can reach more trainers and trainees. This has become more critical since the COVID-19 break-out causing many lockdowns and limiting face-to-face training.

The main limitation of this study is that the technology developed was evaluated by only 25 surgeons, only three of them were experienced surgeons. It is the authors’ intention to recruit trainees in order to assess this innovation further. Technological limitations of this research include the lack of haptic force feedback. The availability of suitable technology and time constraints in developing a realistic haptic force feedback prevented this from being implemented. However, future research on VR training aims to include haptic feedback in the application. This will be considered the next phase of the current research program. The need for expensive headsets and high-specification computers makes desktop VR applications unaffordable for individual trainees. The development of a low-cost version of VR-CMTT system for devices will be a key to addressing this issue [46]. In this study, the validation of the VR-CMTT system was limited to face and content validity tests. As commercially available VR and augmented reality experiences are increasingly used for training, a framework to build effective VR solutions is needed. For global application of these emerging technologies, they should be made more affordable so that they can be extended to low- and middle-income countries with maximum need. Once the challenges are met, applications like the VR-CMTT system will provide an alternative way of learning and could reduce the time taken to train surgeons. Moreover, the ability to experience training remotely will change the way surgeons learn in many ways.

In conclusion, the VR-CMTT system developed by using VR technology realizes the visual simulation of CMF trauma injury judgment analysis and treatment training process in a realistic virtual battlefield environment, which deepens the intuitive understanding and awareness of trainees. The results confirmed the applicability of VR for delivering training in the pre-hospital treatment of CMF trauma. The VR-CMTT system has a satisfactory level of validity and so can be used among surgical trainees to effectively improve the specialized treatment ability of medical staff and junior specialists.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the PLA Logistics scientific research project under Grant No. CLB19R032.