Operators believe patient-specific rehearsal improve individual and team performance

Abstract

Introduction

Rehearsing endovascular aortic aneurysm repair on patient-specific data is recent within virtual reality simulation and opens up new possibilities for operators to prepare for complex procedures. This study evaluated the feasibility of patient-specific rehearsal (PsR) and assessed operators’ appraisal of the VIST-LAB simulator from Mentice.

Material and methods

CT-data was segmented and uploaded to the simulator, and simulated for 30 elective EVAR patients. Operators were asked how they perceived the PsR on a Likert scale after the PsR (once) and after the following procedure (each time).

Results

Patients were simulated and operated by 14 operators, always in pairs of one vascular surgeon and one interventional radiologist. The operators estimated that PsR improved individual and team performance (median 4), and recommended the use of PsR in general (median 4) and for difficult cases (median 5). The simulator realism got moderate scores (median 2–3). Inexperienced operators seemed to appreciate the PsR the most.

Conclusions

PsR was feasible and was evaluated by operators to improve individual and team performance. Inexperienced users were more positive towards PsR than experienced users. PsR realism and the ease of importing patient-specific data can still be improved, and further studies to quantify and precisely identify benefits are needed.

Keywords:

• Patient-specific rehearsal
• virtual reality simulation
• endovascular procedures
• endovascular aortic aneurysm repair
• EVAR
• patient safety

Introduction

VR simulators are seen as useful tools to train basic and procedural endovascular skills [1–5], and to objectively assess competencies in competency-based training [6,7]. A recent development in simulation technology is patient-specific rehearsal (PsR) [8], which opens up possibilities of training on patient-specific cases prior to the actual procedure, thus potentially addressing both experienced and inexperienced operators. Endovascular aortic aneurysm repair (EVAR) is an established procedure for the repair of abdominal aortic aneurysm [9,10] that requires a high level of technical expertise [11–13] and is therefore well suited for PsR. The PsR gives the operators the possibility to rehearse on the simulator with patient-specific data before the real procedure. PsR has been found to influence C-arm angulation and device selection, and is believed to have a positive impact on patient outcome and operation efficiency [8,14–17]. However, there might be factors that can impede the use of such new technology, such as time constraints [18], organisational factors, lack of realism and/or that the believed positive impact cannot be demonstrated.

The introduction of VR simulators has raised questions whether surgical skills could be acquired outside of the operating room without putting patients at risk [6,19]. Introducing simulation-based PsR raises questions regarding the level of preparation that can be expected of the operators, before performing a procedure. Today, operators use 3D SW to prepare for the procedure looking at the patients’ CT images, measuring and ordering stent graft components and estimating optimal C-arm angulations [20]. And they might do what can be called a mental rehearsal [21], during which the operators, either separately or together, mentally go through the CT images, visualize potential difficulties, and plan how to tackle them during the actual procedure. A (mental) rehearsal is therefore not new with regard to EVAR procedures, but PsR adds a more concrete experience that adds haptic sensations and practical aspects such as stent-graft deployment. Compared to a mental rehearsal, the PsR lets the operators train on defined tasks of the procedure, e.g., the contralateral gate cannulation, it gives feedback as the operators can see and verify the landing zone of the stent graft neck or check for endoleaks, and it allows for repetitions.

Simulators have been criticized for being perceived as low-stakes compared to the operating room which is being perceived as a high-stake, and it is considered that a high-stake experience is better retained than a low-stake experience [22]. Training on patient-specific data as in PsR might increase the sentiment of relevance, and make it a higher-stake than traditional training on simulators, potentially increasing the efficiency of time spent on training. Experienced surgeons in open surgery who want to convert to endovascular procedures might find PsR more appealing than regular training on simulators.

The aim of this study was to describe the feasibility of PsR on EVAR procedures using the VIST-LAB simulator (Mentice AB, the Bolton EVAR Case-it module, Gothenburg, Sweden) and to investigate operators’ appraisal. The realism of the simulator was evaluated through visual checks of selected final angiograms from the PsR versus the real procedure, and through a questionnaire investigating operators’ appraisal. A few studies have investigated patient-specific simulation on the Angio Mentor^TM (3D Systems Healthcare, Littleton, CO, USA) [14–18,23–31]. To our knowledge, this is the first study investigating patient-specific rehearsal (PsR) on the VIST-LAB simulator.

Material and methods

Feasibility of preparing CT images for the simulator and the PsR was investigated for 30 elective EVAR procedures from September 2016 to November 2017. All ethical aspects of the study were approved by the Norwegian data inspectorate.

Operators’ appraisal

Operators’ appraisal was investigated by asking the vascular surgeons and interventional radiologists (IR) who did the PsR and the following EVAR to fill out two questionnaires. The post-PsR questionnaire was filled out once after their first PsR, i.e., one for each operator that participated in the study. The post-EVAR questionnaire was filled out after each EVAR, i.e., two for each procedure since there were two operators.

The simulator

The simulator used was the VIST-LAB with VIST-C^TM and Bolton Treo deployment system using the Bolton Case-it EVAR module (version 8.3) (all Mentice AB, Gothenburg, Sweden) (Figure 1).

Figure 1. The patient-specific rehearsal set-up with the VIST-LAB, VIST-C^TM and Bolton Treo deployment system.

Medical imaging viewer, STL viewer

TeraRecon 3D SW (Aquarius Intuition, Version 4.4.12, Foster City, US) was used to prepare the CT images for export to STereoLithography (STL) files [32] of a segmented abdominal aorta. STL files describe surface geometry of three-dimensional objects using triangulated surfaces, and are widely used within 3D printing and computer-aided manufacturing.

Preparations of patient-specific data

From CT angiography of the abdomen and pelvis, the abdominal aorta was segmented from above the renal arteries and below the iliac bifurcation (Figures 2(B) and 3(B)). The segmentation was performed by automatically removing bones, then manually removing surrounding tissue and including areas with uneven contrast spread using the ‘FreeROI’ tool and the ‘Dynamic region growing tool’. Finally, the segmented aorta was smoothed using the ‘smooth surface’ functionality.

Figure 2. Patient with aneurysmal dilatation of the abdominal aorta, male, age 65. (A) Computed tomography angiography shows an aneurysm with a maximal diameter of 65 mm. (B) Segmented surface-rendered 3D reconstruction. (C) Detail from the STL model of the aortic bifurcation and the right common iliac artery. (D) The imported STL model (center) in the simulator stitched to the simulator template (light gray). (E) Angiogram of the stentgraft components on the simulator. (F) Angiogram of the stentgraft components in the patient. Visually E and F show good correlation. (G) CT-angiogram at six months follow-up.

Figure 3. Patient with aneurysmal dilatation of the abdominal aorta, male, age 86 (A) Computed tomography angiography shows an aneurysm with a maximal diameter of 71 mm. (B) Segmented surface-rendered 3D reconstruction. (C) Detail from the STL model of the ostium of the artery to the left kidney. (D) The imported STL model (center) in the simulator stitched to the simulator template (light gray). (E) Angiogram of the stent-graft components on the simulator. (F) Angiogram of the stent-graft components in the patient. Visually E and F show poor correlation. (G) CT-angiogram at six months follow-up.

The STL file was then imported into the simulator, where the renal arteries and the external and internal iliac bifurcations of the patient-specific model were stitched to the simulator template (Figures 2(D) and 3(D)). In addition, patient height was added and adjustments were made so that the exit of the patient’s renal arteries correlated with the corresponding vertebra of the simulator template.

The simulation

The patient-specific rehearsal was performed by one vascular surgeon and one IR, accompanied by one radiographer (the same that created the STL files). The simulated EVAR procedure started after a surgical puncture, was followed by the deployment of the Bolton stent graft system (main body, contra- and ipsilateral legs) and ended with the completion angiogram. Haptic feedback was simulated using three sets of actuators placed at three different depths for three ranges of instrument diameters, generating variable forces on the instruments. The lengths of the stent graft main bodies and the leg extensions were based on the planned lengths for the real procedure and adapted to the lengths available from Bolton Treo.

The real EVAR was performed by the same vascular surgeon and the IR, the same day or the day after, with the planned stent graft systems being Bolton Treo, Cook Zenith and Medtronic Endurant.

Simulation realism was evaluated through visual checks of selected angiograms of the simulation outcome versus the procedure outcome.

Statistical analysis

The Mann–Whitney U-test was used to test for statistical differences in the distribution’s central tendencies (p < .05) between the answers from experienced and inexperienced operators (SPSS 25, IBM Corporation, Armonk, NY, USA).

Results

Seven vascular surgeons and seven IRs performed the PsRs, performing between one and nine cases each. Four vascular surgeons were regarded as inexperienced (less than two years of experience with EVAR procedures), performing EVAR procedures under supervision, three vascular surgeons and all of the IR were experienced being regarded as capable of performing the procedure without any supervision (more than two years of experience with EVAR procedures). None of the operators had previous experience with PsR. Two of the IRs had experience with the simulator. After an initial learning process where a dedicated radiographer learned how to prepare the CT images and upload them into the simulator, she would spend between 30 and 180 min to prepare the STL files that were needed for the simulation. The contrast saturation of the CT images influenced the time and difficulty to create good enough STL files. The simulation itself took between 15 minutes and 1 hour. The duration of the simulations were influenced by several aspects related to the simulator itself (technical errors), minor and major procedural errors or simply that inexperienced operators, in general, spent more time as they seemed to enjoy the opportunity to learn from a more experienced operator. Procedural errors that occurred were occlusion of the renal arteries and/or the iliac internal artery (the PsR was repeated), and a type 1A leakage at the neck of the main body (one occurrence, which was fixed with a stent graft balloon). In comparison, two of the 30 EVAR procedures on the real patients ended with a type 1A leakage, one of which was treated during the procedure. None of the operators of these two procedures said that the PsR indicated that endoleakages could be expected in the real procedure. On one occasion, as a result of the PsR, a marking catheter was used to verify the length of the ordered stent graft during the real procedure. On another occasion, the PsR revealed that the ordered stent grafts were too long. New measurements were done and new stent grafts were ordered.

Selected angiograms from the PsR and the real EVAR procedure were collected and compared by two experienced interventional radiologists. Figures 2 and 3 show images for two patient cases presenting the aneurysm (Figures 2(A) and 3(A)), the PsR (Figures 2(B–E) and 3(B–E)), angiograms from the procedure (Figure 2(F) and 3(F)) and six months follow-up (Figures 2(G) and 3(G)). Figure 4 shows angiograms of four patients after the PsR and after the real procedure. A visual comparison of the angiograms from the PsR with the angiograms from the real procedure showed good to poor correlation (Figures 2(E,F), 3(E,F) and 4).

Figure 4. Comparing angiograms after stent graft deployment of four patients (I-IV) from the simulator (PsR) and from the real procedure (RP). Patient I, age 59, poor correlation; Patient II, age 82, good correlation; Patient III, age 63, poor correlation; Patient IV, age 80, good correlation. Stitching artefacts can be seen at the aortic neck of patient III and IV on the simulated pictures.

Operators’ appraisal

Fourteen operators participated and filled out the operators’ appraisal questionnaire after their first PsR, i.e., a total of 14 questionnaires were filled out and analysed. The questionnaire contained 13 statements that were rated on a Likert scale from 1 to 5, presented in Figure 5. A total of 59 questionnaires of operator appraisal after the procedure were filled out and analysed, two for each procedure, i.e., one questionnaire was not filled out. The questionnaire contained seven statements that were rated on a Likert scale from 1 to 5, presented in Figure 6. For most statements, experienced (10 post PsR questionnaires, 44 post EVAR questionnaires) and inexperienced operators (4 post PsR questionnaires, 15 post EVAR questionnaires) gave similar responses. For the following statements inexperienced operators gave significantly higher scores, after the PsR: ‘PsR will reduce procedure time’ (median 5 versus 4, p-value .01), ‘PsR will improve individual performance’ (median 5 versus 3, p-value 0.003), and ‘PsR will improve team performance’ (median 5 versus 4, p-value .001), and after the procedure: ‘The PsR influenced C-arm angulation through the whole procedure’ (median 4 versus 3, p-value .006) and ‘I recommend the use of PsR in general’ (median 4 versus 4, p-value .005).

Figure 5. Boxplot presenting operator appraisal after the PsR on a Likert scale from 1 to 5, where 1 is disagree and 5 is strongly agree. N = 19. The middle band shows the median value, the bottom and the top of the boxes show the 25th and the 75th percentiles, and the ends of the whiskers show the 5th and the 95th percentiles. Outliers are plotted as circles and extreme outliers as stars.

Figure 6. Boxplot presenting operator appraisal of seven statements after the procedure on a Likert scale from one to five, where one is disagree and five is strongly agree. N = 59. The middle band shows the median value, the bottom and the top of the boxes show the 25th and the 75th percentiles, and the ends of the whiskers show the 5th and the 95th percentiles. Outliers are plotted as circles.

Discussion

Patient-specific rehearsal on VR simulators opens up new possibilities for the operators to prepare themselves before treating a patient. It gives a more concrete experience than image-supported mental rehearsal and opens up a discussion on expectations towards operator preparedness. Operators’ appraisals after the procedure show that it is recommendable (median 4 and 5) and is estimated to have a positive impact on team and individual performance (median 4).

With the introduction of any new technology the users need to learn how to use it, and for it to be successful, a place within existing clinical routines need to be found. The first step in the PsR process was to prepare the CT-images for the simulator. This was not a straightforward process, and we chose to use a dedicated radiographer. After an initial learning phase, the radiographer would normally spend between 30 and 180 min to prepare the images depending on the CT image quality (contrast saturation). This is in accordance with other studies on PsR [27,30]. We used TeraRecon to create the STL files, but other SW packages or a combination of SW packages, that can segment CT images and/or create STL files can be used. A few examples of other software tools that can segment CT-images and export to STL files are Osirix MD (Bernex, Switzerland, payware), 3DSlicer (http://www.slicer.org/, open source), Seg3D (www.sci.utah.edu/cibc-software/seg3d.html, Utah, USA, freeware) and Invesalius 3 (http://www.cti.gov.br/en/node/395/, Campinas, Brazil, open source). Blender (Stichting Blender Foundation, Amsterdam, the Netherlands) and MeshLab (Visual Computing Lab – ISTI – CNR, Pisa, Italy, open source) can be used to verify and improve the STL files before they are imported into the simulator. A more automated process would improve user-friendliness and reduce process time and the need for dedicated, specifically trained personnel.

The simulation

The EVAR procedure was simulated after surgical puncture, followed by the deployment of the Bolton stent graft system and ended with the final angiogram. It was possible to make short-cuts in the procedure through the SW interface. This was practical when a restart was necessary either due to technical or human failure, and made the simulation flexible, e.g., if the operators wanted to train on specific aspects of the procedure. Surgical VR simulators have mainly focused on individual technical skills, whereas PsR opens up possibilities of team-training. In traditional medical simulation, team behaviour is an important aspect, and there is a focus on the role-play with a briefing and a debriefing before and after the simulation [33]. Before the study started, we debated whether the PsR should resemble an operating room situation, with a briefing, a role-play and a debriefing. We chose not to, and gave instead the two operators together with the radiographer an arena where questions could be posed, mistakes were allowed, and where there was time for reflection and preparing for the real procedure. The operators rated the PsR to have a positive impact on team performance both after the simulation and after the procedure (median 4). Especially younger, more inexperienced operators seemed to appreciate the moments of practical training together with a senior operator, which was also reflected in their answers being more positive than those of the more experienced operators. EVAR procedures at this hospital are performed by one vascular surgeon and one IR. The PsR became a new arena where they could practice together, again it seemed like the inexperienced surgeons saw the PsR as an opportunity to learn from the more experienced IR in a way that was not common. The operators were free to repeat specific steps, or perform the simulation as many times as they wanted. Our experience, however, was that the operators, in their busy schedule, focused on getting through one PsR. In some cases, due to occlusions, they would repeat the simulation, but they did not repeat specific steps in order to ‘automate’ them. A future study could investigate whether a briefing, role-play and debriefing set-up would have different outcomes than our set-up.

Physical resemblance is not what counts in the end. Functional task alignment is so, i.e., whether the PsR aligns with learning objectives, or eventually improves patient outcomes or operation efficiency [34]. Nevertheless, physical resemblance as rated through the operators’ appraisal might give indications on functional task alignment. Our results from the operators’ appraisal questionnaire indicated moderate physical resemblance when asked whether ‘the simulated model was realistic’ (median 3), ‘the simulation of the aorta and the iliac arteries was realistic’ (median 3) and ‘the simulated model gave realistic haptic feedback’ (median 2). This is comparable to what has been found on other simulators offering PsR [16,17]. Realistic haptic feedback has been found important for skills transfer on laparoscopic VR simulators [35,36], and probably has similar effects on endovascular skills. The operators would also point out that the stent graft was more slippery in the simulator than they were used to in real patients, making accurate positioning of the components more difficult. The CT images were used as input for the PsR, but biomechanical properties such as effects of rigidity (calcification) or stenosis (atherosclerosis) were not simulated. Neither was the vessel deformation that solid instruments have on tortuous arteries. The vessel deformation influences, e.g., the optimal length of the stent graft components. The operators estimate the lengths of the components according to measurements from the 3D SW and the fact that tortuous arteries straighten out when solid instruments are inserted. As the last aspect cannot be measured the operators might use the ability to do a concertina technique (longitudinal compression) of the stent-graft leg to potentially compensate for too long components. Neither of these two aspects, first the biomechanical properties of the aorta and secondly the mechanical properties of the stent-graft legs were simulated in the PsR, potentially giving misleading expectations and for the younger operators limited opportunities to both practice and keep in mind these important dynamic adaptations that the operators do. The position of the renal arteries relative to the column is often used to pre-position the neck of the stent graft before the first angiogram. The simulator offered the possibility of adjusting the patient’s segmented aorta to the simulator’s template of the column, thus allowing the operators to do so. Several operators, though, pointed out that the proximal part of the renal arteries often had a conic form, as can be seen in Figures 2(E)–3(E), which influenced the ability to pre-position the stent graft accurately. It seemed like the STL file, after it had been stitched to the simulator template, was simplified, which resulted in the conic form of the renal arteries and sometimes unexpected angles of the aorta (Figure 2(E), 3(E) and 4).

Answers to operator appraisal questionnaires can be influenced by several aspects, one of them being the context of the PsR with regard to dedicated time and personnel. It was decided that a dedicated radiographer would prepare the CT images and upload them to the simulator. She would also organise the PsRs and would act as a mediator. This was well perceived by the operators and made the PsRs as smooth and flexible as possible, within the busy schedules of the operators. Despite having a dedicated person organising the PsR, the PsR lasted between fifteen minutes and one hour, time that needs to be justified and incorporated into routine clinical work.

The IRs used dedicated 3D software to measure stent-graft components and compute C-arm angulations prior to the PsR. The operators would use the computed C-arm angulations when performing the PsR, but had different views on the added value of the PsR with regards to C-arm angulation during the real procedure (range 1–5, median 4). Whether the operators and other stakeholders see the added value of PsR, compared to the use of dedicated 3D software combined with a mental rehearsal or other tools that can be used to prepare for EVAR, such as 3 D printing [37,38], has several aspects, such as time, expertise, cost of the simulator and potentially improved patient outcome and operation efficiency. Time and expertise to generate the patient-specific model ought to be limited, requiring less time and no dedicated person. The simulator and the patient-specific module are expensive, but the simulator can be used for training in addition to PsR. If the simulator could replace dedicated 3D SW to also measure and choose stent graft components, that would reduce total cost considerably, but that is not the case today.

In summary, PsR was feasible and was evaluated by operators to improve individual and team performance. Based on operators’ appraisal PsR can further be improved by increasing biomechanical realism and the ease of importing patient-specific data.

Acknowledgements

We thank all those who participated in the study. The work was supported by a grant from the Central Norway Regional Health Authority, SINTEF, the Norwegian University of Science and Technology (NTNU), and the Norwegian National Advisory Unit for Ultrasound and Image-Guided Therapy at St. Olavs Hospital (all Trondheim, Norway).

Declaration of interest

Cecilie Våpenstad, Siv Marit Lamøy, Frode Aasgaard, Asbjørn Ødegård, Torgeir K. Haavik, Toril Nagelhus Hernes, Knut Haakon Stensæth, Edmund Søvik have no conflict of interest or financial ties to disclose.