Towards scarless surgery: An endoscopic ultrasound navigation system for transgastric access procedures

Abstract

Objective: Scarless surgery is an innovative and promising technique that may herald a new era in surgical procedures. We have created a navigation system, named IRGUS, for endoscopic and transgastric access interventions and have validated it in in vivo pilot studies. Our hypothesis is that endoscopic ultrasound procedures will be performed more easily and efficiently if the operator is provided with approximately registered 3D and 2D processed CT images in real time that correspond to the probe position and ultrasound image.

Materials and Methods: The system provides augmented visual feedback and additional contextual information to assist the operator. It establishes correspondence between the real-time endoscopic ultrasound image and a preoperative CT volume registered using electromagnetic tracking of the endoscopic ultrasound probe position. Based on this positional information, the CT volume is reformatted in approximately the same coordinate frame as the ultrasound image and displayed to the operator.

Results: The system reduces the mental burden of probe navigation and enhances the operator's ability to interpret the ultrasound image. Using an initial rigid body registration, we measured the mis-registration error between the ultrasound image and the reformatted CT plane to be less than 5 mm, which is sufficient to enable the performance of novice users of endoscopic systems to approach that of expert users.

Conclusions: Our analysis shows that real-time display of data using rigid registration is sufficiently accurate to assist surgeons in performing endoscopic abdominal procedures. By using preoperative data to provide context and support for image interpretation and real-time imaging for targeting, it appears probable that both preoperative and intraoperative data may be used to improve operator performance.

• Ultrasound
• navigation
• endoscopy
• natural orifice
• transgastric approach

Introduction

For centuries, the peritoneal cavity has been approached through large incisions in the anterior abdominal wall. In the past two decades, the laparoscopic approach has gained wide acceptance because it offers a safe and less invasive alternative: pain and the complications associated with large abdominal incisions are minimized, and recovery from the procedure is much more rapid. To further reduce the invasiveness of abdominal access, the next logical step is to eliminate the incision through the abdominal wall altogether: natural orifices may provide the most acceptable entry points for surgical interventions.

Several research groups have been able to gain access to the peritoneal cavity through per-oral transgastric (i.e., through a small incision in the gastric wall) and also per-anal transcolonic approaches in order to perform organ resections in animal models [1–6]. The first such procedures to be performed in the human abdomen, transgastric appendectomies, have been reported by Rao and Reddy in oral communications [7]. Recently, Marescaux and colleagues performed a transvaginal procedure for a cholecystectomy (http://www.websurg.com/notes). Based on these initial experiences, this new surgical technique has the potential to replace or augment the laparoscopic techniques currently used to treat many diseases. It may be especially beneficial to obese patients (for whom laparoscopic techniques may present practical difficulties), those who have undergone multiple procedures, or those who are at risk for adhesions. More generally, it will reduce scarring and wound exposure, thereby permitting a faster recovery.

Minimally invasive per-oral transgastric and per-anal transcolonic surgery (also known as Natural Orifice Transluminal Endoscopic Surgery–NOTES) is still in its infancy. A multi-disciplinary, multi-institutional team of fourteen leaders in the fields of surgery and endoscopy gathered in mid 2005 to analyze the barriers to widespread use of NOTES procedures in the abdomen [8]. They identified several challenges, summarized as follows:

1. Surgical management issues: Effective access to the peritoneal cavity; near-perfect gastric (intestinal) closure; and prevention of infection, particularly from the contents of the stomach or colon when they are opened to the abdominal cavity.

2. Instrumentation issues: Development of suturing and anastomotic (non-suturing) devices.

3. Navigational issues: Support for spatial orientation and development of a multi-tasking platform to accomplish procedures.

4. Event management: Control of intra-peritoneal hemorrhage; management of iatrogenic events; and identification and management of physiologic untoward events and compression syndromes.

5. Training providers.

We have focused on the issues that can be addressed through advanced navigation and visualization technology (i.e., item 3 in the above list). Our goal is to provide the physician with improved visual feedback, clear indicators of instrument location and orientation, and support in the recognition of anatomic structures.

Laparoscopic ultrasound (LUS) and endoscopic ultrasound (EUS) are often used to guide biopsies and interventional procedures. However, the vast majority of physicians are not comfortable with performing invasive procedures under ultrasound guidance due to difficulties in positioning the probe and also in interpreting the ultrasound (US) image. Understanding the position and orientation of the US B-scan plane is a ubiquitous problem, even for experienced sonographers. There is also difficulty in interpreting the US images because of the low contrast, reduced field of view, and acoustic window constraints, despite the close proximity of the US probe to the target organs.

The navigation of a flexible endoscopic device inside the abdomen presents similar challenges to those encountered in traditional laparoscopy, but new complexities are also added:

• The flexibility of the endoscope tip makes comprehension of its distal orientation difficult. Unlike many laparoscopic procedures, there is no direct observation of the endoscope tip. Due to the lack of a global reference for the tip with respect to the patient's body, successful navigation inside the stomach and in the abdominal cavity generally requires the expertise of a highly trained gastroenterologist (with up to two years sub-specialization).

• Many structures of clinical interest that are accessible through a transgastric access lie in a retrograde orientation with respect to the incision in the stomach wall. Access to such locations requires detailed knowledge of the location of the tip with respect to adjacent structures, particularly blood vessels.

• The appearance of the abdominal structures through the transgastric approach is different to that in an open or laparoscopic approach. Figure 1 shows a typical view from the endoscope in a transgastric procedure. While in laparoscopy the camera view is accomplished through a separate entry port, which permits direct observation of the instruments and organs, in an endoscopic procedure the camera, instrument channel and US probe are combined in the same instrument, which complicates the navigation and intervention tasks. The unique angle of view, the limited light, and the need to insert all instruments through a narrow channel are critical technical challenges. In addition, the introduction of the instruments into the peritoneal cavity (e.g., through a gastrotomy) should be performed in such a way as to preclude damage to surrounding organs and vasculature.

Several groups have attempted to address the challenges of orientation and interpretation in laparoscopic interventions by using preoperative data in conjunction with the intraoperative US data [9–13]. In particular, Lindseth et al. have shown that fusion of intraoperative US images and preoperative MRI enhances the perception by extending the overview of the operating field. Ellsmere et al. showed that a 3D display depicting the main vascular structures and the probe position improved spatial orientation for the operator of an LUS system, thus reducing the time taken to locate the organ of interest and increasing the operator's certainty. Other approaches rely on augmented reality to overlay the laparoscopic US image directly on the live images of a stereo-endoscope [14], [15], [16].

Figure 1. Endoscopic view of a transgastric procedure. [Color version available online.]

Expanding on our prior work [10], we have developed an Image Registered Gastroscopic Ultrasound (IRGUS) system that addresses those challenges and makes intra-cavitary interventional techniques easier to master and use in practice, and thus more likely to be widely adopted. IRGUS relies on the provision of context information relating to the interpretation of the US image based on preoperative CT or MRI data. The system is based on tracking the endoscope tip and thus the US plane with an electromagnetic tracker and establishing the correspondence of the real-time positioning of the instrument tip with respect to preoperative data. The preoperative data is also used to generate 3D models of reference anatomical structures. These structures are displayed with respect to the position of the probe in real time. An enhanced interpretation of the US image is achieved by oblique reformatting of the preoperative dataset according to the US plane location.

Materials and methods

Navigation system

The system consists of three major hardware components:

• A laparoscopic or endoscopic probe equipped with an ultrasound sensor and imaging system.

• A tracking device comprising a transmitter and receiver sensors.

• A host computer with a display for use by the physician.

Four coordinate systems are defined:

1. Ultrasound coordinate system (US): This system is defined with an origin in the top left-hand corner of the cropped US image. The y-axis is in the beam direction of the US image and the x-axis is in the lateral direction.

2. Receiver coordinate system (R): This is local to the sensor mounted on the probe.

3. Transmitter coordinate system (T): The coordinate frame of the stationary transmitter of the tracking device. The tracking system determines its position relative to R.

4. Patient coordinate system (P): The frame of the CT scanner; intraoperatively, this is also the coordinate frame of our display system.

These coordinate frames are related by transformations that are provided either by the tracking system or by computations. During reconstruction, every pixel in every B-scan has to be located with respect to the reconstruction coordinate system P. Let us say that T_A→B is a rigid body homogenous transformation (with a potential scaling) between coordinate system A and coordinate system B. A point in the B-scan plane is transformed to a point in the reconstruction coordinate system by means ofwhere x_US = [s_x u; s_y v; 0; 1]^T is a point in the B-scan plane and x_P is the pixel location in the coordinate system CT. u and v are the column and row indices of the pixel in the US image, and s_x and s_y are the pixel sizes in the lateral and axial directions, respectively. The system keeps track of these transformations, as well as storing them, so that the location of the US plane in the CT space can be known. The transformation T_US→R is known as calibration, T_T→P as registration, and T_R→T as sensor positions.

Calibration

Spatial calibration is performed to find the transformation T_US→R between the coordinate system attached to the US B-scan plane and the coordinate system of the position sensor. The real-time position of the B-scan plane is used to generate two of the principal displays (see Figure 2); this position is unknown until the calibration is performed. We have used the single-wall phantom method as described by Prager et al. [17]. The method consists of scanning the bottom of a flat surface, fitting a line to the bottom surface echo signal, and then solving for the 8 unknowns (6 degrees of freedom and 2 pixel sizes) as a non-linear least squares problem. We have used at least 400 US images and automatically extracted the echo line using a RANSAC fit [18]. The RANSAC fit was applied over the points corresponding to the maximum gradient responses along 20 equally spaced in-depth directions in the US image. The lines that were detected and did not fulfill the RANSAC criterion were discarded and not used in the final non-linear fitting problem. The calibration was repeated up to three times to ensure a consistent result. The overall calibration process took an average of one hour for each probe (laparoscopic and endoscopic). This calibration process was performed only once when the probes were mounted, and recalibration of the probes was not attempted before each new experiment. Based on previously published calibration accuracy results [19], calibration can be considered as one of the lower bounds of the final positioning error for the system.

Figure 2. System description: data flow paths and main displays. [Color version available online.]

Registration

The registration step is performed intraoperatively with the subject placed on the OR table and before the procedure takes place. The registration transformation T_T→P is found in two stages: an initial rigid registration and a real-time adaptive registration.

1. Initial rigid registration. This is performed by using either anatomical fiducials such as the rib tips [6] or high-contrast fiducials placed on the skin before the preoperative imaging. Let us call these fiducials {A₁, …, A_N}. The fiducials are identified in the CT image, generating a set of point coordinates {X_P (A₁), …, X_P (A_N)}. By means of a tracked pointer, the same fiducials are located in the patient, producing the measurements {X_T(A₁), …, X_T(A_N)} in the coordinate system T. We know now that the unknown transformation T_T→P has to fulfill the requirement that

The solution T_T→P to this system of linear equations is computed by Horn's pair-wise point-matching method [20]; the resulting transformation is denoted as .

1. Real-time adaptive registration. This is performed by using an additional position sensor, attached to the subject's thorax, as a local reference frame. This real-time adaptive registration compensates for rigid movements of the patient by updating the initial registration matrix, , based on the reading of the local reference frame. The registration transformation T_T→P is updated according to the expression

where is the transformation given by the sensor attached to the patient and is the initial transformation reported by the sensor after it is attached to the patient.

Display

The display consists of three primary elements (see Figure 2):

• Display 1: A 3D scene showing the patient skeleton and principal vascular structures obtained from the preoperative dataset, a model of the tracked endoscopic probe, and the position and orientation of the EUS plane.

• Display 2: A reformatted CT image in the oblique plane corresponding to the EUS image. The reformatted image is augmented by showing a greater area than that covered by the US plane. A square outline shows the area corresponding to the US field of view.

• Display 3: The EUS image (unmodified).

The navigation system has been integrated as a module in 3D Slicer (www.slicer.org). The three-display interface provided by the module is shown in Figure 3. The left panel allows the computer technician to interact with the tracking system, but essentially no real-time support is required for the use of the system. Different tools may be selected and position data, time stamps and images can be saved for retrospective analysis.

Figure 3. View of the system interface as presented to the clinician. [Color version available online.]

Model generation

The models corresponding to the 3D scene (display 1) are generated using three segmentation steps through a semi-automatic approach implemented in 3D Slicer.

First, the aorta and major vessel branches are extracted using a level set technique for vessel segmentation [21]. The level set incorporates an expansion term based on intensity that is manually set depending on the mean intensity observed in the vessel luminal area. The level set is initialized using a crude approximation of the vessel that is intended to extract the core of the aorta. The core is obtained by using an initial conservative threshold based on a high intensity and isolating the main component connected to a seed point placed in the aorta. The corresponding labelmap is eroded to give the core to be used to initialize the level set.

Second, the bone structures (primarily the spine and rib cage) are extracted. The original CT is masked using the previous vessel segmentation to facilitate the bone extraction. The masked CT is then thresholded and connected components are extracted to isolate the spine and ribs. A simple thresholding process is sufficient to extract the bone once the vessel structures have been masked, since the Hounsfield units corresponding to the bone and other classes of tissue are usually well separated.

Third, the kidney volumes are identified using a region-of-interest (ROI) approach. An ROI for each kidney is manually defined using a 3D box that can be adjusted in size to encompass the left and right kidneys, respectively. Based on this ROI, the CT volume is cropped and each kidney surface is individually extracted based on a geodesic active contour implemented using a level set technique [22]. The level set is initialized by manually placing seeds inside the kidney region. The level set evolution is supervised so that the process is stopped when leakage is observed.

Materials

The probe tracking system was electromagnetic (MicroBIRD, Ascension Technology Corp., Burlington, VT), and was connected to the computer through a PCI board. The sensor attached to the endoscope tip was 1.8 mm in diameter and 8.4 mm in length (see Figure 4). The sensor was mounted on the laparoscopic or endoscopic probe tip using medical-grade shrinkwrap tubing. Shrinkwrap is a thin plastic polymer that when heated shrinks to produce a tight layer around the probe shaft. In our experiments, this packaging has proven stable and easy to manage. The attachment of the sensor to the ultrasound probe causes an increase in overall probe diameter of less than 2 mm, which is acceptable for eventual use in humans. A similar MicroBIRD sensor was used to measure kidney motion (Experiment 2, described below). The US images were provided by a BK Panther Laparoscopic Ultrasound system for LUS acquisitions and an Olympus EU-C60 for EUS acquisitions. Both ultrasound systems have Doppler capabilities. Preoperative CT scans were acquired with a Siemens Sensation 64. Three scans were acquired per study: a baseline scan (with no added contrast media), an intravenous-injected contrast enhanced (I⁺) scan, and a delayed scan to show the venous structures. The I⁺ scan was used for model extraction and system guidance.

Figure 4. The sensor attached to the endoscope tip.

Experiment design

The system's performance and feasibility for in-vivo interventions were tested in a porcine model under general anesthesia. Free breathing was allowed and forced ventilation was only used during CT scanning to reduce breathing artifacts. Before CT scanning, four high-contrast CT markers were placed on the laterals of the rib cage. The CT data were acquired and the segmented models computed as described in the Navigation system section above. Within 24 hours, the subject was placed on the OR table, the fiducial markers were located, and the initial registration was performed as described in the Registration sub-section. The remainder of the experiments were then conducted.

The electromagnetic trackers for the laparoscopic and endoscopic probes were calibrated in the laboratory under ideal conditions. Special attention was paid to achieving a metal-free environment in order to minimize distortions in the tracker workspace. The calibration of both probes was completed within two hours and the results were used for all the experiments.

The OR set-up was as realistic as possible, including standard metal tables and surgical fixtures. The tracker system transmitter was placed next to the OR table using a non-metallic support and was slightly elevated above the OR table. This mounting was intended to minimize perturbations of the tracker system's electromagnetic field.

To assess the performance of our system and its feasibility for laparoscopic and endoscopic interventions, we conducted two sets of experiments to measure the spatial accuracy of the probe guidance system, followed by two sets of experiments to characterize user performance. In all cases, instrument motions and corresponding US images were recorded for retrospective analysis.

The first two spatial accuracy experiments (experiments 1 and 2) were designed to evaluate a global error number, in vivo, with respect to the associated registration error. Other investigators have identified several error components associated with a tracked US system: electromagnetic tracking error and distortions [23], calibration error [17], [24], and patient registration error. The second experiment directly measured the contribution of the breathing motion, specifically of the kidneys, to the global error measure.

The two user performance experiments (experiments 3 and 4) used a formalism developed in prior work [25], [26] to assess from the user's perspective the validity of our approach and the final user satisfaction.

Experiment 1

This experiment, using a tracked LUS probe, was designed to assess the total intraoperative registration error of our system. The goal was to have a global measure of the end-user performance of the system in terms of registration error.

Four vascular landmarks were used as references to assess the error based on the branching points between the aorta and major adjacent arteries, namely the celiac, superior mesenteric (SMA) and right and left renal arteries (see Figure 5). These branch points are used as landmarks for two reasons: they are natural landmarks that can be considered quasi-punctual, and can be easily detected in Doppler US due to the changes in the blood flow direction at the branching point, as can be seen in Figure 5. In addition, since the aorta is rigid and well connected to the spinal structures, these points may be considered as fixed (at the level of 1 mm) with respect to motion induced by respiration and organ deformation due to surgical instrument forces.

Figure 5. Landmark points chosen for system registration error validation and corresponding examples of the Doppler US views for each landmark point. The branching interface is coded in the Doppler US as a change of color due to a change in blood flow direction. [Color version available online.]

To assess the error, an expert was asked to perform a standard laparoscopic exploration and to display several US images where prescribed landmarks were clearly visible. For the vessel landmarks, Doppler ultrasound was employed to give an accurate location of the branch point. Independently, the same landmarks were identified in the CT volume. Given that the branch-point landmark has a finite extent, the expert was asked to define a set of points in the CT image covering the branch location. Then, the error for each selected US image relative to each CT landmark was computed. In this way, the sample set encompassed the variability due to the branch-point location uncertainty.

The system registration error was measured as the distance between a CT landmark and the US plane in which the same landmark was visible (see Figure 6 and Figure 9 [upper row]). The position of the US plane in the CT space was determined by the transformations given by our system. We considered the error in the normal direction to the US plane (out-of-plane error) to be primary, and any error within the US plane (in-plane error) to be residual. Our system is intended to provide contextual information (based on the reformatted CT); the purpose is to enable a good interpretation of the US image. We observed that physician operators could easily accommodate “in-plane” registration errors, since they experienced a greater mental burden compensating for errors in the out-of-plane direction than for those in-plane. Therefore, the normal error is the limit on how far the clinician needs to search for a target. Because the residual error is associated within the 2D plane that is being rendered, the error can be easily assimilated by a direct comparison between the CT and the US. Moreover, the reformatted CT has a greater extent than the US to allow for this integration task.

Figure 6. Registration error definitions.

To test for reliability and repeatability, the same surgical procedure was performed in three porcine specimens on different days. Each case was analyzed in the same fashion. The total number of error measurement samples was N = 2059. The sample set was distributed as follows: 563 samples corresponded to the celiac branch, 540 samples to the SMA branch, and 434 and 522 samples to the right and left renal branches, respectively. Regarding the number of samples per case, cases 1, 2 and 3 comprised 141, 593 and 1325 error measurements, respectively.

Experiment 2

Induced-respiration motion is an intrinsic error source that can preclude the application of a navigation system like the one presented in this paper. By evaluating the amount of respiration-induced motion in the kidneys we have attempted to provide a lower bound error for the expected mis-registration based solely on respiratory motion. Respiration-induced motion in the retro-peritoneum was measured by stitching an electromagnetic sensor to the right kidney surface. Tracking data were acquired during free breathing and forced (ventilated) breathing; we also recorded the insufflation and the heart rate. Motion data were examined with time-series analysis to find the principal harmonic that corresponded with the breathing frequency recorded during the experiment. After filtering that harmonic in the X, Y and Z time-series, a principal component analysis (PCA) was performed. The net motion was computed as 2(λ₁)^1/2 where λ₁ is the principal eigenvalue of the covariance matrix.

Experiment 3

A group of 3 experts and 5 novices in US-guided endoscopic intervention were asked to localize a predefined list of targets within a fixed time of 5 minutes. The users completed this task twice, once using the conventional EUS technique and again using our IRGUS system. We interleaved the tasks between users, so that half used the IRGUS system first and the rest used the EUS first. During the experiment, the location and orientation of the probe were recorded and we noted which structures were properly identified. Novice users were always assisted by an expert. From the positioning data, a kinematic evaluation of the user's motion was performed to characterize performance during the task [25]. Kinematic analysis provides measurements of the amount and smoothness of motion that is required to achieve the task. Finally, a questionnaire was used at the end of the task to assess subjective responses to the navigation system.

Experiment 4

A pilot transgastric procedure was conducted to test the feasibility of our system in transgastric interventions by an expert gastroenterologist. First, a transcutaneous radiofrequency (RF) ablation was performed in the liver to simulate a focal lesion. The experiment end-goal was to confirm the lesion location through a transgastric observation.

Results

Experiment 1

The registration error across landmarks and cases is plotted in Figure 7 using box-and-whisker type plots as described by Tukey [27]. In both analyses, the median error is consistent across different landmarks and across cases. The upper quartile is below 7 mm for all cases. The main sources of this error are factors such as specimen repositioning between the CT and the OR, CO₂ insufflation, respiratory motion, residual calibration error, and residual metallic distortions. The uncertainty in the location of the branch points that have been defined as point landmarks may also contribute to the total error. An analysis of the dispersion of the CT landmarks chosen by the expert was performed, and for all landmarks the mean distance between the landmark pair measurements was less than 2 mm.

Figure 7. Registration error measurements using box-and-whisker plots. (a) Error for each case considering the landmarks altogether. (b) Error for each landmark considering the cases altogether. [Color version available online.]

The total registration error for all landmarks and cases can be assessed from the error histogram shown in Figure 8. A high probability mode and a low probability mode for the error can be visually differentiated, with the boundary being around 5 mm. The low probability mode can be accounted for by outliers in the landmark positioning and therefore should not be taken into account for error evaluation. Table I summarizes the mean, median, standard deviation and cumulative probabilities at different levels for the total registration error and their deviation intervals. The sampling distributions of the measured statistics were obtained using a bootstrap resampling technique with 1000 bootstrap samples. Both mean and median registration error are less than 4 mm. The cumulative probability for registration errors greater than 5 mm is approximately 20%. This cumulative probability falls to 7.4% for registration error greater than 8 mm.

Figure 8. Histogram of the total registration error considering all the landmarks and all the cases in our sample population.

Table I.  Total registration error statistical analysis. Left: Bootstrap estimates for the mean, median and standard deviation (SD) of the out-of-plane error. Right: Bootstrap estimates for the cumulative probability of the out-of-plane error at 5 mm, 7 mm and 8 mm.

	Bootstrap estimate			Bootstrap estimate
Statistic	Mean (mm)	Std (mm)	Cumulative probability	Mean	Std
Mean	3.09	±0.055	P(E > 5 mm)	0.198	±0.0087
Median	2.383	±0.052	P(E > 7 mm)	0.119	±0.0072
SD	2.527	±0.045	P(E > 8 mm)	0.074	±0.0057

Figure 9 shows two examples of the system performance in the identification of the celiac branch and the right kidney. The context information provided by the reformatted CT facilitates the interpretation of the US image, while the 3D view gives a general reference frame. In both cases, the targeted branch point can be identified in both the US and the reformatted CT and corroborates the usefulness of the proposed paradigm.

Figure 9. Two examples of system performance during the evaluation process. The images show how the contextual information added by the three displays improves the awareness of the operator regarding the structure being imaged by the US. [Color version available online.]

In our experiments in the porcine model system, we placed fiducial markers on the skin of the subject. These are visible in the CT images and are directly probed before laparoscopy or endoscopy to determine the overall location and orientation of the body. While this approach would also be possible with human subjects, it would restrict the practical use of the system, and hence should ideally be replaced with an approach using anatomical features which may be reliably correlated with the CT model.

Experiment 2

Our data for respiration-induced kidney motion were acquired during free breathing and forced ventilation with an insufflation of 200 ml. The subject's heart rate was steady at 70 beat/s. A spectral analysis of the time series (Figure 10) shows a principal harmonic of motion at 0.6 Hz, which corresponds with the breathing frequency that was recorded during the experiment. The total displacement in the direction of maximum variance was 1.8 mm for forced breathing and 1.4 mm for free breathing.

Figure 10. Spectrum of the time series corresponding to forced breathing. A harmonic at approximately 0.6 Hz represents the main breathing component.

Experiment 3

Our third experiment showed that using the conventional EUS, novices identified only 29% of the structures and experts identified 50% of them within the allotted time. However, with the use of IRGUS these metrics increased to 71% and 80%, respectively. In addition, the analysis of kinematic data showed that when using IRGUS physicians not only identified more structures, but were also more efficient (see Table II). IRGUS improved the efficiency of conventional EUS by 17–27% in the analyzed characteristics. All differences were statistically significant at the level of p < 0.05. When asked about the user experience, both experts and novices agreed that IRGUS facilitates the navigation and the interpretation of the US content, thereby increasing overall confidence.

Table II.  Kinematic analysis comparing our system (IRGUS) and the conventional endoscopic approach (EUS). See reference 25 for a description of the parameters and analysis process.

Modality	Path length (cm)	Smoothness of motion d3d/dt3	Depth perception (cm)	Response orientation (radians)
EUS	1600.3	12.6	9877.5	52.5
IRGUS	1245.7	9.2	8174.2	42.3

Experiment 4

The transgastric pilot experiment showed that our system successfully assisted the intervention, leading to a positive confirmation of a selected lesion location. The primary benefit of the guidance system was the display of the probe position and orientation relative to the liver lesion location. Since the system displays the tip location and orientation in context with no lag, the operator could easily move in the right direction, confidently identify anatomic landmarks, and move smoothly to the target site. The gastric puncture and the access to the peritoneal cavity were completed directly, and the RF-induced lesion was identified and outlined successfully.

Discussion

The utility of our system does not depend on absolute spatial precision. Rather, we have relaxed the accuracy requirements for registration of patient anatomy given by the US to the preoperative volumetric images. This “reference” registration consists of relying on an initial rigid registration of the scanner space to the patient space, plus a real-time correction of this initial rigid registration computed by tracking the patient position with a sensor. By complementing real-time imaging with closely registered preoperative images, we aim to improve the way in which real-time images are interpreted, but without relying on the high-accuracy registration methods required by traditional image-based navigation systems [28]. We believe that reference registration is particularly suited to endoscopic abdominal surgery where, by using preoperative data for context and real-time imaging for targeting, distortions that limit the use of preoperative data can be overcome. It was observed that the accuracy of our approach lies within surgically acceptable limits and that the contextual information provided by our navigation system improves the performance of both expert and novice users.

We also found that when targets appeared in both the US plane and in the reformatted CT–regardless of the amount of displacement within that plane–users found the contextual information very useful in guiding interventions. A major concern was that the motion of organs (such as the kidneys) induced by respiration would compromise the utility of our system, but our experiments showed that this motion was limited, with the displacement being significantly smaller than the registration errors.

Novice clinicians performing US-guided endoscopic interventions found the system easy to master and stated that it improved their confidence in the identification of anatomic structures. The number of structures that they were able to correctly identify with the guidance system was double the number that they identified without the assistance of the system. Expert clinicians also found the navigation system to be of great help; it increased their confidence in the structures imaged by the real-time US and reduced their navigation time.

Although the main aortic trunk and the rib cage form the basic roadmap used to provide the clinician with a rough localization of the probe plane with respect to the patient's body, we have observed that adding kidney models provides a useful additional reference. More generally, we have found that conditions in the retroperitoneal portion of abdomen (at least in the porcine models investigated to date) are sufficiently stable with respect to both respiratory motion and deformations arising from the procedure that the IRGUS and IRLUS techniques can provide useful assistance to the operator. That is, the deformations in the CT models are manageable for navigation in practice. Confirmation of this finding in human subjects is clearly a critical next step.

Our system is a first step towards overcoming some key barriers to the implementation of transgastric interventions. The pilot transgastric procedure suggests that our system facilitates the navigation of the endoscope, thereby reducing the burden of a transgastric intervention in which the surgical field is limited with respect to both the view and the motions that are permitted. The nature of the intervention also limits the amount of deformation that can be induced in the abdominal cavity due to external forces, thereby increasing the applicability range of our system beyond structures that lie highly affixed to the retroperitoneum.

One of the major difficulties during the intervention is the need to avoid major stomach vessels when puncturing the stomach wall. This puncture can be performed with minimal bleeding if the vessels are avoided. However, if a vessel is accidentally compromised, the iatrogenic injury could lead to the death of the patient. We have simulated this situation by generating models–after stomach insufflation–of the stomach's major vessels and surface from the preoperative CT [29]. Figure 11 shows a synthetic endoscopic view from inside the stomach as it would appear in a real situation while defining the puncture location in the stomach wall. Our system can assist in this crucial task by tracking the position of the tip of the instrument in relation to the vessels of the stomach and other abdominal structures. An ideal situation would be to show the operator a confidence map based on the distance to the main vessels (Figure 11 b) to assist in the decision-making process. While our system is able to assist in this task, some problems still remain: The stomach surface before insufflation cannot be known a priori unless either the patient is scanned during the procedure or predictive models for stomach deformation under insufflation are developed. The latter is the most appealing option in terms of enabling the implementation of transgastric procedures in the surgical field, and opens the door to the exploration of new challenging computational problems in medicine.

Figure 11. Challenges for optimal stomach access to the abdominal cavity. (a) View of the vasculature anatomy of the stomach in a porcine model. The right image is an exterior view of the right portion of the specimen. The right gastrodoudenal artery (white arrow) can be seen lying on top of the stomach wall. The left image is an endoscopic view of the same site from inside the stomach looking towards the right-anterior part. The vessel structure has been overlaid with the semi-transparent model of the stomach wall, but this will not be visible in a standard endoscopic view. (b) A distance map to stomach vessels on the stomach surface. [Color version available online.]