Abstract
Colonoscopy screening with a conventional 2D colonoscope is known to reduce mortality due to colorectal cancer by half. Unfortunately, the protective value of this procedure is limited by missed lesions. To improve the sensitivity of colonoscopy to precancerous lesions, 3D imaging techniques could be used to highlight their characteristic morphology. While 3D imaging has proved beneficial for laparoscopic procedures, more research is needed to assess how it will improve applications of flexible endoscopy. In this editorial, we discuss the possible uses of 3D technologies in colonoscopy and factors that have hindered the translation of 3D imaging to flexible endoscopy. Emerging 3D imaging technologies for flexible endoscopy have the potential to improve sensitivity, lesion resection, training and automated lesion detection. To maximize the likelihood of clinical adoption, these technologies should require minimal hardware modification while maintaining the robustness and quality of regular 2D imaging.
In conventional 2D endoscopy, physicians must infer the 3D environment they are examining from indirect cues, such as shading and movement parallax. The lack of direct depth information requires significant training to become accustomed to and can lead to fatigue, reduced efficiency and decreased accuracy. These drawbacks have prompted commercial manufacturers to incorporate a variety of technologies capable of 3D imaging in their rigid laparoscopes, which are now used in applications ranging from urology and gynecology, to general surgery and training. Surprisingly, 3D technologies have yet to translate to flexible endoscopes on a wide scale. This is likely due to both differing clinical needs and technological constraints in laparoscopy compared with flexible endoscopy. Laparoscopes are most often used in surgical procedures, which frequently involve complex 3D tasks such as suturing and dissection. Flexible endoscopes, on the other hand, are primarily used in diagnostic procedures and for taking simple biopsies, where the need for 3D visualization is less apparent. Furthermore, flexible endoscopes must pack more features into a single device than a typical laparoscope, which limits the space available to include the components conventionally needed to achieve 3D functionality. Here, we argue that there are several applications of flexible endoscopy that would benefit from 3D information and discuss emerging 3D technologies that may be suitable within the unique constraints of these applications. In particular, we focus on one of the most common procedures using a flexible endoscope – the colonoscopy.
Colonoscopy screening has been shown in large populations to reduce the risk of death from colorectal cancer by approximately half. The primary goal of this procedure is to examine the colon and identify and remove precancerous lesions called adenomas, which often have characteristic 3D morphology. However, these adenomas are frequently missed during screening colonoscopy, with polyp miss rates of 10–30% observed in tandem colonoscopy studies Citation[1]. Moreover, non-polypoid lesions are even more difficult to identify and are not screened for outside of Japan Citation[2].
Clearly, missed lesions are significantly limiting the protective value of screening colonoscopy. One accepted way to enhance lesion contrast is by spraying an indigo dye into the lumen and gently rinsing it away. In this technique, called chromoendoscopy, the dye accumulates at edges and crevices in the mucosa surface, effectively highlighting the high-frequency surface topology. Chromoendoscopy is known to increase adenoma detection rate but is not used routinely in colonoscopy screening because it typically doubles or triples the procedure time. This indicates that there is a need for 3D imaging techniques that increase lesion contrast without increasing procedure time. A 3D imaging technique may be valuable in improving the adenoma detection rate in routine colonoscopy screening.
In addition to improved lesion detection rates, there are several other applications where 3D imaging may be valuable in improving colonoscopy. When a clinically significant lesion is found, it can be challenging to remove. The polypectomy must remove the entire lesion, while minimizing bleeding and maintaining the integrity of the colon wall. There are a variety of techniques and tools for these procedures, most of which represent a challenging, inherently 3D task. The placement of the snare, in particular, may benefit from depth information. With non-ideal placement, the lesion can escape the snare, or, worse yet, the colon can be perforated. Better 3D guidance of the lesion removal may reduce complications and decrease procedure time in screening colonoscopy. Similar to benefits already observed in laparoscopy, 3D colonoscopy may also reduce training requirements and the problem of physician-fatigue. 3D imaging may also be helpful for reconstructing a large area of the colon (i.e., ‘mosaicking’) and for estimating the portion of the colon surface that was missed to provide a quality or coverage metric Citation[3]. Lastly, to date there has been little success in implementing reliable algorithms for computer-aided detection (CAD) of lesions. The additional information gathered in 3D imaging has enormous potential to improve the performance of CAD algorithms. Improved CAD algorithms could be especially useful considering that, in order to be cost-effective, the examination part of a routine colonoscopy screening must be performed quickly (typically in less than 10 min), and time pressures may be contributing to missed lesions Citation[4].
Most implementations of 3D laparoscopy incorporate two cameras in the endoscope tip to record stereo images. This approach works in laparoscopy because different entry ports can be used to introduce the needed tools and it is sometimes practical to have one port dedicated to imaging. But this approach is incompatible with colonoscopy because everything must be done in one endoscope. In addition to imaging and illumination, this includes channels for fluid exchange and manipulation tools. These features must be packed into an endoscope of less than 14 mm outer diameter. Considering these physical limitations in colonoscopy, an ideal approach to obtaining 3D information would be to implement a software-based technique. Software techniques to infer the 3D structure of the colon from video streams exist, and include simultaneous localization and mapping Citation[5], computational stereo Citation[6] and shape from motion Citation[7]. Changes in illumination with depth and surface orientation have also been used to recover the 3D structure of the colon from a single image using the shape from shading technique Citation[8,9]. A fusion of shape from shading and shape from motion has also been explored Citation[10]. Tubular models of the colon and its folds in conjunction with contour detection techniques have also been used to recover its 3D structure Citation[3,5]. However, all these techniques require either images from different points of view of the same region of the colon or strong assumptions to solve an ill-posed problem. Furthermore, the colon environment is constantly deforming during a colonoscopy, making video-based software reconstruction especially challenging.
In addition to software-based approaches, 3D endoscopy techniques utilizing dedicated hardware have also been explored. In general, these techniques are more robust for capturing the 3D environment, but are difficult to implement within the physical constraints of a colonoscope. The use of shape from polarization Citation[11] requires the addition of a rotating polarization filter to the objective, and is incompatible with anti-specular cross-polarization filters. Structured illumination Citation[12] requires a light pattern projection system that is complex to implement in forward-viewing endoscopes. Time-of-flight 3D endoscopy Citation[13] sensors are not available in a miniaturized form factor that would allow their use in colonoscopy. Our recent work on photometric stereo endoscopy (PSE) Citation[14] is an advance toward compatibility of 3D imaging with flexible endoscopy systems. A major strength of this technology is that it requires two relatively minor changes to an existing colonoscope: synchronized switching of multiple light sources and updated software implementing the PSE algorithm to compute surface orientations.
PSE can acquire the high frequency topography of the field of view in real-time using a conventional endoscope camera. Initial tests of the technology validated the ability to capture the shape of small features in tissues with heterogeneous optical properties, including lesions in gastrointestinal tissue. Because of the minor modifications required to an existing colonoscope, PSE is straightforward to implement with a commercial endoscope and the technique is now undergoing evaluation in a human clinical colonoscopy trial. Nevertheless, there are some disadvantages and obstacles that may ultimately limit PSE from clinical utility. First, the current implementation of the technique does not measure the low frequency variations in object shape. Second, PSE is an inherently qualitative technique. The absolute depths in the field of view cannot currently be determined. Despite these disadvantages, there is still evidence that qualitative, high-frequency-only topography may be useful in identifying lesions – this is, in fact, the same contrast that chromoendoscopy shows.
Several open questions remain concerning the ultimate use-model for 3D colonoscopy. What will be the most useful way to visualize the additional data acquired by 3D colonoscopy? Will the value resulting from a 3D view of the colon be enough to merit the adoption of these more expensive, more complex technologies in routine colonoscopy screening? Can 3D colonoscopy be used in such a way as to not increase procedure time? Can sufficient sensitivity be achieved for suggestions from 3D-enhanced CAD to be trusted by the physician?
The first implementations of 3D laparoscopy were introduced over two decades ago, but were not adopted until recently because they generated poorer-quality images and were difficult to visualize when compared with the conventional 2D alternatives Citation[15]. Now that more sophisticated 3D technologies are proving beneficial in laparoscopy, it is time to investigate how 3D imaging can improve applications of flexible endoscopy, such as colonoscopy.
Financial & competing interests disclosure
The authors are inventors in a US patent application filed by the Massachusetts Institute of Technology comprising Photometric Stereo Endoscopy.The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
No writing assistance was utilized in the production of this manuscript.
References
- Heresbach D, Barrioz T, Lapalus MG et al. Miss rate for colorectal neoplastic polyps: a prospective multicenter study of back-to-back video colonoscopies. Endoscopy 40(4), 284–290 (2008).
- Soetikno RM, Kaltenbach T, Rouse RV et al. Prevalence of nonpolypoid (flat and depressed) colorectal neoplasms in asymptomatic and symptomatic adults. JAMA 299(9), 1027–1035 (2008).
- Hong D, Tavanapong W, Wong J, Oh J, de Groen PC. Colon fold contour estimation for 3D visualization of colon structure from 2D colonoscopy images. 8th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, ISBI 2011. Chicago, IL, USA, 30 March–2 April 2011.
- Barclay RL, Vicari JJ, Doughty AS, Johanson JF, Greenlaw RL. Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N. Engl. J. Med. 355(24), 2533–2541 (2006).
- Noonan DP, Mountney P, Elson DS, Darzi A, Yang G-Z. A stereoscopic fibroscope for camera motion and 3D depth recovery during minimally invasive surgery. 2009 IEEE International Conference on Robotics and Automation. Kobe International Conference Center, Kobe, Japan, 12–17 May 2009.
- Liu Q, Sclabassi RJ, Yao N, Sun M. 3D construction of endoscopic images based on computational stereo. Bioengineering Conference, 2006. Proceedings of the IEEE 32nd Annual Northeast. Easton, PA, USA, 01–02 April 2006.
- Koppel D, Chen C-I, Wang Y-F et al. Toward automated model building from video in computer-assisted diagnoses in colonoscopy. Proceedings of SPIE 6509, Medical Imaging 2007: Visualization and Image-Guided Procedures, 65091L. doi:10.1117/12.709595 (2007).
- Deguchi K. Shape reconstruction from endoscope image by its shadings. Multisensor Fusion and Integration for Intelligent Systems, 1996. IEEE/SICE/RSJ International Conference. Washington, DC, USA, 8–11 December 1996.
- Suter M, Reinhardt JM, Hoffman EA, McLennan G. 3D pulmonary airway color image reconstruction via shape from shading and virtual bronchoscopy imaging techniques. Proceedings of SPIE 5747, Medical Imaging 2005: Image Processing, 755, doi:10.1117/12.596544 (2005).
- Kaufman A, Wang J. 3D surface reconstruction from endoscopic videos. In: Visualization in Medicine and Life Sciences. Linsen L, Hagen H, Hamann B (Eds). Springer, Heidelberg, Germany, 61–74 (2008).
- Herrera SEM, Malti A, Morel O, Bartoli A. Shape-from-polarization in laparoscopy. Biomedical Imaging (ISBI), 2013 IEEE 10th International Symposium. San Francisco, CA, USA, 7–11 April 2013.
- Schmalz C, Forster F, Schick A, Angelopoulou E. An endoscopic 3D scanner based on structured light. Med. Image Anal. 16(5), 1063–1072 (2012).
- Penne J, Höller K, Stürmer M et al. Time-of-flight 3-d endoscopy. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009. Springer, Heidelberg, Germany, 467–474 (2009).
- Parot V, Lim D, González G et al. Photometric stereo endoscopy. J. Biomed. Opt. 18(7), 076017 (2013).
- Storz P, Buess GF, Kunert W, Kirschniak A. 3D HD versus 2D HD: surgical task efficiency in standardised phantom tasks. Surg. Endosc. 26(5), 1454–1460 (2012).