Synthetic laparoscopic video generation for machine learning-based surgical instrument segmentation from real laparoscopic video and virtual surgical instruments

References

• Alaker M, Wynn GR, Arulampalam T. 2016. Virtual reality training in laparoscopic surgery: A systematic review & meta-analysis. Int J Surg. 29:85–94. doi:10.1016/j.ijsu.2016.03.034.
   Google Scholar
• Fuentes-Hurtado F, Kadkhodamohammadi A, Flouty E, Barbarisi S, Luengo I, Stoyanov D. 2019. EasyLabels: weak labels for scene segmentation in laparoscopic videos. Int J Comput Assist Radiol Surg. 14(7):1247–1257. doi:10.1007/s11548-019-02003-2.
   Google Scholar
• Funke I, Mees ST, Weitz J, Speidel S. 2019. Video-based surgical skill assessment using 3D convolutional neural networks. Int J Comput Assist Radiol Surg. 14(7):1217–1225. doi:10.1007/s11548-019-01995-1.
   Google Scholar
• García-Peraza-Herrera LC, Li W, Fidon L, Gruijthuijsen C, Devreker A, Attilakos G, Deprest J, Poorten EBV, Stoyanov D, Vercauteren T, et al. 2017. ToolNet: holistically-nested real-time segmentation of robotic surgical tools. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada. p. 5717–5722.
   Google Scholar
• Gibson E, Robu MR, Thompson S, Edwards P, Schneider C, Gurusamy K, Davidson B, Hawkes DJ, Barratt DC, Clarkson MJ. 2017. Deep residual networks for automatic segmentation of laparoscopic videos of the liver. In: Medical Imaging 2017: Image-Guided Procedures, Robotic Interventions, and Modeling, Proceedings of SPIE, Orlando, FL. vol. 10135, p. 101351M.
   Google Scholar
• Nicolau S, Soler L, Mutter D, Marescaux J. 2011. Augmented reality in laparoscopic surgical oncology. Surg Oncol. 20:189–201. doi:10.1016/j.suronc.2011.07.002.
   Google Scholar
• Nwoye CI, Mutter D, Marescaux J, Padoy N. 2019. Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int J Comput Assist Radiol Surg. 14(6):1059–1067. doi:10.1007/s11548-019-01958-6.
   Google Scholar
• Pfeiffer M, Funke I, Robu MR, Bodenstedt S, Strenger L, Engelhardt S, Roß T, Clarkson MJ, Gurusamy K, Davidson BR, et al. 2019. Generating large labeled data sets for laparoscopic image processing tasks using unpaired image-to-image translation. In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A, editors. Medical image computing and computer assisted intervention - MICCAI 2019 part V. LNCS (Vol. 11768). Cham: Springer; p. 119–127.
   Google Scholar
• Ronneberger O, Fischer P, Brox T. 2015. U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells W, Frangi A, editors. Medical image computing and computer-assisted intervention - MICCAI 2015 part III. LNCS (Vol. 9351). Cham: Springer; p. 234–241.
   Google Scholar
• Ros G, Sellart L, Materzynska J, Vazquez D, Lopez AM. 2016. The SYNTHIA dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV. p. 3234–3243.
   Google Scholar
• Roth HR, Oda M, Shimizu M, Oda H, Hayashi Y, Kitasaka T, Fujiwara M, Misawa K, Mori K. 2018. Towards dense volumetric pancreas segmentation in CT using 3D fully convolutional networks. In: Medical Imaging 2018: Image Processing, Proceedings of SPIE, Houston, TX. vol. 10574, p. 105740B.
   Google Scholar
• Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N. 2017. EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging. 36(1):86–97. doi:10.1109/TMI.2016.2593957.
   Google Scholar
• Zhu J-Y, Park T, Isola P, Efros AA. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: The IEEE International Conference on Computer Vision (ICCV), Venice, Italy. p. 2223–2232.
   Google Scholar