Simultaneous estimation of projector and camera poses for multiple oneshot scan using pixel-wise correspondences estimated by U-Nets and GCN

References

• Defferrard M, Bresson X, Vandergheynst P. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in neural information processing systems. p. 3844–3852. https://papers.nips.cc/.
   Google Scholar
• Felzenszwalb PF, Huttenlocher DP. 2006. Efficient belief propagation for early vision. Int J Comput Vis. 70(1):41–54. doi:10.1007/s11263-006-7899-4.
   Google Scholar
• Furukawa R, Masutani R, Miyazaki D, Baba M, Hiura S, Visentini-Scarzanella M, Morinaga H, Kawasaki H, Sagawa R. 2015. 2-dof auto-calibration for a 3d endoscope system based on active stereo. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Milan, Italy; IEEE. p. 7937–7941.
   Google Scholar
• Furukawa R, Mizomori M, Hiura S, Oka S, Tanaka S, Kawasaki H. 2018. Wide-area shape reconstruction by 3d endoscopic system based on cnn decoding, shape registration and fusion. In: or 2.0 context-aware operating theaters, computer assisted robotic endoscopy. In: clinical image-based procedures, and skin image analysis. Cham: Springer; p. 139–150.
   Google Scholar
• Furukawa R, Morinaga H, Sanomura Y, Tanaka S, Yoshida S, Kawasaki H. 2016. Shape acquisition and registration for 3d endoscope based on grid pattern projection. In: European Conference on Computer Vision Amsterdam, The Netherlands; Cham: Springer. p. 399–415.
   Google Scholar
• Furukawa R, Oka S, Kotachi T, Okamoto Y, Tanaka S, Sagawa R, Kawasaki H. 2020. Fully auto-calibrated active-stereo-based 3d endoscopic system using correspondence estimation with graph convolutional network. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) EMBS Virtual Academy; IEEE. p. 4357–4360.
   Google Scholar
• Geurten J, Xia W, Jayarathne U, Peters TM, Chen EC. 2018. Endoscopic laser surface scanner for minimally invasive abdominal surgeries. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; Granada, Spain: Springer. p. 143–150.
   Google Scholar
• Ikeuchi K, Matsushita Y, Sagawa R, Miyazaki D, Mukaigawa Y, Furukawa R, Kawasaki H. 2020. Active lighting and its application for computer vision –40 years of history of active lighting techniques. Cham: Springer.
   Google Scholar
• Kawasaki H, Furukawa R, Sagawa R, Yagi Y. 2008. Dynamic scene shape reconstruction using a single structured light pattern. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Anchorage, Alaska, USA; Ieee. p. 1–8.
   Google Scholar
• Liu X, Sinha A, Unberath M, Ishii M, Hager GD, Taylor RH, Reiter A. 2018. Self-supervised learning for dense depth estimation in monocular endoscopy Stoyanov, Danail. In: Or 2.0 context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis. Cham: Springer; p. 128–138.
   Google Scholar
• Ma R, Wang R, Pizer S, Rosenman J, McGill SK, Frahm JM. 2019. Real-time 3d reconstruction of colonoscopic surfaces for determining missing regions. In: International Conference on Medical Image Computing and Computer-Assisted Intervention Shenzhen, China; Springer. p. 573–582.
   Google Scholar
• Mahmood F, Chen R, Durr NJ. 2018. Unsupervised reverse domain adaptation for synthetic medical images via adversarial training. IEEE Trans Med Imaging. 37(12):2572–2581. doi:10.1109/TMI.2018.2842767.
   Google Scholar
• Mahmood F, Durr NJ. 2018. Deep learning and conditional random fields-based depth estimation and topographical reconstruction from conventional endoscopy. Med Image Anal. 48:230–243. doi:10.1016/j.media.2018.06.005.
   Google Scholar
• Maurice X, Albitar C, Doignon C, de Mathelin M. 2012. A structured light-based laparoscope with real-time organs’ surface reconstruction for minimally invasive surgery. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. San Diego, CA, USA; IEEE. p. 5769–5772.
   Google Scholar
• Nagakura T, Michida T, Hirao M, Kawahara K, Yamada K. 2006. The study of three-dimensional measurement from an endoscopic images with stereo matching method. In: 2006 World Automation Congress Budapest , Hungary; IEEE. p. 1–4.
   Google Scholar
• O’Toole M, Achar S, Narasimhan SG, Kutulakos KN. 2015. Homogeneous codes for energy-efficient illumination and imaging. ACM Trans Graph. 34(4):35:1–35:13. Available from: doi:10.1145/2766897.
   Google Scholar
• Rau A, Edwards PE, Ahmad OF, Riordan P, Janatka M, Lovat LB, Stoyanov D. 2019. Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int J Comput Assist Radiol Surg. 14(7):1167–1176. doi:10.1007/s11548-019-01962-w.
   Google Scholar
• Ronneberger O, Fischer P, Brox T. 2015. U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention Munich, Germany; Springer. p. 234–241.
   Google Scholar
• Salvi J, Pages J, Batlle J. 2004. Pattern codification strategies in structured light systems. Pattern Recognit. 37(4):827–849. doi:10.1016/j.patcog.2003.10.002.
   Google Scholar
• Schmalz C, Forster F, Schick A, Angelopoulou E. 2012. An endoscopic 3d scanner based on structured light. Med Image Anal. 16(5):1063–1072. doi:10.1016/j.media.2012.04.001.
   Google Scholar
• Song L, Tang S, Song Z. 2017. A robust structured light pattern decoding method for single-shot 3d reconstruction. In: 2017 IEEE International Conference on Real-time Computing and Robotics (RCAR) Okinawa, Japan; IEEE. p. 668–672.
   Google Scholar
• Stoyanov D, Scarzanella MV, Pratt P, Yang GZ. 2010. Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: International Conference on Medical Image Computing and Computer-Assisted Intervention Beijing, China. Springer. p. 275–282.
   Google Scholar
• Visentini-Scarzanella M, Sugiura T, Kaneko T, Koto S. 2017. Deep monocular 3d reconstruction for assisted navigation in bronchoscopy. Int J Comput Assist Radiol Surg. 12(7):1089–1099. doi:10.1007/s11548-017-1609-2.
   Google Scholar
• Wang J, Sankaranarayanan AC, Gupta M, Narasimhan SG. 2016. Dual structured light 3d using a 1d sensor. In: Computer Vision - ECCV 2016-14th European Conference; Amsterdam, The Netherlands, Oct 11-14, Proceedings, Part VI. p. 383–398. doi: 10.1007/978-3-319-46466-4_23.
   Google Scholar
• Zagoruyko S, Komodakis N. 2015. Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, Massachusetts, USA. p. 4353–4361.
   Google Scholar
• Žbontar J, LeCun Y. 2016. Stereo matching by training a convolutional neural network to compare image patches. J Mach Learn Res. 17(1):2287–2318.
   Google Scholar
• Zhang L, Curless B, Seitz SM. 2002. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: Proceedings First International Symposium on 3D Data Processing Visualization and Transmission. Padova, Italy; IEEE. p. 24–36.
   Google Scholar
• Zhang Z. 2012. Microsoft kinect sensor and its effect. IEEE MultiMedia. 19(2):4–12. Available from: https://www.microsoft.com/en-us/research/publication/microsoft-kinect-sensor-and-its-effect.
   Google Scholar