On the effectiveness of virtual reality-based training for surgical robot setup

ABSTRACT

Virtual Reality (VR) is rapidly increasing in popularity as a teaching tool. It allows the creation of a highly immersive, three-dimensional virtual environment. With more robots saturating the industry, there is a need to train end-users on how to set up, operate, tear down, and troubleshoot the robot. While VR has become widely used in training surgeons on the psychomotor skills associated with operating the robot, little research has been done to see how the benefits of VR could translate to teaching the bedside staff, tasked with supporting the robot during the surgical procedure. We trained 30 participants on how to set up a robotic arm in an environment mimicking clinical setup. We divided these participants into three groups with one group trained with paper-based instructions, one with video-based instructions and one with VR-based instructions. We then compared these three different training methods. VR and paper were highly favoured training mediums over video. VR-trained participants achieved slightly higher fidelity of robotic joint angles, suggesting better comprehension of the spatial awareness skills necessary to achieve desired arm positioning. In addition, VR resulted in higher reproducibility of setup fidelity and more consistency in user confidence levels.

KEYWORDS:

• Medical training
• surgical robotics
• virtual reality

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Johnson and Johnson [Partial support]; Johns Hopkins University [Internal funding].

Notes on contributors

Arian Mehrfard

Arian Mehrfard earned his BSc degree in Computer Science from the University of Bremen in 2018. He is currently pursuing a MSc degree in Biomedical Computing from the Technical University of Munich and is expected to graduate in 2021. During his masters degree he has conducted research in the Laboratory for Computational Sensing and Robotics at the Johns Hopkins University. Arians research interests include the applications of mixed reality and robotics in clinical applications.

Javad Fotouhi

Javad Fotouhi holds a PhD degree in Computer Science from Johns Hopkins University. Prior to his PhD at the Laboratory for Computational Sensing and Robotics, he earned his MSE degree in Robotics from Johns Hopkins University, MSc degree in Biomedical Computing from Technical University of Munich, and BSc degree in Electrical Engineering from the University of Tehran. During his PhD, he was selected as a Siebel Scholar that recognizes the top students from the world’s leading graduate schools for their academic excellence and demonstrated leadership. Javad’s research focus includes the applications of augmented reality, machine learning, and robotics in interventional medicine.

Tess Forster

Tess Forster holds a B.S. in Biomedical Engineering from Boston University. In her nearly 10 year career in the healthcare sector, she has found her passion primarily focused on leveraging technology to advance education and improve patient outcomes. She is currently at Johnson & Johnson leading efforts to enable surgical teams to be more informed and efficient in the operating rooms. Her research interests include applications of virtual reality and machine learning to aid clinical teams in and beyond the operating room.

Giacomo Taylor

Giacomo Taylor received a MSE in computer science, as well as a dual BS in computer science and applied mathematics & statistics from the Johns Hopkins University in 2019. During his time at JHU, he conducted research in the Laboratory for Computational Sensing and Robotics. After graduation, he joined Verb Surgical Inc.’s applied research team developing computer vision and advanced image processing techniques for medical robotics. Giacomo now works as a software engineer for the perception team at Zoox Inc.

Danyal Fer

Danyal Fer is a general surgery resident at University of California San Francisco East Bay and a Captain in the United States Air Force. He is currently the Surgical Translational Research Lead in Applied Research at Johnson and Johnson Medical Device and visiting scholar at University of California Berkeley Automation Laboratory. His research focuses on transmission of surgical knowledge and action utilizing robotics and artificial intelligence.

Deborah Nagle

Dr. Deborah Nagle is the Chief of the Division of Colon and Rectal Surgery at Stony Brook University Hospital, Stony Brook, NY. She was born in Trenton, NJ, received a BA from Barnard College, Columbia University, New York, NY, and her MD from the University of Pennsylvania, Philadelphia, PA. Her general surgery training was conducted at the University of Pennsylvania, Graduate Hospital, Philadelphia, PA, and was followed by colorectal surgery training at the Thomas Jefferson University Hospital in Philadelphia, PA. She was subsequently on the staff at Drexel University College of Medicine and Cooper University Hospital. In 2006, she moved to Boston to begin the Division of Colon and Rectal Surgery and Beth Israel Deaconess Medical Center, where she was the Division Chief of Colon and Rectal Surgery at the Beth Israel Deaconess Medical Center in Boston, MA until 2016, and an assistant professor of Surgery at Harvard Medical School in Boston, MA until 2018. From 2016 to 2020 she worked at Ethicon, Inc. on Endomechanical and Surgical Oncology as well as Digital Surgery. Dr. Nagle is Board certified in general and colon and rectal surgery. Her contribution to the medical literature includes 16 book chapters/reviews and 27 peer review articles. She teaches residents and students on a daily basis. In addition to her administrative duties, Dr. Nagle is an active researcher and maintains a busy clinical practice. She is a fellow of the American College of Surgeons and the ASCRS. She is an active member of several surgical societies and has chaired several national committees of the ASCRS. She also served as program co-chair for the ASCRS annual meeting in 2003. She also currently serves on the ASCRS Executive Council as a member-at-large.

Mehran Armand

Mehran Armand received Ph.D. degrees in mechanical engineering and kinesiology from the University of Waterloo, Ontario, Canada. He is a Professor of Orthopaedic Surgery and Research Professor of Mechanical Engineering at the Johns Hopkins University (JHU) and a principal scientist at the JHU Applied Physics Laboratory. Prior to joining JHU/APL in 2000, he completed postdoctoral fellowships at the JHU Orthopaedic Surgery and Otolaryngology - head and neck surgery. He currently directs the laboratory for Biomechanical and Image Guided Surgical Systems (BIGSS) at JHU Whiting School of Engineering. He also co-directs the AVECINNA Laboratory for advancing surgical technologies, located at the Johns Hopkins Bayview Medical Center. His lab encompasses research in continuum manipulators, biomechanics, medical image analysis, and augmented reality for translation to clinical applications of integrated surgical systems in the areas of orthopaedic, ENT, and craniofacial reconstructive surgery.

Nassir Navab

Nassir Navab is a full Professor and Director of the Laboratory for Computer Aided Medical Procedures, Technical University of Munich and Johns Hopkins University. He has also secondary faculty appointments at both affiliated Medical Schools. He completed his PhD at INRIA and University of Paris XI, France, and enjoyed two years of post-doctoral fellowship at MIT Media Laboratory before joining Siemens Corporate Research (SCR) in 1994. At SCR, he was a distinguished member and received the Siemens Inventor of the Year Award in 2001. He received the SMIT Society Technology award in 2010 for introduction of Camera Augmented Mobile C-arm and Freehand SPECT technologies, and the ‘10 years lasting impact award’ of IEEE ISMAR in 2015. In 2012, he was elected as a Fellow of the MICCAI Society. He has acted as a member of the board of directors of the MICCAI Society, 2007-2012 and 2014-2017, and serves on the Steering committee of the IEEE Symposium on Mixed and Augmented Reality (ISMAR) and Information Processing in Computer Assisted Interventions (IPCAI). He is the author of hundreds of peer reviewed scientific papers, with more than 35644 citations and an h-index of 87 as of October 16, 2020. He is author of more than thirty awarded papers including 11 at MICCAI, 5 at IPCAI and three at IEEE ISMAR. He is the inventor of 50 granted US patents and more than 50 International ones. His current research interests include medical augmented reality, computer-aided surgery, medical robotics, and machine learning.

Bernhard Fuerst

Bernhard Fuerst is the Digital Technologies & Medical Imaging Lead at Digital Solutions, Johnson & Johnson. He and his team strive to bring innovative technologies to clinical stakeholders to improve patient care. Prior to Verb Surgical’s acquisition through Johnson & Johnson he was a project lead with Verb Surgical’s Applied Research. His previous work also included leading a research group at The Johns Hopkins University, focusing on robotics ultrasound, interventional imaging, and augmented reality applications for surgical navigation. He holds a PhD (summa cum laude) and MSc in biomedical computing from the Technical University Munich, and a BSc in biomedical informatics. to bring innovative technologies to clinical stakeholders to improve patient care. Prior to Verb Surgical’s acquisition through Johnson & Johnson he was a project lead with Verb Surgical’s Applied Research. His previous work also included leading a research group at The Johns Hopkins University, focusing on robotics ultrasound, interventional imaging, and augmented reality applications for surgical navigation. He holds a PhD (summa cum laude) and MSc in biomedical computing from the Technical University Munich, and a BSc in biomedical informatics.