Towards Exploring the Benefits of Augmented Reality for Patient Support During Radiation Oncology Interventions

ABSTRACT

Traditionally, patient education has been limited to verbal exchanges between providers and patients, along with paper handouts that summarise relevant information. While such exchanges are a natural step in educating patients, they are limited for several reasons, including the lack of time that provider teams are afforded, and the inherent challenge of communicating nuanced concepts related to complex medical procedures. A clear example of this is radiation oncology, in which traditional routes of patient education may not satisfy the patient’s needs. Although existing work has demonstrated the ability of audio-visual systems to improve patient engagement during medical procedures, the integration of emerging technologies such as Augmented Reality (AR) remains largely untapped. In this work, we propose an innovative proof-of-concept AR system as a first step towards exploring the benefits of using this technology during radiotherapy sessions. Our concept uses an AR headset to provide visual feedback of the patient's respiratory trace presented using two different forms: (i) a bi-dimensional graph and (ii) a game-based user interface. Moreover, we explore how interactive environments have the potential to contribute to better user experience and improve engagement, and discuss different challenges that must be addressed to deploy this technology to radiation treatment sessions.

KEYWORDS:

• Augmented reality
• radiotherapy
• deep inspiration breath hold
• audio-visual biofeedback breathing guidance

Acknowledgments

This research was partially supported by the Bayerische Forschungsstiftung, Grant DOK-178-17. We also want to extend our appreciation to the anonymous reviewers for all their valuable feedback.

Disclosure statement

Clinical trial research support from Boston Scientific..

Notes

1. SDX Respiratory Gating System (DYNR, Provence-Alpes-Cote dAzur, France)

2. ABC Coordinator 3.0 (ABC, Elekta, Stockholm, Sweden)

3. Unity3D (Unity Technologies, California, United States of America)

4. HoloLens 2 (Microsoft, Washington, United States of America)

5. CleanBox (Cleanbox Technology Inc, California, United States of America)

6. xvision Spine System (Augmedics, Illinois, United States of America)

7. Medivis (Medivis, New York, United States of America)

Additional information

Funding

This work was supported by the Bayerische Forschungsstiftung [DOK-178-17].

Notes on contributors

A. Martin-Gomez

Alejandro Martin-Gomez is a Ph.D. candidate in Computer Science at the Informatics Department of the Technical University of Munich. Prior to his Ph.D. at the Chair for Computer Aided Medical Procedures and Augmented Reality, he earned his MSc degree in Electronic Engineering from the Universidad Autonoma de San Luis Potosi, Mexico, and BSc degree in Electronic Engineering from the Instituto Technologico de Aguascalientes, Mexico. His research focus includes visual perception and the applications of augmented and virtual reality in interventional medicine.

C. Hill

Colin Hillmatriculated at the University of Virginia School of Medicine and is currently a fourth-year resident physician at the Department of Radiation Oncology and Molecular Sciences at the Johns Hopkins University School of Medicine. His research interests are currently in gastrointestinal radiation oncology, stereotactic radiation, motion management, and in promoting the importance of diversity for health equity for all.

H.Y. Lin

Hui-Yun Lin is a Master Student of the Robotics program at the Laboratory for Computational Sensing and Robotics of Johns Hopkins University. She earned her BSc degree of Mechanical Engineering from National Taiwan University. Her interests vary from robot navigation, haptic interface design for human-robot interaction to Augmented Reality.

J. Fotouhi

Javad Fotouhi holds a Ph.D. degree in Computer Science from Johns Hopkins University. Prior to his Ph.D. at the Laboratory for Computational Sensing and Robotics, he earned his MSE degree in Robotics from Johns Hopkins University, MSc degree in Biomedical Computing from the Technical University of Munich, and BSc degree in Electrical Engineering from the University of Tehran. During his Ph.D., he was selected as a Siebel Scholar that recognizes the top students from the world’s leading graduate schools for their academic excellence and demonstrated leadership. Javad’s research focus includes the applications of augmented reality, machine learning, and robotics in interventional medicine.

S. Han-Oh

Dr. Han-Oh is a medical physicist specializing in 4D image guidance and stereotactic radiotherapy in the Department of Radiation Oncology and Molecular Radiation Sciences at the Johns Hopkins University School of Medicine. She obtained her B.S. in physics at Ewha Womans University in Seoul, Republic of Korea, her M.S. in physics at Seoul National University in Seoul, Republic of Korea, her Ph.D. in medical physics at the George Washington University in Washington, D.C., and completed her post-doctoral fellowship at the National Cancer Institute, National Institutes of Health, in Bethesda, Maryland before joining Johns Hopkins Medicine. Her primary clinical and research interests focus on real-time tumor imaging, tumor motion management, and mixed reality patient education and feedback system for radiotherapy.

K.K.-H. Wang

Dr. Wang is an associate professor at the University of Texas Southwestern Medical Center. Before joining the University of Texas Southwestern, he was an assistant professor and senior clinical physicist at Johns Hopkins University. Dr. Wang’s group has made significant advances in image-guided radiation therapy for pre-clinical research, being the first to integrate 3D quantitative bioluminescence tomography (QBLT) to guide conformal irradiation. It laid the groundwork for quantitative optical imaging in target/tumor delineation and longitudinal assessment. Dr. Wang has two active R01 and one R21 grants in supporting the research of optical-guided radiation therapy.

N. Navab

Nassir Navab is a full Professor and Director of the Laboratory for Computer Aided Medical Procedures at the Technical University of Munich and Johns Hopkins University. He has also secondary faculty appointments at both affiliated Medical Schools. He completed his Ph.D. at INRIA and University of Paris XI, France, and enjoyed two years of post-doctoral fellowship at MIT Media Laboratory before joining Siemens Corporate Research (SCR) in 1994. At SCR, he was a distinguished member and received the Siemens Inventor of the Year Award in 2001. He received the SMIT Society Technology award in 2010 for the introduction of the Camera Augmented Mobile C-arm and Freehand SPECT technologies, and the ‘10 years lasting impact award’ of IEEE ISMAR in 2015. In 2012, he was elected as a Fellow of the MICCAI Society. He has acted as a member of the board of directors of the MICCAI Society, 2007-2012 and 2014-2017, and serves on the Steering committee of the IEEE Symposium on Mixed and Augmented Reality (ISMAR) and Information Processing in Computer Assisted Interventions (IPCAI). He is the author of hundreds of peer reviewed scientific papers, with more than 35644 citations and an h-index of 87 as of October 15, 2020. He is the author of more than thirty awarded papers including 11 at MICCAI, 5 at IPCAI, and 3 at IEEE ISMAR. He is the inventor of 50 granted US patents and more than 50 International ones. His current research interests include medical augmented reality, computer-aided surgery, medical robotics, and machine learning.

A.K. Narang

Dr. Narang is an Assistant Professor and specializes in the treatment of gastrointestinal malignancies, including the treatment of pancreatic, hepatobiliary, and ano-rectal cancers. Dr. Narang serves on the NCCN guidelines panel for pancreatic adenocarcinoma. At Johns Hopkins, he helps lead the pancreatic cancer care as co-director of the Pancreatic Multi-Disciplinary Clinic and as co-director of the Johns Hopkins Rectal Multi-Disciplinary Clinic. In his research, Dr. Narang has served as Principal Investigator for several clinical trials that are exploring novel ways to increase the intensity of radiation that safely be administered to patients with pancreatic cancer. Dr. Narang is also interested in developing higher value paradigms for gastrointestinal malignancies that can alleviate the financial pressures on patients and healthcare systems while preserving and improving outcomes. Dr. Narang additionally serves as Associate Program Director for the residency program and directs the medical student clerkship, helping to mentor trainees.