Touchless Interaction for Smart Glasses in Emergency Medical Services: User Needs and Experiences

Abstract

The application of smart glasses in healthcare, particularly for providers engaged in hands-on patient care tasks, presents unique design challenges. This study combines participatory design and usability testing to assess the user experiences of touchless interaction methods for smart glasses in the context of Emergency Medical Services (EMS). The participatory design workshops with 16 EMS providers reveal a preference for touchless interaction methods such as voice commands and pinch hand gestures, driven by the need to keep hands free and minimize cross-contamination risk. Despite this preference, the laboratory-based usability testing with 16 EMS providers indicates that both voice commands and hand gestures fall short in task performance compared to the default tangible buttons on smart glasses, primarily due to software limitations and EMS providers’ unfamiliarity with touchless techniques. Our findings reveal specific issues associated with using different interaction methods when operating smart glasses. Building on these insights, we discuss design implications for smart glasses to better align with the dynamic and unique characteristics of fast-paced medical work.

Keywords:

• Smart glasses
• touchless interaction
• usability
• user experience
• emergency medical services
• healthcare

Acknowledgements

We extend our heartfelt gratitude to all the EMS providers who participated in our research. We would also like to express our sincere appreciation to Jack Finkelstein at Interfaith Medical Center (New York City, New York), Dr. Jared Kutzin at Mout Sinai Health System (New York City, New York), Humaira Ali at Maimonides Medical Center (New York City, New York), Dr. Kathleen Adelgais at the University of Colorado and Children’s Hospital Colorado (Aurora, Colorado), and Brittany Olvera at Colorado Springs Fire Department (Colorado Springs, Colorado) for their support in coordinating the studies and recruiting participants.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 https://intercom.help/vuzix/en/articles/5954802-overview.

2 https://www.crunchfish.com/gestures/.

Additional information

Funding

This study was supported by the National Science Foundation [grant# 1948292] and the Agency for Healthcare Research and Quality [grant# 1R21HS028104-01A1].

Notes on contributors

Zhan Zhang

Zhan Zhang is an Associate Professor in the School of Computer Science and Information Systems at Pace University. His research spans the fields of Human-Computer Interaction and Health Informatics, with a particular focus on the design and development of innovative technologies that facilitate the work of fast-paced medical teams.

Enze Bai

Enze Bai is a PhD candidate in Computer Science in the School of Computer Science and Information Systems at Pace University. Supervised by Dr. Zhan Zhang, his research focuses on developing smart glass applications to support patient information sharing and collection in pre-hospital care.

Aram Stepanian

Aram Stepanian graduated from the School of Computer Science and Information Systems at Pace University with a concentration on user experience design. He was supervised by Dr. Zhan Zhang and is now a Senior User Experience Designer at Walmart Global Tech.

Swathi Jagannath

Swathi Jagannath is a User Experience Researcher at Microsoft currently working on Azure Cloud products. Her prior research focus was on healthcare information systems, specifically understanding activity delays in time-critical settings to inform the design of clinical decision-support systems.

Sun Young Park

Sun Young Park is an Associate Professor in Stamps School of Art and Design and the School of Information at the University of Michigan. Her research focuses on evaluating and designing health information systems and technologies in clinical and non-clinical settings.