Artificial intelligence in virtual standardized patients: Combining natural language understanding and rule based dialogue management to improve conversational fidelity

Abstract

Introduction

Advances in natural language understanding have facilitated the development of Virtual Standardized Patients (VSPs) that may soon rival human patients in conversational ability. We describe herein the development of an artificial intelligence (AI) system for VSPs enabling students to practice their history taking skills.

Methods

Our system consists of (1) Automated Speech Recognition (ASR), (2) hybrid AI for question identification, (3) classifier to choose between the two systems, and (4) automated speech generation. We analyzed the accuracy of the ASR, the two AI systems, the classifier, and student feedback with 620 first year medical students from 2018 to 2021.

Results

System accuracy improved from ∼75% in 2018 to ∼90% in 2021 as refinements in algorithms and additional training data were utilized. Student feedback was positive, and most students felt that practicing with the VSPs was a worthwhile experience.

Conclusion

We have developed a novel hybrid dialogue system that enables artificially intelligent VSPs to correctly answer student questions at levels comparable with human SPs. This system allows trainees to practice and refine their history-taking skills before interacting with human patients.

Keywords:

• Communication skills
• simulation
• standardized patients

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.

Glossary

Virtual Standardized Patient: Avatar representation of a human standardized patient that can communicate using natural language.

Natural Language Understanding: The ability of a computing device to understand typical human speech.

Automated Speech Recognition: Any computing system that allows users to speak as a means of data input as opposed to traditional devices such as mice and keyboards.

Additional information

Funding

This project was funded (in part) by a National Board of Medical Examiners (NBME) Edward J. Stemmler, MD Medical Education Research Fund grant [NBME 1112-064]. The project does not necessarily reflect NBME policy, and NBME support provides no official endorsement. This project was also supported by funding from the Department of Health and Human Services Health Resources and Services Administration [HRSA D56HP020687]. This material builds upon work supported by the National Science Foundation under Grant No. 1618336.

Notes on contributors

Kellen R. Maicher

Kellen Maicher, MS, MFA, is a Learning and Development Consultant for The James Cancer Hospital and Solove Research Institute and Coordinator for the Ohio State University EdTech Incubator. His research interests include user experience design for virtual reality and affective virtual human development.

Adam Stiff

Adam Stiff was a PhD student in the Department of Computer Science and Engineering at Ohio State University when this work was completed. He is currently employed as an NLP Scientist at Infoscitex Corporation, working on dialogue system applications in the defense sector.

Marisa Scholl

Marisa Scholl, BS, is a Research Data Analyst in the Department of Obstetrics and Gynecology at The Ohio State University.

Michael White

Michael White, PhD, is a Professor in the Department of Linguistics. His primary research interests are in natural language generation and dialogue systems.

Eric Fosler-Lussier

Eric Fosler-Lussier, PhD, is the John Makhoul Professor and Associate Chair of Computer Science and Engineering. His research interests are in technology for understanding spoken language.

William Schuler

William Schuler, PhD, is a Professor in the Department of Linguistics at the Ohio State University. His research interests include computational psycholinguistics and computational cognitive modeling.

Prashant Serai

Prashant Serai was a PhD student in the Department of Computer Science and Engineering, The Ohio State University.

Vishal Sunder

Vishal Sunder was a PhD student in the Department of Computer Science and Engineering, The Ohio State University.

Hannah Forrestal

Hannah Forrestal was an Undergraduate student in Neuroscience at The Ohio State University.

Lexi Mendella

Lexi Mendella was an Undergraduate student in Biology at The Ohio State University.

Mahsa Adib

Mahsa Adib was an Undergraduate student in Biology at The Ohio State University.

Camille Bratton

Camille Bratton was a Medical Student at the University of Toledo, College of Medicine and Life Sciences.

Kevin Lee

Kevin Lee was a Master's student in the Department of Computer Science at Purdue University.

Douglas R. Danforth

Douglas Danforth, PhD, is a Professor in the Department of Obstetrics and Gynecology at The Ohio State University College of Medicine. His research interests involve using virtual patients and virtual reality for medical education.