Real-Time Workload Estimation Using Eye Tracking: A Bayesian Inference Approach

Abstract

Workload management is a critical concern in shared control of unmanned ground vehicles. In response to this challenge, prior studies have developed methods to estimate human operators’ workload by analyzing their physiological data. However, these studies have primarily adopted a single-model-single-feature or a single-model-multiple-feature approach. The present study proposes a Bayesian inference model to estimate workload, which leverages different machine learning models for different features. We conducted a human subject experiment with 24 participants, in which a human operator teleoperated a simulated High Mobility Multipurpose Wheeled Vehicle (HMMWV) with the help from an autonomy while performing a surveillance task simultaneously. Participants’ eye-related features, including gaze trajectory and pupil size change, were used as the physiological input to the proposed Bayesian inference model. Results show that the Bayesian inference model achieves a 0.823 F₁ score, 0.824 precision, and 0.821 recall, outperforming the single models.

KEYWORD:

• Human-automation interaction
• human-autonomy interaction
• Bayesian inference
• workload estimation

Acknowledgement

We acknowledge the technical and financial support of the University of Michigan Automotive Research Center (ARC) in accordance with Cooperative Agreement W56HZV-19-2-0001 U.S. Army CCDC Ground Vehicle Systems Center (GVSC), Warren, MI.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

These reserch financial supported by the Automotive Research Center (ARC) in accordance with Cooperative Agreement W56HZV19-2-0001, U.S. Army Ground Vehicle Systems Center (GVSC) Warren, MI [OPSEC6336].

Notes on contributors

Ruikun Luo

Ruikun Luo is a PhD student in Robotics, University of Michigan. He obtained a MS in Mechanical Engineering from Carnegie Mellon University in 2014 and a BS in Mechanical Engineering and Automation from Tsinghua University, China in 2012.

Yifan Weng

Yifan Weng is a PhD student in Mechanical Engineering, University of Michigan. He received his MS in Mechanical Engineering from the University of Michigan in 2018 and his BS in Mechanical Engineering from Purdue University in 2016.

Paramsothy Jayakumar

Paramsothy Jayakumar is a Senior Technical Expert at the U.S. Army DEVCOM Ground Vehicle Systems Center in Warren, Michigan. He received his PhD in structural dynamics from the California Institute of Technology in 1987.

Mark J. Brudnak

Mark J. Brudnak is a Senior Technical Expert at the U.S. Army DEVCOM Ground Vehicle Systems Center in Warren, Michigan. He obtained his PhD degree in Systems Engineering from Oakland University, Rochester, MI, in 2005.

Victor Paul

Victor Paul is a Technical Expert at the U.S. Army DEVCOM Ground Vehicle Systems Center in Warren, Michigan. He holds extensive knowledge in the area of motion base simulation and its application in both man and hardware in the loop experiments.

Vishnu R. Desaraju

Vishnu R. Desaraju is a Senior Research Scientist at Toyota Research Institute. He received his PhD in Robotics from Carnegie Mellon University in 2017.

Jeffrey L. Stein

Jeffrey L. Stein is a Professor of Mechanical Engineering at the University of Michigan. He received his PhD degrees in Mechanical Engineering from the Massachusetts Institute of Technology, Cambridge in 1983.

Tulga Ersal

Tulga Ersal is an Associate Research Scientist in the Department of Mechanical Engineering, University of Michigan. He received his PhD in Mechanical Engineering from the University of Michigan in 2007.

X. Jessie Yang

X. Jessie Yang is an Associate Professor in the Department of Industrial and Operations Engineering, University of Michigan. She received her PhD in Mechanical and Aerospace Engineering (Human Factors) from Nanyang Technological University, Singapore in 2014.