Human-agent teaming and trust calibration: a theoretical framework, configurable testbed, empirical illustration, and implications for the development of adaptive systems

Abstract

Given new technologies and algorithmic capabilities, human-agent teaming (HAT) is expected to dominate environments where complex problems are solved by heterogenous teams. In such teams, trust calibration is key; i.e. humans and agents working symbiotically, with humans trusting and relying on agents as appropriate. In this paper, we focus on understanding trust-calibration in HATs. We propose a theoretical framework of calibrated trust in HATs. Next, we provide a configurable testbed designed to investigate calibrated trust in HATs. To demonstrate the flexible testbed and our framework, we conduct a study investigating hypotheses about agent transparency and reliability. Results align with research to date, supporting the notion that transparency results in calibrated trust. Further, high transparency yielded more positive affect and lower workload than low transparency. We also found that increased agent reliability resulted in higher trust in the agent, as well as more positive valence. This suggests that participants experienced more engagement with the task when the agent was reliable and presumably trustworthy. We also build on our framework and testbed to outline a research agenda for the assessment of human trust dynamics in HATs and the development of subsequent real-time, intelligent adaptive systems.

Keywords:

• Human-agent teaming
• trust calibration
• transparency
• agent reliability

Acknowledgements

We thank the Army Research Office (Award Reference W911NF-19-1-0401) for support of this research. Opinions expressed in this article are those of the authors and not necessarily those of the ARO.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Philip Bobko

Philip Bobko is Professor Emeritus of Management and Psychology at Gettysburg College. He earned his PhD in Economic and Social Statistics from Cornell University and is past Editor of the Journal of Applied Psychology. His current interests are in human-­automation teaming, suspicion, human resources, and talent management.

Leanne Hirshfield

Leanne Hirshfield is an Associate Research Professor at the Institute of Cognitive Science at University of Colorado (CU), Boulder. Hirshfield directs the System Human Interaction with NIRS and EEG (SHINE) Lab at CU, where her research explores the use of non-­invasive brain measurement to passively classify users’ social, cognitive, and affective states in order to enhance usability testing and adaptive system design.

Lucca Eloy

Lucca Eloy is a PhD student in the Department of Computer Science and Institute of Cognitive Science at the University of Colorado Boulder. His research focuses on applying linear and nonlinear methods to non-invasive brain measurement (fNIRS) data to monitor interpersonal trust and trust in AI to improve human-agent collaborations via adaptive systems.

Cara Spencer

Cara Spencer is a first year computer science PhD student in the computer science department at University of Colorado Boulder. Her research interests are in measurement and modeling of human and team performance through non-invasive, unobtrusive measurement and machine learning, with a focus on high performing populations such as pilots and astronauts in extreme conditions.

Emily Doherty

Emily Doherty is a PhD student also in the Department of Computer Science and Institute of Cognitive Science at the University of Colorado Boulder advised by Leanne Hirshfield in the SHINE Lab. Her research is focused on how physiological signals, specifically, brain patterns, can be used to improve team collaboration and human-AI interactions.

Jack Driscoll

Jack Driscoll is a research assistant under Leanne Hirshfield at the University of Colorado (CU). He recently graduated from CU with an undergraduate degree in psychology and data science. He is interested in research that explores the intersection of the two fields, and particularly interested in using machine learning to gain insight from various forms of brain imaging data.

Hannah Obolsky

Hannah Obolsky is a research assistant under Leanne Hirshfield at the University of Colorado (CU). She is currently pursuing an undergraduate degree in aerospace engineering with a focus in bioastronautics. She is interested in researching the psychological and neurological changes that occur in the space environment, with an emphasis on building resilience for extreme stress scenarios.