How Being Outvoted by AI Teammates Impacts Human-AI Collaboration

Abstract

Recent advances in artificial intelligence (AI) enable AI agents to go beyond simply supporting human activities and, instead, take more control in team decision-making. While significant literature has studied human-AI collaboration through the lens of AI as a “second opinion system,” this type of interaction is not fully representative of many human-human team collaboration scenarios, such as scenarios where each decision maker is granted equal voting rights for the team decision. In this research, we explore how imparting AI agents with equal voting rights to the human impacts human-AI decision-making and team performance. Using a human subjects experiment in which participants collaborate with two AI teammates for truss structure (aka, bridge) design, we manipulate a series of voting scenarios (e.g., AI agents outvoting the human vs. AI agents agreeing with the human) and AI performance levels (high vs. low performing). The results indicate that changes in human self-confidence are not consistent with whether the quality of the final team-voted design action is advantageous or disadvantageous relative to their own actions. The results also show that when humans are outvoted by their AI teammates, they do not show strong negative emotional reactions if the team-voted decision has an advantageous outcome. Additionally, AI performance significantly influences the human-AI team decision-making process and even one low-performing AI (i.e., an AI that is frequently incorrect) on the team can significantly deteriorate team performance. Taken together, this research provides empirical evidence on the effects of AI voting with equal decision authority on human-AI collaboration, as well as valuable insights supporting real-world applications of human-AI collaboration via voting.

Keywords:

• Artificial intelligence
• trust
• human-computer interaction
• decision-making
• decision authority
• voting

Data availability statement

The data that support the findings of this study are available from the corresponding author KGL upon reasonable request.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Ethic statement

This study was approved by the Carnegie Mellon University Institutional Review Board (IRB ID: MODCR202200000088). All participants provided written informed consent prior to enrolment in the study.

Notes

1 AI suggestions are recommendations by an AI agent on a contribution to be made to improve the state of a problem.

Additional information

Funding

This material is partially supported by the Air Force Office of Scientific Research through cooperative agreements FA9550-18-0088 and FA9550-21-1-0442. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsor.

Notes on contributors

Mo Hu

Mo Hu is an assistant professor of Construction Science at Texas A&M-College Station. Her research mainly focuses on decision-making, human-AI interaction, and design neurocognition. She uses human subject experiments and neuroimaging techniques to explore cognition-behavior links in complex environments.

Guanglu Zhang

Guanglu Zhang is a research scientist in the Department of Mechanical Engineering at Carnegie Mellon University. His research interests include artificial intelligence, engineering design, GPU computing, and numerical methods.

Leah Chong

Leah Chong is a postdoctoral associate in the Department of Mechanical Engineering at Massachusetts Institute of Technology. Her research focuses on human-AI collaboration in engineering design, computational design methods, and human-centered design, effectively and ethically harnessing the strengths of humans and AI.

Jonathan Cagan

Jonathan Cagan is the David and Susan Coulter Head of Mechanical Engineering and George Tallman and Florence Barrett Ladd Professor at Carnegie Mellon University. Cagan’s research focuses on design automation and methods, problem solving, and medical technologies, merging AI, machine learning, and optimization methods with cognitive- and neuro-science problem solving.

Kosa Goucher-Lambert

Kosa Goucher-Lambert is an assistant professor in the Department of Mechanical Engineering and Jacobs Institute for Design Innovation at University of California, Berkeley. Goucher-Lambert’s research is in design theory, methodology, and automation. He combines computational methods with studies of human cognition and behavior to investigate problems in engineering and design.