Towards Human Objective Real-Time Trust of Autonomy Measures for Combat Aviation

ABSTRACT

Objective

Can objective trust indicators be measured and computed in real time in a within visual range aerial combat “dogfight” scenario consisting of a manned autonomous own-ship versus an adversary?

Background

Previous research has focused on human trust of automation during simulated ground combat and human trust of civil aviation automation. Those studies largely consisted of subjective measures of trust analyzed post-hoc. Human trust of autonomy in high-risk situations such as dogfighting had not yet been studied.

Method

Nine evaluation pilots participated in an experiment that consisted of nine operationally relevant live-flight vignettes during which they conducted a mission manager task while simultaneously monitoring an autonomous agent controlling their own-ship in an aerial dogfight. Eye tracking, objective real-time mental workload, and mission manager task performance were all recorded.

Results

Potential indicators of trust were objectively measured in real-time as a function of autonomy disengagement, human cross-check ratio of autonomy, and real-time objective mental workload, as validated by traditional subjective trust measures. Measured trust still needs to be categorized as appropriate trust, overtrust, or undertrust in future work.

Conclusion

This first-ever real-time measure of objective indicators of trust in operationally relevant live-flight paves the way for determining appropriate human trust of autonomy in high-risk situations such as dogfighting. These measures could have utility in high-risk manned-unmanned teaming applications such as working with robots and automated trading.

Acknowledgments

We would like to thank the United States Air Force Test Pilot School for funding this research as well as for providing expert Evaluation Pilots who are future leaders in the field of flight test and flight research.

Disclosure statement

No potential conflict of interest was reported by the author(s).