Unmanned aerial systems (UAS) operators’ accuracy and confidence of decisions: Professional pilots or video game players?

Abstract

Unmanned Aerial Systems (UAS) operations have outpaced current training regimes resulting in a shortage of qualified UAS pilots. Three potential UAS operator groups were explored for suitability (i.e. video game players [VGP]; private pilots; professional pilots) and examined to assess levels of accuracy, confidence and confidence-accuracy judgements (W-S C-A) during a simulated civilian cargo flight. Sixty participants made 21 decision tasks, which varied across three levels of danger/risk. Scales of Tolerance of Ambiguity, Decision Style and NEO-PIR were also completed. Professional pilots and VGPs exhibited the highest level of decision confidence, with VGPs maintaining a constant and positive W-S C-A relationship across decision danger/risk. As decision danger/risk increased, confidence, accuracy and W-S C-A decreased. Decision danger also had a role to play in the confidence expressed when choosing to intervene or rely on automation. Neuroticism was negatively related, and conscientiousness positively related, to confidence. Intolerance of ambiguity was negatively related to W-S C-A. All groups showed higher levels of decision confidence in decisions controlled by the UAS in comparison to decisions where the operator manually intervened. VGPs display less overconfidence in decision judgements. Findings support the idea that VGPs could be considered a resource in UAS operation.

Keywords:

• human factors
• automation
• decision-making
• confidence
• accuracy

Public Interest Statement

The move to significant automation has been a feature of aviation over the last 40 years. This paper describes three potential unmanned aerial supervisor (UAS) groups; video game players, private pilots and professional pilots who made 21 decisions across three levels of dangerousness. As danger increased levels of confidence, accuracy and the relationship between how accurate the decision was and the level of confidence applied to those decisions decreased. The dangerousness of the decision also affected how confident participants were when choosing to intervene or rely on the automation; confidence was lower when the operator chose to intervene. Understanding which potential supervisory group has the best skills to make the best decisions can help to improve UAS supervision. Overall, video game players were less overconfident in their decision judgements. The outcome supports the idea that this group could be a useful resource in UAS operation.

Additional information

Notes on contributors

Jacqueline M. Wheatcroft

Jacqueline M. Wheatcroft is a Chartered and Forensic Psychologist in the Institute for Psychology, Health & Society at the University of Liverpool, UK and is Chair of the British Psychological Society Division of Forensic Psychology Training Committee. Her research interests are in the enhancement of information, intelligence and evidence with a focus on process and procedural techniques to increase accuracy and appropriate confidence in information forms. She upholds interdisciplinary research and has published widely in areas that relate to security, law enforcement and legal process. Her work has contributed to governments, police, professional investigators and the courts.