Considering the importance of autonomous weapon system design factors to future military leaders

ABSTRACT

Despite the growing breadth of research related to the perceived risks and benefits of Autonomous Weapon Systems (AWS), there remains a dearth of research into understanding how perceptions of AWS among military officers are affected by design factors. This paper demonstrates that ease of use, and user perception of the concept of using an autonomous weapon system, would be less of a barrier to trusted deployment by this emerging generation of military leaders than ensuring that autonomous systems have robust, transparent and reliable decision-making processes and that operators or supervisors are able to meaningfully monitor the systems nominally under their command. The core contribution of this paper addresses the question of how deliberate design choices could improve or diminish the capacity of junior officers to exercise meaningful human control over autonomous systems.

KEYWORDS:

• Lethal Autonomous Weapon Systems
• meaningful human control
• design factors
• Military Ethics

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 These factors were:

• how well you are able to understand how the system works?

• how easy it is to use?

• how you feel about it while it is in use?

• how well it serves your purpose?

• how well it fits into the operational context?

2 The second list of factors was:

• a means to determine who has authorised a robot’s actions.

• a means to receive a description of the robot’s planned actions.

• a means to determine the level of risk to be assumed by a robot in targeting decisions.

• a means to confirm that the robot has understood my input or command.

• a means to determine the robot’s level of confidence in decisions that will drive its actions.

Additional information

Funding

This work was supported by Trusted Autonomous Systems Defence Cooperative Research Centre.

Notes on contributors

Jai Galliott

Dr Jai Galliott is Director of the Values in Defence & Security Technology Group within the University of New South Wales at Australian Defence Force Academy, where he also sits on the Faculty & University Boards. He is further a former Royal Australian Navy Officer and has authored/edited the following books: Military Robots: Mapping the Moral Landscape (Routledge 2016), Force Short of War in Modern Conflict (Edinburgh University Press 2019), Big Data and Democracy (Edinburgh University Press 2020) and Lethal Autonomous Weapons: Re-Examining the Law and Ethics of Robotic Warfare (Oxford University Press 2021).

Austin Wyatt

Dr Austin Wyatt is a research associate in the Values in Defence and Security Technology Group at The University of New South Wales at the Australian Defence Force Academy. His research concerns autonomous weapons, with a particular emphasis on their disruptive effects in Southeast Asia.