Autonomous technology – sources of confusion: a model for explanation and prediction of conceptual shifts

Abstract

Today, autonomous is often used for technology with a more intelligent self-management capability than common automation. This concept usage is maladaptive, ignoring both the distinction between autonomy and heteronomy according to Kant's categorical imperative and that the meaning of autonomy implies qualities technology cannot have. Being autonomous is about having the right to be wrong, a right justified by accountability and insightful understanding of real-life values, and it is about being externally uncontrollable. The contemporary use of autonomy as well as similar concepts is discussed and a model is presented showing how six sources of confusion interact in a vicious circle that impede human authority and autonomy. Our goal is to sort out these confusions and contribute to a development in which the different roles of machines and people, and human responsibilities, are explicated rather than blurred, which should facilitate the forming of truly beneficial and complementary systems.

Abstract

Practitioner Summary: The maladaptive use of autonomous and similar human concepts for technological systems is discussed in relation to the current situation for human–systems interaction. A model is presented combining six sources of confusion into a vicious circle that predict two problems that lead to reduced human authority.

Keywords::

• autonomous
• automation
• human–machine interaction
• ethics
• risk analysis

Acknowledgements

The authors are grateful to the anonymous reviewers for comments helpful in improving this paper.

Notes

^1.See http://www.ias-society.org

^2.See http://blog.scania.com/modas/

^3. ‘Finally, there is an imperative which commands a certain conduct immediately, without having as its condition any other purpose to be attained by it. This imperative is categorical. It concerns not the matter of the action, or its intended result, but its form and the principle of which it is itself a result; and what is essentially good in it consists in the mental disposition, let the consequence be what it may. This imperative may be called that of morality' (Kant 1785).

^4. This is in accordance with the encyclopedia definition of autonomy: ‘1: the quality or state of being self-governing; especially: the right of self-government, 2: self-directing freedom and especially moral independence, 3: a self-governing state' (http://www.merriam-webster.com/dictionary/autonomy).

^5. Supposedly, you buy a laptop computer primarily for its performance that is provided mainly by its machine properties, but you can always utilise its shape and weight to function as a doorstop, a fact that under extreme conditions actually might be what saves your life.

^6. For instance, http://en.wikipedia.org/wiki/AIBO

^7. For example, the Air France 447 crash (BEA 2012), a wake-up call regarding automation and simulator training (Thomas 2011).

^8. This corresponds rather well with the encyclopedia definition of automatic: ‘1 a: largely or wholly involuntary, especially: REFLEX < automatic blinking of the eyelids>, b: acting or done spontaneous or unconsciously, c: done or produced as if by machine: MECHANICAL <  the answers were automatic>, 2: having a self-acting or self-regulating mechanism < an automatic transmission>, 3 of a firearm: firing repeatedly until the trigger is released' (http://www.merriam-webster.com/dictionary/automatic), while autonomous (in footnote 4) is focusing more on rights and moral independence.