How Artificial Intelligence Systems Challenge the Conceptual Foundations of the Human Rights Legal Framework

ABSTRACT

Few recent developments have captured the human imagination as much as those within the field of artificial intelligence (AI). The increasing pervasiveness and ubiquity of AI affect both individual lives and society at large. Yet the human rights concerns raised in connection to AI have primarily concentrated around infringements of enumerated discrete rights, such as to privacy, non-discrimination, and freedom of expression and information. While it is important to examine the discrete rights under threat, a fundamental disconnect is present at the foundational level between human rights and AI, because AI systems increasingly challenge the notions of how, by whom, and what human rights violations are being committed. This paper takes a problem-finding perspective and argues that the conceptual foundations of the human rights protection framework are facing a serious challenge. The legal-positivist framing of the discrete rights themselves is examined through an analysis of the misaligned harm typology between the objects of human rights protection and the threats posed by AI systems. The paper highlights three main misalignments, from the perspective of saliency, temporality, and causality of harms, and argues that these misalignments challenge the structural enabling conditions for the exercise and meaningful protection of human rights.

KEYWORDS:

• Human rights
• human rights foundations
• artificial intelligence
• conceptual foundations
• ethics

Disclosure Statement

No potential conflict of interest was reported by the author.

Notes

1 This paper adopts the position of the increasing ubiquity of AI on account of it being considered as a ‘general purpose technology’ that can be applied across a wide range of domains. See Jade Leung, ‘Who Will Govern Artificial Intelligence? Learning from the History of Strategic Politics in Emerging Technologies’ (http://purl.org/dc/dcmitype/Text, University of Oxford 2019) <https://ora.ox.ac.uk/objects/uuid:ea3c7cb8-2464-45f1-a47c-c7b568f27665>; Analogous comparisons of AI being a general purpose technology can be made to steam power and electricity. See Hin-Yan Liu and Matthijs M Maas, ‘“Solving for X?” Towards a Problem-Finding Framework to Ground Long-Term Governance Strategies for Artificial Intelligence’ (2021) 126 Futures 102672; The first use of the words general purpose technology can be traced to the work of Timothy F Bresnahan and M Trajtenberg, ‘General Purpose Technologies “Engines of Growth”?’ (1995) 65 Journal of Econometrics 83.

2 Sara Brown, ‘Machine Learning, Explained’ (MIT Sloan, 21 April 2021) <https://mitsloan.mit.edu/ideas-made-to-matter/machine-learning-explained> accessed 18 October 2021.

3 See among others Lorna McGregor, Daragh Murray and Vivian Ng, ‘International Human Rights Law as a Framework for Algorithmic Accountability’ (2019) 68 International & Comparative Law Quarterly 309; Filippo A Raso and others, ‘Artificial Intelligence & Human Rights: Opportunities & Risks’ (Berkman Klein Center for Internet and Society at Harvard University 2018) <https://cyber.harvard.edu/publication/2018/artificial-intelligence-human-rights>; Eileen Donahoe and Megan MacDuffee Metzger, ‘Artificial Intelligence and Human Rights’ (2019) 30 Journal of Democracy 115.

4 On the importance of taking a problem-finding perspective for complex emerging problems such as artificial intelligence, see Liu and Maas (n 1); Liu makes a similar point, observing that the human rights framework is too ‘brittle’ when faced with AI challenges: see Hin-Yan Liu, ‘AI Challenges and the Inadequacy of Human Rights Protections’ (2021) 40 Criminal Justice Ethics 2.

5 Karen Yeung, ‘Keynote Speech, IACL Conference: Constitutional Principles in a Networked Digital Society’ (Social Science Research Network 2022) SSRN Scholarly Paper ID 4049141 <https://papers.ssrn.com/abstract=4049141> (‘ …  clear that live FRT is far more powerful than DNA or fingerprinting biometric identification technologies, due to a number of technological properties which enables the police to engage in an entirely novel and extraordinarily powerful new practice: that is, to engage in mass surveillance of populations as they go about their lawful activities in public, at scale, on a continuous basis, for the purposes of identifying multiple “persons of interest” in real time. But by failing to properly grasp the true novelty and power of AFR Locate, the court’s evaluation of the legal arguments was necessarily inadequate.’).

6 See the guiding principles of the GDPR, Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) 2016 (OJ L).

7 There is a plethora of work that charts the human rights impacts of AI systems. See e.g. Cathy O’Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy (Crown 2016); Virginia Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St Martin’s Press 2017); Rinie van Est and Joost Gerritsen, ‘Human Rights in the Robot Age: Challenges Arising From the use of Robotics, Artificial Intelligence, and Virtual and Augmented Reality’ (Rathenau Instituut 2017) <www.rathenau.nl/en/digitale-samenleving/human-rights-robot-age>; McGregor, Murray and Ng (n 3); ‘Getting the Future Right: Artificial Intelligence and Fundamental Rights’ (European Union Agency for Fundamental Rights, 18 November 2020) <https://fra.europa.eu/en/publication/2020/artificial-intelligence-and-fundamental-rights>; Mireille Hildebrandt, Smart Technologies and the End(s) of Law, Novel Entanglements of Law and Technology (Edward Elgar Publishing 2015) ch 5.

8 Molly K Land, ‘Regulating Private Harms Online: Content Regulation under Human Rights Law’, Human Rights in the Age of Platforms (MIT Press 2019) <https://direct.mit.edu/books/book/4531/chapter/202549/Regulating-Private-Harms-Online-Content-Regulation>; see also the report by the Special Rapporteur on the promotion and protection of the right to freedom of opinion and expression, ‘Report of the Special Rapporteur to the General Assembly on AI and Its Impact on Freedom of Opinion and Expression’ (Office of the High Commissioner for Human Rights (OHCHR) 2018) A/73/348 <www.ohchr.org/EN/Issues/FreedomOpinion/Pages/ReportGA73.aspx> accessed 4 February 2022.

9 Donahoe and Metzger (n 3).

10 McGregor, Murray and Ng (n 3).

11 Christiaan van Veen, ‘Artificial Intelligence: What’s Human Rights Got To Do With It?’ (Medium, 18 May 2018) <https://points.datasociety.net/artificial-intelligence-whats-human-rights-got-to-do-with-it-4622ec1566d5>; Alessandro Mantelero, ‘Regulating AI within the Human Rights Framework: A Roadmapping Methodology’ [2020] European Yearbook on Human Rights 477.

12 See however Molly K Land and Jay D Aronson, ‘The Promise and Peril of Human Rights Technology’ in Jay D Aronson and Molly K Land (eds), New Technologies for Human Rights Law and Practice (Cambridge University Press 2018); Hin-Yan Liu, ‘The Digital Disruption of Human Rights Foundations’ in Mart Susi (ed), Human Rights, Digital Society and the Law: A Research Companion (Routledge 2019).

13 Karen Yeung, ‘“Hypernudge”: Big Data as a Mode of Regulation by Design’ (2017) 20 Information, Communication & Society 118.

14 Daniel Susser, Helen Nissenbaum and Beate Roessler, ‘Online Manipulation: Hidden Influences in a Digital World’ (2020) 4 Georgetown Law Technology Review <https://georgetownlawtechreview.org/online-manipulation-hidden-influences-in-a-digital-world/GLTR-01-2020/>.

15 Roger Brownsword, ‘Technological Management and the Rule of Law’ (2016) 8 Law, Innovation and Technology 100; Roger Brownsword, ‘From Erewhon to AlphaGo: For the Sake of Human Dignity, Should We Destroy the Machines?’ (2017) 9 Law, Innovation and Technology 117; Roger Brownsword, ‘Law Disrupted, Law Re-Imagined, Law Re-Invented’ [2019] Technology and Regulation 10.

16 Hildebrandt, Smart Technologies and the End(s) of Law (n 7); Julie E Cohen, ‘Affording Fundamental Rights: A Provocation Inspired by Mireille Hildebrandt’ (2017) 4 Critical Analysis of Law <https://cal.library.utoronto.ca/index.php/cal/article/view/28151>.

17 Evan Selinger and Hyo Joo (Judy) Rhee, ‘Normalizing Surveillance’ (2021) 22 SATS: Northern European Journal of Philosophy 49.

18 Cohen (n 16) 78.

19 See Jacqui Ayling and Adriane Chapman, ‘Putting AI Ethics to Work: Are the Tools Fit for Purpose?’ [2021] AI and Ethics <https://doi.org/10.1007/s43681-021-00084-x>; see also AlgorithmWatch, ‘AI Ethics Guidelines Global Inventory’ <https://algorithmwatch.org/en/project/ai-ethics-guidelines-global-inventory/>; Anna Jobin, Marcello Ienca and Effy Vayena, ‘Artificial Intelligence: The Global Landscape of Ethics Guidelines’ arxiv.org <https://arxiv.org/abs/1906.11668> accessed 15 January 2020.

20 See also John Tasioulas, ‘Saving Human Rights from Human Rights Law’ (2019) 52 Vanderbilt Journal of Transnational Law 1167.

21 Amartya Sen, ‘Elements of a Theory of Human Rights’ (2004) 32 Philosophy & Public Affairs 315, 345.

22 Tasioulas (n 20) 1175.

23 Tasioulas (n 20); Sen (n 21) 345; see however the expanding scope of Article 8 ECHR: Registry, European Court of Human Rights, ‘Guide on Article 8 of the European Convention on Human Rights’ 7–10 <www.echr.coe.int/documents/guide_art_8_eng.pdf> accessed 17 May 2022.

24 For the nature of state obligations in relation to economic, social and cultural rights, see ‘CESCR General Comment No 3: The Nature of States Parties’ Obligations (Art 2, Para 1, of the Covenant) Adopted at the Fifth Session of the Committee on Economic, Social and Cultural Rights, on 14 December 1990’ Document E/1991/23.

25 Samuel Moyn, Not Enough: Human Rights in an Unequal World (Belknap Press: 2018).

26 See Philip Alston, ‘Report of the Special Rapporteur on Extreme Poverty and Human Rights’ (Office of the High Commissioner for Human Rights (OHCHR) 2015) A/HRC/29/31 <https://digitallibrary.un.org/record/798707>.

27 On the need for ‘upstream’ measures, see Shoshana Zuboff, ‘The Coup We Are Not Talking About’ The New York Times (New York, 29 January 2021) <www.nytimes.com/2021/01/29/opinion/sunday/facebook-surveillance-society-technology.html>.

28 Lea Shaver, ‘Safeguarding Human Rights from Problematic Technologies’ in Jay D Aronson and Molly K Land (eds), New Technologies for Human Rights Law and Practice (Cambridge University Press 2018).

29 See the criticisms mounted by David Kennedy, ‘International Human Rights Movement: Part of the Problem? Boundaries in the Field of Human Rights’ (2002) 15 Harvard Human Rights Journal 101.

30 Lyria Bennett Moses, ‘Regulating in the Face of Sociotechnical Change’ in Roger Brownsword, Eloise Scotford and Karen Yeung (eds), The Oxford Handbook of Law, Regulation and Technology (Oxford University Press 2017); Lyria Bennett Moses, ‘Recurring Dilemmas: The Law’s Race to Keep Up With Technological Change’ (2007) 21 University of New South Wales Law Research Series <http://www.austlii.edu.au/au/journals/UNSWLRS/2007/21.html> accessed 17 February 2020; see also Lyria Bennett Moses, ‘Agents of Change: How the Law “Copes” with Technological Change’ (2011) 20 Griffith Law Review 763.

31 David Collingridge, The Social Control of Technology (St Martin’s Press 1980).

32 Moses, ‘Regulating in the Face of Sociotechnical Change’ (n 30).

33 Mireille Hildebrandt, Law for Computer Scientists and Other Folk (Oxford University Press 2020).

34 James Gibson first used the concept to denote ‘affordance’ as ‘ …  the affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill’ within the field of psychology. See James J Gibson, The Ecological Approach to Visual Perception (Houghton Mifflin 1979) 127; Norman subsequently used ‘affordances’ to mean the visible properties of the design of everyday objects. See Donald A Norman, The Design of Everyday Things (Rev and expanded edn, Basic Books 2013).

35 Mireille Hildebrandt, ‘Chapter 11. Closure: On Ethics, Code and Law’ in Law for Computer Scientists (PubPub 2019).

36 See Daniel Moeckli, ‘Interpretation of the ICESCR: Between Morality and State Consent’ in The Human Rights Covenants at 50 (Oxford University Press 2018). I am grateful for the anonymous reviewer who highlighted this point.

37 Joel Feinberg, The Moral Limits of the Criminal Law Volume 1: Harm to Others (Oxford University Press 1987) 34.

38 Liu, ‘The Digital Disruption’ (n 12); Liu, ‘AI Challenges’ (n 4); see also Alan M Dershowitz, Rights from Wrongs (Basic Books 2004).

39 David McGrogan, ‘The Problem of Causality in International Human Rights Law’ (2016) 65 International & Comparative Law Quarterly 615; AnnJanette Rosga and Margaret L Satterthwaie (eds), ‘The Trust in Indicators: Measuring Human Rights’ [2009] Berkeley Journal of International Law.

40 Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information (Harvard University Press 2015).

41 Liu, ‘The Digital Disruption’ (n 12) 76.

42 Alex Hern, ‘Google’s Solution to Accidental Algorithmic Racism: Ban Gorillas’ The Guardian (London, 12 January 2018) <www.theguardian.com/technology/2018/jan/12/google-racism-ban-gorilla-black-people>; Liu, ‘The Digital Disruption’ (n 12) 79.

43 ‘Facebook Apology as AI Labels Black Men “Primates”’ (BBC News, 6 September 2021) <www.bbc.com/news/technology-58462511>.

44 Liu, ‘The Digital Disruption’ (n 12) 77–79.

45 David Lehr and Paul Ohm, ‘Playing with the Data: What Legal Scholars Should Learn About Machine Learning’ (2017) 51 UC Davis Law Review 653.

46 Elle Hunt, ‘Tay, Microsoft’s AI Chatbot, Gets a Crash Course in Racism from Twitter’ The Guardian (London, 24 March 2016) <www.theguardian.com/technology/2016/mar/24/tay-microsofts-ai-chatbot-gets-a-crash-course-in-racism-from-twitter>; Luciano Floridi, ‘True AI Is Both Logically Possible and Utterly Implausible’ (Aeon Essays) <https://aeon.co/essays/true-ai-is-both-logically-possible-and-utterly-implausible>.

47 Niemietz v Germany, Court (Chamber) 1993 16 EHRR 97 [29].

48 Julie E Cohen, ‘What Privacy Is For’ (2013) 126 Harvard Law Review 1904.

49 Oscar H Gandy, ‘Engaging Rational Discrimination: Exploring Reasons for Placing Regulatory Constraints on Decision Support Systems’ (2010) 12 Ethics and Information Technology 29, 29.

50 Geoffrey C Bowker and Susan Leigh Star, Sorting Things out: Classification and Its Consequences (MIT Press 2008); Cynthia Dwork and Deirdre K Mulligan, ‘It’s Not Privacy, and It’s Not Fair’ (2013) 66 Stanford Law Review Online <www.stanfordlawreview.org/online/privacy-and-big-data-its-not-privacy-and-its-not-fair/>; Personal data is here taken to mean ‘ …  any information relating to an identified or identifiable natural person (“data subject”)’. See Regulation (EU) 2016/679, art 4, of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation).

51 Sandra Wachter and Brent Mittelstadt, ‘A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI Survey: Privacy, Data, and Business’ (2019) 2019 Columbia Business Law Review 494, 502–14.

52 Ibid. 515–42.

53 Solon Barocas and Andrew D Selbst, ‘Big Data’s Disparate Impact’ (2016) 104 California Law Review 671.

54 Omer Tene and Jules Polonetsky, ‘Big Data for All: Privacy and User Control in the Age of Analytics’ (2013) 11 Northwestern Journal of Technology and Intellectual Property 239 (emphasis in original).

55 Janneke Gerards and Frederik Zuiderveen Borgesius, ‘Protected Grounds and the System of Non-Discrimination Law in the Context of Algorithmic Decision-Making and Artificial Intelligence’ Colorado Technology Law Journal (forthcoming 2021) <http://works.bepress.com/frederik-zuiderveenborgesius/65/>; see also Frederik J Zuiderveen Borgesius, ‘Strengthening Legal Protection against Discrimination by Algorithms and Artificial Intelligence’ (2020) 24 The International Journal of Human Rights 1572.

56 Kate Crawford, Keynote Speech: The Trouble with Bias (Neural Information Processing System Conference (NIPS) 2017 Keynote 2017) <www.youtube.com/watch?v=fMym_BKWQzk>.

57 Barbara Ortutay, ‘Facebook Shuts out NYU Academics’ Research on Political Ads’ (AP NEWS, 5 August 2021) <https://apnews.com/article/technology-business-5d3021ed9f193bf249c3af158b128d18> accessed 7 October 2021.

58 Linnet Taylor, ‘Safety in Numbers? Group Privacy and Big Data Analytics in the Developing World’ in Bart van der Sloot, Luciano Floridi and Linnet Taylor (eds), Group Privacy, vol 126 (Springer International 2017).

59 Vladislava Stoyanova, ‘Causation between State Omission and Harm within the Framework of Positive Obligations under the European Convention on Human Rights’ (2018) 18 Human Rights Law Review 309.

60 Carissa Véliz, Privacy Is Power: Why and How You Should Take Back Control of Your Data (Bantam Press 2020) 76–77.

61 Tamara Khandaker, ‘Canada Is Using Ancestry DNA Websites to Help It Deport People’ (Vice, 28 June 2018) <www.vice.com/en/article/wjkxmy/canada-is-using-ancestry-dna-websites-to-help-it-deport-people>.

62 See also Eubanks (n 7).

63 Mark Latonero, ‘Stop Surveillance Humanitarianism’ The New York Times (New York, 11 July 2019) <www.nytimes.com/2019/07/11/opinion/data-humanitarian-aid.html>.

64 See e.g. Eileen Guo and Hikmat Noori, ‘This Is the Real Story of the Afghan Biometric Databases Abandoned to the Taliban’ (MIT Technology Review, 30 August 2021) <www.technologyreview.com/2021/08/30/1033941/afghanistan-biometric-databases-us-military-40-data-points/>.

65 Kennedy (n 29).

66 Wachter and Mittelstadt (n 51).

67 Silla Brush, Tom Schoenberg and Suzi Ring, ‘How a Mystery Trader With An Algorithm May Have Caused the Flash Crash’ (Bloomberg.com, 21 April 2015) <www.bloomberg.com/news/articles/2015-04-22/mystery-trader-armed-with-algorithms-rewrites-flash-crash-story>.

68 Buttonwood, ‘Flash and the firestorm’, (The Economist, 15 October 2016) <www.economist.com/buttonwoods-notebook/2016/10/07/sterling-takes-a-pounding>; ‘Algorithms Probably Caused a Flash Crash of the British Pound’ (MIT Technology Review) <https://www.technologyreview.com/2016/10/07/244656/algorithms-probably-caused-a-flash-crash-of-the-british-pound/>; see however ‘No Single Factor behind Sterling Flash Crash, BIS Says’ (BBC News, 13 January 2017) <www.bbc.com/news/business-38607480>.

69 Ruha Benjamin, ‘The New Jim Code? Race, Carceral Technoscience, and Liberatory Imagination’: 400 Years of Resistance to Slavery and Injustice <https://400years.berkeley.edu/videos/newjimcode> accessed 17 May 2022.

70 Will Douglas Heaven, ‘Predictive Policing Algorithms Are Racist: They Need to Be Dismantled’ (MIT Technology Review) <www.technologyreview.com/2020/07/17/1005396/predictive-policing-algorithms-racist-dismantled-machine-learning-bias-criminal-justice/> accessed 17 July 2020; see also the RUSI report commissioned by the UK government’s Centre for Data Ethics and Innovation Alexander Babuta and Marion Oswald, ‘Data Analytics and Algorithmic Bias in Policing’ (Royal United Services Institute for Defence and Security Studies) Briefing Paper <www.gov.uk/government/publications/report-commissioned-by-cdei-calls-for-measures-to-address-bias-in-police-use-of-data-analytics> accessed 16 September 2019.

71 McGrogan (n 39).

72 Karen Yeung, ‘Responsibility and AI: A Study of the Implications of Advanced Digital Technologies (Including AI Systems) for the Concept of Responsibility within a Human Rights Framework’ s 3.4.3 <https://rm.coe.int/responsability-and-ai-en/168097d9c5> accessed 30 October 2019.

73 Stoyanova (n 59); see also the procedural requirements under arts 34 and 35 of the European Convention on Human Rights 1950; Kent Roach, ‘Damages’ in Remedies for Human Rights Violations: A Two-Track Approach to Supra-national and National Law (Cambridge University Press 2021).

74 Hin-Yan Liu, ‘The Power Structure of Artificial Intelligence’ (2018) 10 Law, Innovation and Technology 197.

75 ‘Guiding Principles on Business and Human Rights, UN Doc A/HRC/17/31’ <www.ohchr.org/documents/publications/guidingprinciplesbusinesshr_en.pdf> (emphasis added).

76 See ibid. commentary to Guiding Principle 13 (emphasis added).

77 Matthew U Scherer, ‘Regulating Artificial Intelligence Systems: Risks, Challenges, Competencies, and Strategies’ (2016) 29 Harvard Journal of Law & Technology 353.

78 Liu, ‘The Power Structure’ (n 74) 213.

79 van de Poel IR and others, ‘The Problem of Many Hands : Climate Change as an Example’ (2012) 18 Science and Engineering Ethics 49.

80 Andreas Matthias, ‘The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata’ (2004) 6 Ethics and Information Technology 175.

81 Wachter and others distinguish between the separate but related concepts of technical bias and societal bias. Bias issues within machine learning systems typically engage the interplay between these two types of biases. See Sandra Wachter, Brent Mittelstadt and Chris Russell, ‘Bias Preservation in Machine Learning: The Legality of Fairness Metrics Under EU Non-Discrimination Law’ (2021) 123 West Virginia Law Review 735, 742.

82 Liu, ‘The Digital Disruption’ (n 12).

83 Barocas and Selbst (n 53); Scherer (n 77).

84 Liu, ‘The Digital Disruption’ (n 12) 80.

85 Ziad Obermeyer and others, ‘Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations’ (2019) 366 Science 447.

86 Wachter, Mittelstadt and Russell (n 81).

87 Borgesius (n 55).

88 Such practices are especially prevalent in online shopping whereby prices are determined based upon individual profiles. See Tal Zarsky, ‘“Mine Your Own Business!”: Making the Case For the Implications of the Data Mining of Personal Information in the Forum of Public Opinion’ (2003) 5 Yale Journal of Law and Technology 29.; see also Sandra Wachter, ‘Affinity Profiling and Discrimination By Association in Online Behavioral Advertising’ (2021) 35 Berkeley Technology Law Journal 367.

89 This idea is similarly captured by the notion of ‘significant disadvantage’ under European Convention on Human Rights, art 35(3)(b).

90 Liu, ‘AI Challenges’ (n 4).

91 European Center for Not-for-Profit Law (ECNL), ‘Collective Power for Rights-Based and Just AI: Going beyond the AI Buzzword’ (ecnl.org, 29 November 2021) <https://ecnl.org/news/collective-power-rights-based-and-just-ai-going-beyond-ai-buzzword> accessed 29 March 2022 (in which disability rights activist Mia Ahlgren observes that ‘ …  often you are not even aware of the concrete harm done by AI. For example if a person with disability’s job application is rejected in 1st phase of selection, how will that person know that the rejection is based on an algorithm and related to disability?’).

92 Sandra Wachter, Brent Mittelstadt and Chris Russell, ‘Why Fairness Cannot Be Automated: Bridging the Gap between EU Non-Discrimination Law and AI’ (2021) 41 Computer Law & Security Review 105567, 6.