Unsavory medicine for technological civilization: Introducing ‘Artificial Intelligence & its Discontents’

KEYWORDS:

• Artificial Intelligence
• history of science
• Science & Technology Studies
• modernity
• well-being
• medicine
• critical theory
• governance

This is, once again, the Age of Artificial Intelligence (Garvey 2018c). AI, the suite of techniques intended to make machines capable of performing tasks considered ‘intelligent’ when performed by people, is an epochal technology now colonizing an increasing number of domains, from Internet search and social media to the natural and social sciences; agriculture, banking, criminal sentencing, decision-making, and beyond. AI may soon become ubiquitous, coextensive with technological civilization itself: a taken-for-granted feature of modernity like running water or electricity.

But this does not mean that all is well. While AI promises to liberate and empower users, improve well-being, support social institutions, and enable sustainable development, it also threatens to automate and entrench precarity and illiberalism, degrade mental health, and accelerate the Earth’s ecological collapse.

Freud (1961) famously observed that civilization, despite being ostensibly intended to protect humanity from misery, is paradoxically a great source of unhappiness. Similarly, AI is both touted as the solution to humanity’s biggest problems and decried as one of the biggest problems humankind has ever faced – even, perhaps, its last. A plethora of pundits posit that AI poses an existential risk to human survival on this planet: If not nuclear war, climate catastrophe, or another global pandemic, then it will be ‘superintelligent’ machines that herald the Apocalypse (Barrat 2013; Bostrom 2014; Clark 2014; Yampolskiy 2015; Müller 2016; Cava 2018; Russell 2019). Or not. Other AI advocates claim a new wave of ethical developments will usher in the ‘Good AI Society’ (Floridi et al. 2018), free from scarcity and strife, thus bringing the West, as Japanese technologist Akihito Kodama (2016) has argued, to its teleological zenith: the return to Eden – abundance without work, life without pain – albeit digitized (Hilton 1964; Noble 1999; Geraci 2010; Diamandis and Kotler 2012).

Dystopian hellscape or Edenic utopia? Surely neither of these extremes are the only possible consequence of this vast, sociotechnical system of data, people, places, and things we call ‘AI’ – but what are the alternatives, and where are they to be found when expert partisans on each side dispute both the claims and the credentials of their counterparts? Who can help society make sense of the controversial technoscience of AI? The technoscientists whose careers depend on the success of AI? The business people who employ them? The policymakers devoted to the profits promised by the AI-powered ‘Fourth Industrial Revolution’ (World Economic Forum 2016; Schwab 2017; Mak n.d.; Brynjolfsson and McAfee 2012, 2016; McAfee and Brynjolfsson 2017; Brynjolfsson and Mitchell 2017; Ford 2009, 2015; though see also Wiener 1989; Gimpel 1977; Jenkins and Sherman 1979; White 1980; Johannessen 2019)?

None other than the critics. Little sense can be made of AI without reference to its discontents – those who doubt, question, challenge, reject, reform, and otherwise reprise ‘AI’ as it is practiced, promoted, and (re)produced. With the hope of scaffolding deeper, more nuanced understandings of both the epochal transformations being wrought by AI technologies and the range of responses required, possible, and as-yet unimagined, this special issue brings together critical accounts of AI and its discontents, past and present, in order to capture the significance of this historical moment, expand the horizons of the possible, and catalyze sociotechnical action on behalf of diverse publics and future generations whose autonomy – and humanity – are at stake.

AI criticism: the tradition of discontent

Often defined tongue-in-cheek by practitioners as ‘what computers can’t do, yet’ (Hendler and Mulvehill 2016), in its relentless focus on future horizons of technical capability, AI is amnesiac – scarcely aware of its own history, much less that of its critics. This may be a consequence of the fact that AI’s history has been written primarily by insiders and developers themselves (McCorduck 1979, 2004, 2019; Crevier 1993; Brooks 1999; Boden 1996; Nilsson 2010), all of whom tell triumphant stories of progress towards the current pinnacle upon which the world now stands, with a few bumps along the road thrown in for good measure.

According to this canon, AI began in 1956 at the Dartmouth Conference (Kline 2011), and there have only been two ‘bumps’ worth noting ever since – both, coincidentally, professors of philosophy at the University of California, Berkeley: Hubert Dreyfus (1965, 1972, 1992, 2007; Dreyfus and Dreyfus 1986, 1988), whose phenomenological attack on the Platonic formalism of early AI confounded its pioneers and presciently anticipated their failures; and John Searle (1980, 1999, 2014; Searle and Kurzweil 1999; Denton et al. 2002), whose ‘Chinese Room’ thought experiment ‘badly shook the little world of [AI] by claiming and proving (so he said) that there was no such thing’ (Motzkin and Searle 1989).

In fact, the tradition of AI criticism – to which the articles of this special issue make an important and timely contribution – is older, richer, and more diverse than suggested by the internalist history of AI (Hoos 1960a, 1960b, 1978; Wiener 1960; Greenberger 1962; Michael 1962; Bureau of Labor Statistics 1963; United States Congress Senate Committee on Labor and Public Welfare 1963; Neisser 1963; Ellul 1964; Terborgh 1965; Pierce et al. 1966; Silberman 1966; Hunt 1968; Jaki 1969; Wheeler 1972; Lighthill 1973; McDermott 1976; Glenn and Feldberg 1977; Noble 1978; Mori 1981; Ornstein, Smith, and Suchman 1984, 1985; Leontief and Duchin 1986; Bloomfield 1987; Born 1987; Suchman 1987; Beusmans and Wieckert 1989; Penrose 1989; Rosenbrock 1989; Brödner 1990; Corbett, Rasmussen, and Rauner 1991; Ennals 1991; Negrotti 1991; Collins 1992; Maturana and Varela 1992; Forsythe 1993; Ford 2001; Martin 1993; Bainbridge et al. 1994; Newquist 1994; Göranzon 1995; Göranzon and Florin 1990; 1991; Hutchins 1995; Edwards 1996; Hendriks-Jansen 1996; Kling 1996; Olazaran 1996; Adam 1998). Whereas Dreyfus and Searle were lambasted by AI partisans as ignorant outsiders (though see Armstrong, Sotala, and Ó hÉigeartaigh 2014), earlier critics, such as the mathematicians Richard Bellman (1958) and Hubert’s brother Stuart (Dreyfus 1962, 2004, 2009, 2014), came from within the technical community. Perhaps the most fearsome of these discontents was Mortimer Taube (1911–1965), director of multiple research divisions of the Library of Congress, ‘who, besides being an outstanding theorist and inventor, [was] one of the most successful business practitioners of the computer-based, data-processing art’ (Solo 1963, 173; see also Smith 1993). Whereas Dreyfus, Searle, and many subsequent critics stayed squarely on theoretical terrain, Taube’s now forgotten masterpiece, Computers and Common Sense: The Myth of Thinking Machines (1961), critiqued the social irresponsibility and economic profligacy of AI as well as its flawed philosophical foundations. Decades before ‘deconstruction’ became a term of art in academe, Taube analysed the AI literature to reveal its hidden assumptions, gaps in reasoning, and antiquated worldviews. Prior to Berger and Luckmann (1966), he showed how AI pioneers used language to socially construct the reality of ‘thinking machines’ through an interlocking web of peer-citation, long before theorists of the actor–network showed this to be a fundamental aspect of technoscientific power (Callon, Law, and Rip 1986). As it would become clear shortly after Taube’s sudden death in 1965, the AI pioneers were defrauding everyone, from their military funders to their peers and the broader public, with overhyped claims about their machines. Yet Taube distinguished them from simple criminals by noting that while the creators of sophisticated literary and artistic forgeries are ostensibly aware that they are committing fraud, the creators of ‘thinking machines’ apparently believed they were actually doing science.

Dreyfus famously attacked the scientific credentials of AI by equating it to alchemy (1965). Like Darwin’s Wallace, he arrived at this conclusion independently of Taube, who had argued several years prior that AI was a ‘scientific aberration’ like astrology or physiognomy (1961, 118–128). Taube looked forward to the day when its central dogma, the doctrine of ‘Man-Machine Identity’ – that the human brain is ‘nothing but’ a machine, and therefore can be simulated by another machine (76) – would not only be rejected, but, like the eugenics of biology or the colonial origins of anthropology, disowned entirely as an embarrassment to science. Understanding technoscience as a social activity that is ultimately meaningless if not helpful to society at large, Taube decried AI pioneers for using the term ‘science’ to ‘peddle nostrums to a gullible public’ while avoiding scrutiny by ‘insisting on the pure scientific nature of their intentions’ (124). To counteract their impairing influence on society, Taube attempted to introduce the criticism of technoscience ‘as an enterprise similar in its aims to the established arts of literary, musical, art, and religious criticism,’ one that ‘views the [techno]scientific enterprise as an activity carried out by men [sic], not by demigods, nor even high priests’ (v).

Widely influential at the time of its publication, the significance of Taube’s critical project – which provides an excellent frame and sets a high bar for this special issue – was recognized by no less a technoscientist than Alvin M. Weinberg, Director of Oak Ridge National Laboratory, who observed that while the ‘arts have always taken art critics and art criticism for granted,’ technoscientists typically assume they have no need for critics:

Bad science is science that does not agree with nature; there are, in principle, objective criteria for deciding between good and bad science. But Taube's main contention is that in a field such as [AI] which deals with human artifacts (computers) and with logical, not empirical, issues, the tried-and-true criterion of agreement with experiment no longer serves to cull the bad from the good. Nor is the review of editors or fellow workers or government administrators sufficient—in Taube’s opinion all are tainted with the same poison and, being taken in by the same alleged scientific fraud, can criticize only in detail, not in principle. If the scientific activities Taube criticizes were cheap, not much harm would be done; but since computers (like so much of modern Big Science) are expensive and are supported by public money, Taube argues that it is necessary and valid to subject these activities as a whole to the kind of criticism to which art is subjected, to criticize broadly the essential validity of the enterprise rather than to argue about the details within an accepted conceptual framework. That such a course is excruciatingly difficult, if for no other reason than that science is done by specialists and broad criticism of science must of necessity be done by people who know less than the specialists, does not deter Taube; he sees his duty and he states his opinions without pulling punches. (Weinberg 1962, 310)

Weinberg concluded by stating his hope that Taube’s critique would go on to have influence far beyond the narrow confines of AI, for ‘Much of modern Big Science could be helped by a dose of such unsavory, but necessary, medicine.’

Having edited and assembled the 10 articles in this special issue, I can assure the reader that each does its part to bring the bold critical enterprise begun by Taube into the twenty-first century – and that not a single one pulls its punches. Now allow me to echo Weinberg in hoping the contribution of these discontents proves broadly influential, for much of modern technoscience – and civilization itself – could be helped by the unsavory, but necessary, medicine offered herein.

Overview of the issue

In ‘The Lamp and the Lighthouse,’ Zachary Loeb examines the career of one of AI’s most notorious discontents, computer scientist Joseph Weizenbaum (1976), the first of a handful of notable defectors (e.g. Winograd and Flores 1987; Agre 1997; Suchman 2007; Marcus and Davis 2019; Smith 2019) from what he called the ‘Artificial Intelligentsia.’ Delving into Weizenbaum’s correspondence with the historian Lewis Mumford, whose own work affords considerable cultural context on the development of AI (Mumford 1963, 1964, 1965, 1967, 1970), Loeb provides an important corrective to the official canon: Weizenbaum was not – as so often portrayed by his former colleagues and adversaries in AI – a lone wolf, howling in the wilderness. Rather, his work, and the tradition of AI discontent more generally, was part of a larger tradition of social criticism responding to the accelerating automation, computerization, and complexification of technological civilization in the twentieth century.

Depending upon how ‘AI’ is defined, however, discontent is ancient (Wiener 1964; Cohen 1966; Winner 1989; Noble 1999; Herzfeld 2002; Geraci 2010; Russell and Norvig 2010; Garfinkel and Grunspan 2018). By excavating a minor literature on ‘Artificial Stupidity,’ Michael Falk’s article extends the critical tradition of the discontented beyond well-trod tomes like Forster’s ‘The Machine Stops’ (1909), the apocryphal ‘Book of the Machines’ in Butler’s Erewhon (1872), as well as Shelley’s 1818 Frankenstein, into the seventeenth and eighteenth centuries. Inverting narratives about the potential dangers of a hypothetical ‘superintelligence,’ Falk explicates the real risks of actual machine stupidity to open up a new (old) line of inquiry: Instead of always probing whether a given machine is truly intelligent – whatever that means – we ought instead to enquire, ‘What kind of stupid is it?’

Somewhat surprisingly, there is little if any consideration of stupidity in AI. But even more surprisingly, there is scarcely any more attention paid to what would seem to be the central concern of the entire AI enterprise: intelligence. Harry Collins, a sociologist and longtime discontent who critiques AI from the standpoint of Science and Technology Studies, summarizes and distills much of his oeuvre (1989, 1990, 1992, 2018; Collins and Kusch 1998) into ‘The Science of Artificial Intelligence and its Critics.’ It provides an important tool for puncturing contemporary versions of the ‘myth of thinking machines’: a six-level scale of intelligence that clarifies what it actually means to be intelligent from a sociological perspective. In addition to combating hype by helping ordinary people distinguish fact from fiction in AI, Collins proposes that his framework could be used by experts to make AI a respectable science again – but do they have the ears to hear his ‘productive criticism’?

As Taube pointed out decades ago, hype-busting is important because the contemporary AI enterprise is not, and has never been, just a bit of harmless technoscientific experimentation (Garvey 2018a; Garvey and Maskal 2019). With its renewed geostrategic importance (Lee 2018; Scharre 2018; Comiter 2019; Garvey 2019b; Lin 2019; Mecklin 2019; NSCAI 2019; Prakash 2019; Bulletin of the Atomic Scientists 2020; Johnson 2020) and increasing presence in nearly every sector of society (Citron and Pasquale 2014; Pasquale 2015; Pasquinelli 2015; O’Neil 2016; Eubanks 2017; Tegmark 2017; Shoham et al. 2017, 2018; Wachter-Boettcher 2017; Broussard 2018; Foer 2018; S. U. Noble 2018; Taplin 2018; Susskind 2018; Vaidhyanathan 2018; Zuboff 2018; Atanasoski and Vora 2019; Benjamin 2019; Frey 2019; Perrault et al. 2019; Topol 2019; Webb 2019; Nourbakhsh and Keating 2020), enormous national budgets have been and are being planned on the promise that more AI will be ‘good’ for society (Webster et al. 2017; Dutton, Barron, and Boskovic 2018; The White House 2018; European Commission and Joint Research Centre 2018; Schmidt et al. 2020). Yet as Ulnicane and colleagues demonstrate in ‘Good Governance as a Response to Discontents? Déjà vu, or Lessons for AI from other Emerging Technologies,’ the ‘AI for Good’ narrative at the centre of these (inter)national initiatives is ahistorical, reproducing the amnesia of the AI canon by ignoring and resisting multiple relevant precedents in the governance of emerging technologies, such as public engagement and responsible innovation (Jasanoff 1996; Rowe and Frewer 2005; Wynne 2006; Stilgoe, Lock, and Wilsdon 2014; Özdemir and Springer 2018; Garvey 2019a).

Cheryl Holzmeyer’s ‘Beyond ‘AI for Social Good’ (AI4SG): Social Transformations – Not Tech-Fixes – for Health Equity,’ the fifth article in our special issue, similarly shows that these initiatives, however well-intentioned, breed discontent by distracting from root causes of social inequity and the meliorative potential of challenging existing systems of power. While advocates claim AI will improve public health, perhaps by making it possible to predict ‘well-being’ at the population level (Jaidka et al. 2020) or rapidly screen women for breast cancer (McKinney et al. 2020), all sociotechnical failures aside (Herper 2017; Ross 2018; Strickland 2019), the technical community’s narrow focus on potential downstream interventions contributes to neglect of the social determinants of health upstream, such as adequate income and housing, which in turn amplifies inequality and puts more of society at risk. In other words, although AI is supposed to be a powerful tool for improving well-being, by diverting attention and resources away from fundamentals into expensive, experimental, expert-driven technical systems, it is paradoxically one of the greatest potential sources of social harm.

The next two articles expand upon the medical theme by exploring the role of AI in the clinic. In ‘Don’t Touch My Stuff: Historicizing Resistance to AI and Algorithmic Computer Technologies in Medicine,’ Ariane Hanemaayer documents the long tradition of doctors’ discontent as a struggle with and against machines over the authorship of medical truth. Rather than continuing to contest AI unaided, Hanemaayer suggests that physicians might find allies in their patients, with whom they share a partisan interest in reducing and protecting against the biases and other potential harms of medical AI systems (Coiera 1996; Cabitza, Rasoini, and Gensini 2017; Char, Shah, and Magnus 2018; Garvey 2018b; Krittanawong 2018). Saheli Datta Burton, Tara Mahfoud, and colleagues’ ‘Clinical Translation of Computational Brain Models: Understanding the Salience of Trust in Clinician-Researcher Relationships’ explores the physician’s predicament from another angle – the quintessential social bond of trust (Yamagishi 2011; Adali 2013). Drawing on their extensive experience with the Human Brain Project and interviews with experts, they show that without gaining clinicians’ trust through upstream collaboration that builds upon practitioners’ tacit knowledge, medical AI systems are unlikely to make meaningful contributions to patients’ health, even if they are adopted into the clinic.

The last three articles expand the frame of discontent to include the issues of biology, humanity, and identity. In ‘Truth from the Machine: Artificial Intelligence and the Materialisation of Identity,’ Keyes, Hitzig, and Blell explore how the natural sciences change when investigators utilize the quantitative techniques of AI to ‘discover’ qualitative social constructs such as ‘disease’ and ‘sexuality.’ Rachel Adams, in asking ‘Can Artificial Intelligence Be Decolonised?,’ authoritatively unpacks the intertwined legacies of colonialism, racism, and Western cultural hegemony that underlie the conceptual foundations of AI, in order to establish an erudite theorization of what the strategy of decoloniality must mean for the field if it is not to be (re)appropriated as yet another ‘ethic.’ And finally, Alan Blackwell offers a personal account of his own discontent as a longtime practitioner that builds into a proposal for a future ethnography that would make it possible to think and conduct AI otherwise.

The future of AI and its discontents

Some discontents come from outside the field, others from within. Most if not all of them, however, critique AI in order to address larger issues of continued cultural relevance, such as the nature of ‘intelligence’; the development, implementation, and governance of large-scale sociotechnical systems (Garvey 2018d); the consequences of doing AI within the military–industrial–university complex; the problems of mind, brain, and consciousness in a material universe; the relationship between language, thought, and society; as well as what it means to be ‘human’ in an increasingly computerized world.

These criticisms from the discontented reveal the deceptively simple two-letter moniker ‘AI’ to be a microcosm of technological civilization in dire need of strong medicine. While no single dose is strong enough to serve as an antidote to the cyclical malaise of modern machine-driven madness (Garvey 2018c), as AI grows more pervasive, its discontents will grow more numerous, and their critical prescriptions, however unsavory, ever more important to heed.

Acknowledgments

Shunryu Colin Garvey would like to thank the editor of ISR, Willard McCarty, for his patience, wisdom, and support on this special issue; Alan Blackwell for reading an early draft; Ariane Hanemaayer for her close collaboration and continued dedication to the project of ‘AI & its Discontents’; and all the contributors to this issue, as well as AI discontents past, present, and future.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Shunryu Colin Garvey

Dr. Shunryu Colin Garvey is trained in the interdisciplinary field of Science & Technology Studies (STS). He uses AI as a case study to probe decision making under the conditions of complexity, uncertainty, and disagreement, in order understand how societies can more safely, fairly, and wisely govern controversial sociotechnical systems.