Building truths in AI: Making predictive algorithms doable in healthcare

ABSTRACT

Increasingly, artificial intelligent (AI) algorithms are being applied to automatically assist or automate decisions. Such statistical models have been criticized in the existing literature especially for producing cultural biases and for challenging our notions of knowledge. However, few studies have contributed to an essential understanding of the way in which algorithms are designed with particular truths to enable systematic decision-making. Drawing on an ethnographic study in a Scandinavian AI company, this article analyzes how truth is built through layered interpretative practices in applied AI for healthcare, and critically assesses how such practices shed light on the pragmatic notion of truth(s) in AI. The study identifies five practices that all show difficulty in modeling fuzzy patient conditions into one firm truth. The key contribution is that truth goes from being a process of discovering a more ‘right’ truth to become a process of reinventing the existing truth and healthcare practice. These findings suggest that truth in applied AI is a key devise for making predictive algorithms a viable business, and that developers are in a favorable position to make not only AI doable but also the very truth they intend to find and model. The study in this way shows how change is an inherent part of making AI systems, and that centralizing truth practices is a fruitful way of analyzing such changes and developers’ agency. We argue for analytical awareness of how AI truth practices may prompt a world that is fit to algorithms rather than a world to which algorithms are fit.

KEYWORDS:

• Artificial intelligence
• machine learning
• automation
• ground truth
• knowledge production
• algorithm studies

Acknowledgements

We would like to thank the R&D project steering group and its research database steering group, the head of the company, employees at the company for allowing participant observation and for participating in interviews. Also, we are grateful for constructive comments from anonymous reviewers that helped improve the article.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes on contributors

Anne Henriksen is a PhD fellow at the School of Communication and Culture, Aarhus University, Denmark, and a member of DATALAB – Center for Digital Social Research. Her research focuses on the design and introduction of AI predictive analytics and automation systems into knowledge-intensive practices and industries and has been presented at several international conferences [email: [email protected]].

Anja Bechmann is a professor of media studies and the director of DATALAB – Center for Digital Social Research, Aarhus University, Denmark. She is also a guest professor at the Political Science University of Antwerp, Belgium, the director of EU REMID Center of Excellence for Research in Social Media and Information Disorders, and a former member of the EU Commission HLEG on Disinformation. She was thinker in residence at the Royal Flemish Academy of Belgium for Science and the Arts in 2019. Her research focuses on AI, social media, and collective behavior, and her work has been published in journals such as New Media & Society, The Information Society, and Big Data & Society [email: [email protected]].

ORCID

Anne Henriksen http://orcid.org/0000-0003-3317-5424

Anja Bechmann http://orcid.org/0000-0002-5588-5155

Notes

1 We distinguish between models and algorithms, with models being the mathematical representations and formulas used for producing algorithms, which are understood as the programs used for processing data when employed in a system. We focus particularly on deep learning models including artificial neural networks, as this is the branch of ML models that is primarily used in the study (Russell & Norvig, 2018).

Additional information

Funding

Funding for the research has been granted from Aarhus University and Aarhus University Research Foundation grant number AUFF-E-2015-FLS-8-55.