Ethical, political and epistemic implications of machine learning (mis)information classification: insights from an interdisciplinary collaboration between social and data scientists

ABSTRACT

Machine learning (ML) classification models are becoming increasingly popular for tackling the sheer volume and speed of online misinformation. In building these models data scientists need to make assumptions about the legitimacy and authoritativeness of the sources of ‘truth’ employed for model training and testing. This has political, ethical and epistemic implications which are rarely addressed in technical papers. Despite (and due to) their reported high performance, ML-driven moderation systems have the potential to shape public debate and create downstream negative impacts. This article presents findings from a responsible innovation (RI) inflected collaboration between science and technology studies scholars and data scientists. Following an interactive co-ethnographic process, we identify a series of algorithmic contingencies—key moments during ML model development which could lead to different future outcomes, uncertainties and harmful effects. We conclude by offering recommendations on how to address the potential failures of ML tools for combating online misinformation.

KEYWORDS:

• misinformation
• reflexivity
• content moderation
• fact-checking
• machine learning
• responsible innovation collaboration

Disclosure statement

No potential conflict of interest was reported by the author(s).

Ethics declaration

This study was designed as an internal research project and approved by the ethics committee of the authors’ institution following fair data management practices, informed consent and responsible research and innovation considerations.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1 While several terms related to misinformation (e.g., disinformation and fake news) are used throughout this paper, the term misinformation, in its broadest sense, is preferred for analysis as it encompasses any type of misleading or false content presented as factual, regardless of intent.

2 Machine learning algorithms allow computers to ‘learn’ based on examples derived from data. This process demands human labour—in what is known as supervised ML—to develop training and testing datasets containing pieces of information typically annotated by humans.

3 A non-exhaustive list of datasets is found on D’Ulizia et al. (2021)

4 A database of global fact-checking websites has identified more than 300 (Reporter’s Lab 2022)

5 The National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online REPHRAIN.

6 We note that some of the technical papers reviewed here are at the time of writing still in preprint form, which is instructive of the faster pace of AI development compared with the academic peer review and publication cycles.

7 This service aggregates claims which have been fact-checked by eligible news organisations. To be included in Google’s fact check tool, news organisations need to comply with Google’s standards for publishers and content policies (Google 2023).

8 An illustration of this is what Rietdijk and Archer (2021) problematise as ‘false balance’ in journalism. This issue has been particularly salient in the debate around climate change, where journalists have given disproportionate attention to a minority of climate sceptics within the scientific community, who may also qualify as experts, in their efforts to show both sides of the debate.

9 For example, PolitiFact, a well-known fact-checker organisation, decided to archive their original assessment on the COVID lab leak controversy by removing it from their database and revising their assessment as ‘widely disputed’ (see PolitiFact 2021)

10 Some fact-checking organisations focus exclusively on false and misleading claims (e.g., factcheck.org)

11 Here accuracy is the proportion of the model’s predictions which are correct, recall is the proportion of the positive samples which the model correctly predicted, precision is the proportion of the model’s positive predictions which are correct. The F1-score is the harmonic mean of the recall and precision, which implies that if one of these two metrics are low then the F1-score will be correspondingly low as well.

12 The concept of language performativity is used here in the same sense as within language anthropology, gender studies and sociology of expectations. A claim or statement is thought of as performative insofar as it constitutes and act which has an effect in the world (see Borup et al. 2006; Hall 1999).

13 This recursive feedback loop involving algorithms and humans influencing one another introduces yet further contingencies, however we do not have space here to explore these in detail.

14 As admitted by YouTube’s representative: ‘One of the decisions we made [at the beginning of the pandemic] when it came to machines who couldn’t be as precise as humans, we were going to err on the side of making sure that our users were protected, even though that might have resulted in a slightly higher number of videos coming down.’ (Neal Mohan quoted in Barker and Murphy 2020)

Additional information

Funding

This research was supported by REPHRAIN: The National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online, under UKRI grant: EP/V011189/1.

Notes on contributors

Andrés Domínguez Hernández

Andrés Domínguez Hernández is Senior Research Associate at the University of Bristol, United Kingdom. He has a background in science and technology studies, technology and innovation management, and electronics engineering. He has a longstanding interest in interdisciplinary collaborations across engineering, computer science, design and the social sciences. He has taken part in international projects investigating responsibility, justice and ethics with emerging digital technologies. Prior to academia, he led innovation and technology transfer policy and the implementation of large-scale telecommunications infrastructure.

Richard Owen

Richard Owen is a Professor of Innovation Management in the School of Management, Faculty of Social Sciences, University of Bristol, U.K. He is interested in the power of innovation and techno-visionary science to create futures in profound and uncertain ways, how we can engage as a society with those futures and how we can take responsibility for them. He is interested in the politics, risks, ethics and governance of innovation and new technologies in society. His research sits at the intersection of innovation governance and science and technology studies as a critical, interdisciplinary scholar.

Dan Saattrup Nielsen

Dan Saattrup Nielsen is a postdoctoral researcher in machine learning at the University of Bristol, where he also was awarded his PhD in Mathematics. Previously, he has been working with graph machine learning for fraud detection at the Danish Business Authority. His research interests include graph machine learning and natural language processing for low-resource languages

Ryan McConville

Ryan McConville was appointed a Lecturer in Data Science, Machine Learning and AI within the Intelligent Systems Laboratory and Department of Engineering Mathematics at the University of Bristol in September 2019. He gained his PhD working with the Centre for Secure Information Technologies (CSIT) at Queen's University Belfast in 2017 where he researched large scale unsupervised machine learning for complex data. He has worked with inter-disciplinary academic and industrial partners on numerous projects, including large-scale fraud detection and large-scale pervasive personal behaviour analysis for clinical decision support. His research interests lie around unsupervised machine learning, deep learning on multimodal and complex data with applications to social network analysis, recommender systems, healthcare and cybersecurity.