Black box measures? How to study people’s algorithm skills

ABSTRACT

Considerable scholarship has established that algorithms are an increasingly important part of what information people encounter in everyday life. Much less work has focused on studying users’ experiences with, understandings of, and attitudes about how algorithms may influence what they see and do. The dearth of research on this topic globally with diverse populations may be in part due to the difficulty of studying a subject about which there is no known ground truth given that details about algorithms are proprietary and rarely made public. This paper explicitly takes on the methodological challenges of studying people’s algorithm skills to shed light on the special considerations required when studying a topic about which even the researchers possess limited know-how. The paper advocates for more such scholarship to accompany existing system-level analyses of algorithms’ social implications and offers a blueprint for how to do so.

KEYWORDS:

• algorithms
• Internet skills
• methods
• interviews
• algorithm skills

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Eszter Hargittai is Professor and Chair of Internet Use and Society at the Institute of Communication and Media Research, University of Zurich. In 2019, she was elected Fellow of the International Communication Association and also received the William F. Ogburn Mid-Career Award from the American Sociological Association’s section on Communication, Information Technology and Media Sociology. For two decades, she has researched people’s Internet uses and how these relate to questions of social inequality [email: [email protected], [email protected]].

Jonathan Gruber is a doctoral student in the Internet Use and Society Division at the Institute of Communication and Media Research, University of Zurich. He obtained his BA in Media and Communication Science at the University of Passau and his MA in Journalism and Communication Science at the University of Hamburg. He is interested in people's experiences with algorithms and how these influence their lives.

Jaelle Fuchs is currently pursuing her master’s degree in Internet and Society at the Institute of Communication and Media Studies of the University of Zurich. She finished her BA in Media and Communication Studies and wrote her Bachelor thesis on the focusing effect of terrorist attacks on political communication on Twitter. One of her research interests is political participation on social media and the motivations and actions of active Internet users such as Wikipedia editors.

Teodora Djukaric is a research assistant in the Internet Use & Society Division at the University of Zurich’s Institute of Communication and Media Research. She received her bachelor’s degree in Media and Communication Studies & Cinema Studies from the University of Zurich. Her research focuses on intergenerational communication practices and uses of new media as well as Internet and algorithm skills.

Lisa Brombach is finishing her master’s degree in Communication and Media Research with a minor in Sociology at the University of Zurich. Her Master’s thesis is on the smart city strategy developed for the city of Zurich. Her research interests include the history and sociology of knowledge, interdisciplinary research and the interaction of society and technology.

ORCID

Eszter Hargittai http://orcid.org/0000-0003-4199-4868

Notes

1 While algorithms extend beyond online interactions, we focus on Internet users to control for experiences in that realm.

2 One example of this is the European Union’s General Data Protection Regulation, which, if nothing else, may have had an effect on people’s awareness in the associated countries (DE, HU) given the multitudes of announcements they have received on email and on Web sites about the directive when it kicked in in 2018. Certainly, it is an empirical question worth exploring as a variation in policy context may be relevant.

3 Some of the transcription and translation happened in-house, some of it we contracted with outside experts to save time, but always verified internally.

4 Although not all team members had participated in interviewing, everyone was involved in transcription and translation, so everyone had experiences with at least some interviews by the time we started the analysis phase of the project.