Seek and you shall find? A content analysis on the diversity of five search engines’ results on political queries

ABSTRACT

Search engines are important political news sources and should thus provide users with diverse political information – an important precondition of a well-informed citizenry. The search engines’ algorithmic content selection strongly influences the diversity of the content received by the users – particularly since most users highly trust search engines and often click on only the first result. A widespread concern is that users are not informed diversely by search engines, but how far this concern applies has hardly been investigated. Our study is the first to investigate content diversity provided by five search engines on ten current political issues in Germany. The findings show that sometimes even the first result is highly diverse, but in most cases, more results must be considered to be informed diversely. This unreliability presents a serious challenge when using search engines as political news sources. Our findings call for media policy measures, for example in terms of algorithmic transparency.

KEYWORDS:

• Content diversity
• search engines
• quantitative content analysis
• news
• democracy

Acknowledgements

Many thanks go to our coders Lisa Gast, Hanna Paulke, Daniel Stegmann and Stephan Thalmann as well as to Vanessa Haselbach who collected the material (SERPs, search results) for us.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 A third form of possible search engine bias – personalization – refers to different filtering and sorting between individual users based on users’ previous search behavior and personal preferences (Jürgens & Stark, 2017), which together with the ranking bias leads to the overall output bias (Kulshrestha et al., 2019). However, current studies (e.g., Haim et al., 2018; Robertson et al., 2018) concluded that the related concerns on ‘filter bubbles' (Pariser, 2011) are exaggerated; personalization is mainly conditioned by geolocation (Hannák et al., 2013), which is why we neglect this aspect in our study.

2 Reflective diversity demands that the real world distribution of the aspects of interest (e.g., viewpoints) should be reflected in news coverage, which tends to reinforce the status quo since it neglects minority groups and positions that also contribute to diversity (McQuail, 1992).

3 If the first SERP comprised less than ten results, she proceeded to the second SERP.

4 As we coded each speaker and information unit only once per article, we can consider the equality of distribution only across articles but not within one article. However, an additional analysis in which we only considered the breadth of the frequency distribution (the share of considered information elements/speakers across all articles) yielded very similar results, albeit at a lower level.

5 At this level, the entropy value in each case is based only on the frequency distribution of the first article.

Additional information

Funding

This work was supported by the research program ‘media convergence’ (located at the Johannes Gutenberg University Mainz).

Notes on contributors

Miriam Steiner

Miriam Steiner is a PhD Candidate at the Department of Communication at the Johannes Gutenberg University Mainz, Germany. Her research interests focus on media performance (particularly on tabloidization and media diversity) and on current news consumption.

Melanie Magin

Melanie Magin is an Associate Professor in media sociology at the Department of Sociology and Political Science at the Norwegian University of Science and Technology (NTNU) in Trondheim, Norway. Her research focuses on the societal role and importance of traditional and new media and spans political communication, media performance, information intermediaries, media systems, media structures and comparative research.

Birgit Stark

Birgit Stark is a Full Professor at the Department of Communication at the Johannes Gutenberg University Mainz, Germany. Her work focuses on media convergence, media quality, fragmentation, and comparative media research. This includes research on algorithm-based information intermediaries like Google and Facebook, their societal impact and the chances and risks associated with them.

Stefan Geiß

Stefan Geiß is an Associate Professor of political communication at the Department of Sociology and Political Science at the Norwegian University of Science and Technology (NTNU) in Trondheim, Norway. His research focuses on the dynamics of framing and agenda-setting/ agenda-building and on opinion formation in political scandals, conflicts and crises.