Do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity

ABSTRACT

In the debate about filter bubbles caused by algorithmic news recommendation, the conceptualization of the two core concepts in this debate, diversity and algorithms, has received little attention in social scientific research. This paper examines the effect of multiple recommender systems on different diversity dimensions. To this end, it maps different values that diversity can serve, and a respective set of criteria that characterizes a diverse information offer in this particular conception of diversity. We make use of a data set of simulated article recommendations based on actual content of one of the major Dutch broadsheet newspapers and its users (N=21,973 articles, N=500 users). We find that all of the recommendation logics under study proved to lead to a rather diverse set of recommendations that are on par with human editors and that basing recommendations on user histories can substantially increase topic diversity within a recommendation set.

KEYWORDS:

• News
• recommender systems
• diversity metrics
• filter bubbles
• automated content classification

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Judith Möller is a postdoctoral researcher at the Amsterdam School of Communication Research. In her research she focuses on the effects of political communication, in particular social media and digital media [email: [email protected]].

Damian Trilling is an Assistant Professor at the Department of Communication Science and affiliated with the Amsterdam School of Communication Research. He is intrigued by the question how citizens nowadays are informed about current affairs and events in their society [email: [email protected]].

Natali Helberger is professor in Information Law at the Institute for Information Law. She specializes in the regulation of converging information and communications markets. Focus points of her research are the interface between technology and information law, user rights and the changing role of the user in information law and policy [email: [email protected]].

Bram van Es works is a Research Engineer at the eScience Centre, Amsterdam. He holds a PhD in Physics and has extensive experience in data analysis and machine learning. He has co-developed the a plugin to collect tracking data, the data pipeline and the analysis library for the personalised communication project [email: [email protected]].

Notes

1 A different approach could be to compare the recommendations for different people (see also Haim et al., 2017).

2 First, we calculate the Euclidean distance for all topic pairs, resulting in a fully symmetric matrix , which we call our topic dissimilarity matrix. Second, we multiply with and , for document 1 and 2 respectively, resulting in two matrices and ,(3) which represent the topic dissimilarity matrices weighed by the topic occurrence in the document in question. These matrices and contain mixed terms such as which are ambiguous since it does not include the weight for one of the topics in the pair. To discard these mixed terms, we need to rewrite the above formulation to(4) We can then calculate the Euclidian distance. We assume that each feature space has the same bounds, hence all feature spaces need to be normalized before being combined between the two matrices using the Frobenius norm: .

Additional information

Funding

This work was supported by the European Research Council, ERC-grant 638514, and the Research Priority Area of the University of Amsterdam. It was carried out on the Dutch national e-infrastructure with the support of SURF Foundation.