Do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity

References

• Azarbonyad, H., Dehghani, M., Kenter, T., Marx, M., Kamps, J., & De Rijke, M. (2017). Hierarchical re-estimation of topic models for measuring topical diversity. In J. M. Jose, C. Hauff, I. S. Altıngovde, D. Song, D. Albakour, S. Watt, & J. Tait (Eds.), Advances in information retrieval: 39th European Conference on IR research (ECIR 2017), Aberdeen, UK (pp. 68–81). Cham: Springer.
   Google Scholar
• Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130–1132. doi: 10.1126/science.aaa1160
   PubMed Web of Science ®Google Scholar
• Beam, M. A. (2014). Automating the news: How personalized news recommender system design choices impact news reception. Communication Research, 41(8), 1019–1041. doi: 10.1177/0093650213497979
   Web of Science ®Google Scholar
• Benson, R. (2009). What makes news more multiperspectival? A field analysis. Poetics, 37(5), 402–418. doi: 10.1016/j.poetic.2009.09.002
   Web of Science ®Google Scholar
• Blei, D., Ng, A., & Jordan, M. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.
   Web of Science ®Google Scholar
• Bozdag, E. (2013). Bias in algorithmic filtering and personalization. Ethics and Information Technology, 15(3), 209–227. doi: 10.1007/s10676-013-9321-6
   Web of Science ®Google Scholar
• Bridge, D., & Kelly, J. P. (2006). Ways of computing diverse collaborative recommendations. In International conference on adaptive hypermedia and adaptive web-based systems (pp. 41–50). Springer.
   Google Scholar
• De Smedt, T., & Daelemans, W. (2012). Pattern for Python. The Journal of Machine Learning Research, 13, 2063–2067.
   Web of Science ®Google Scholar
• Duncan, J., & Reid, J. (2013). Toward a measurement tool for the monitoring of media diversity and pluralism in South Africa: A public-centred approach. Communication, 39(4), 483–500.
   Google Scholar
• Dylko, I. B. (2016). How technology encourages political selective exposure. Communication Theory, 26(4), 389–409. doi: 10.1111/comt.12089
   Web of Science ®Google Scholar
• Dylko, I. B., Dolgov, I., Hoffman, W., Eckhart, N., Molina, M., & Aaziz, O. (2017). The dark side of technology: An experimental investigation of the influence of customizability technology on online political selective exposure. Computers in Human Behavior, 73, 181–190. doi: 10.1016/j.chb.2017.03.031
   Web of Science ®Google Scholar
• Ferree, M. M., Gamson, W. A., Gerhards, J., & Rucht, D. (2002). Four models of the public sphere in modern democracies. Theory and Society, 31(3), 289–324. doi: 10.1023/A:1016284431021
   Web of Science ®Google Scholar
• Flaxman, S. R., & Rao, J. M. (2016). Filter bubbles, echo chambers, and online news consumption. Public Opinion Quarterly, 80, 298–320. doi: 10.1093/poq/nfw006
   Web of Science ®Google Scholar
• Fletcher, R., & Nielsen, R. K. (2017). Are news audiences increasingly fragmented? A cross-national comparative analysis of cross-platform news audience fragmentation and duplication. Journal of Communication, 67(4), 476–498. doi: 10.1111/jcom.12315
   Web of Science ®Google Scholar
• Haim, M., Graefe, A., & Brosius, H.-B. (2017). Burst of the filter bubble? Effects of personalization on the diversity of Google News. Digital Journalism, 1–14.
   Google Scholar
• Helberger, N., Karppinen, K., & D'Acunto, L. (2018). Exposure diversity as a design principle for recommender systems. Information, Communication & Society, 21(2), 191–207. doi: 10.1080/1369118X.2016.1271900
   Web of Science ®Google Scholar
• Junqué de Fortuny, E., De Smedt, T., Martens, D., & Daelemans, W. (2012). Media coverage in times of political crisis: A text mining approach. Expert Systems with Applications, 39(14), 11616–11622. doi: 10.1016/j.eswa.2012.04.013
   Web of Science ®Google Scholar
• Karppinen, K. (2013). Rethinking media pluralism. Donald McGannon Research Center. New York, NY: Fordham University Press.
   Google Scholar
• Karppinen, K. (2015). The limits of empirical indicators: Media pluralism as an essentially contested concept. In P. Valcke, M. Sükösd, & R. G. Picard (Eds.), Media pluralism and diversity (pp. 287–296). London: Palgrave Macmillan.
   Google Scholar
• Kitchin, R. (2017). Thinking critically about and researching algorithms. Information, Communication & Society, 20(1), 14–29. doi: 10.1080/1369118X.2016.1154087
   Web of Science ®Google Scholar
• Knijnenburg, B. P., Willemsen, M. C., Gantner, Z., Soncu, H., & Newell, C. (2012). Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22(4–5), 441–504. doi: 10.1007/s11257-011-9118-4
   Web of Science ®Google Scholar
• Kotkov, D., Wang, S., & Veijalainen, J. (2016). A survey of serendipity in recommender systems. Knowledge-Based Systems, 111(Supplement C), 180–192. doi: 10.1016/j.knosys.2016.08.014
   Google Scholar
• Kunaver, M., & Požrl, T. (2017). Diversity in recommender systems – a survey. Knowledge-Based Systems, 123(Supplement C), 154–162. doi: 10.1016/j.knosys.2017.02.009
   Google Scholar
• Masini, A., Aelst, P. V., Zerback, T., Reinemann, C., Mancini, P., Mazzoni, M., Damiani, M., & Coen, S. (2017). Measuring and explaining the diversity of voices and viewpoints in the news. Journalism Studies, 54, 1–20. doi: 10.1080/1461670X.2017.1343650
   Google Scholar
• Napoli, P. M. (2015). Assessing media diversity in the US: A comparative analysis of the FCC's diversity index and the EU's media pluralism monitor. In P. Valcke, M. Sükösd, & R. G. Picard (Eds.), Media pluralism and diversity. London: Palgrave. Assessing media diversity in the US: A comparative analysis of the FCC's diversity index and the EU's media pluralism monitor. In Media pluralism and diversity (pp. 141–151). Springer.
   Google Scholar
• O'Callaghan, D., Greene, D., Conway, M., Carthy, J., & Cunningham, P. (Aug 2015). Down the (white) rabbit hole: The extreme right and online recommender systems. Social Science Computer Review, 33(4), 459–478. doi: 10.1177/0894439314555329
   Web of Science ®Google Scholar
• Owen, D., & Smith, G. (2015). Survey article: Deliberation, democracy, and the systemic turn. Journal of Political Philosophy, 23(2), 213–234. doi: 10.1111/jopp.12054
   Web of Science ®Google Scholar
• Pariser, E. (2011). The filter bubble: What the internet is hiding from you. New York, NY: Penguin.
   Google Scholar
• Council of Europe (2007). Recommendation Rec(2007)2 of the committee of ministers to member states on media pluralism and diversity of media content.
   Google Scholar
• Řehůřek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks (pp. 45–50). ELRA, Valletta, Malta. Retrieved from http://is.muni.cz/publication/884893/en.
   Google Scholar
• Ribeiro, M. T., Ziviani, N., Moura, E. S. D., Hata, I., Lacerda, A., & Veloso, A. (2015). Multiobjective pareto-efficient approaches for recommender systems. ACM Transactions on Intelligent Systems and Technology (TIST), 5(4), 53.
   Web of Science ®Google Scholar
• Ricci, F., Rokach, L., & Shapira, B. (2011). Introduction to recommender systems handbook. In F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender systems handbook (pp. 1–35). New York, NY: Springer.
   Google Scholar
• Ricotta, C., & Szeidl, L. (2006). Towards a unifying approach to diversity measures: Bridging the gap between the Shannon entropy and Rao's quadratic index. Theoretical Population Biology, 70(3), 237–243. doi: 10.1016/j.tpb.2006.06.003
   PubMed Web of Science ®Google Scholar
• Schäfer, R., & Bildhauer, F. (2012). Building large corpora from the web using a new efficient tool chain. In Proceedings of the tenth international conference on language resources and evaluation (LREC 2012) (pp. 486–493), Istanbul, Turkey.
   Google Scholar
• Shoemaker, P. J., Eichholz, M., Kim, E., & Wrigley, B. (2001). Individual and routine forces in gatekeeping. Journalism & Mass Communication Quarterly, 78(2), 233–246. doi: 10.1177/107769900107800202
   Web of Science ®Google Scholar
• Sievert, C., & Shirley, K. (2014). LDAvis: A method for visualizing and interpreting topics. In Proceedings of the workshop on interactive language learning, visualization, and interfaces (pp. 63–70). Retrieved from http://www.aclweb.org/anthology/W/W14/W14-3110.
   Google Scholar
• Smyth, B., & McClave, P. (2001). Similarity vs. diversity. In International conference on case-based reasoning (pp. 347–361). Springer, Vancouver, Canada.
   Google Scholar
• Sørensen, J. K., & Schmidt, J. -H. (2016). An algorithmic diversity diet? Questioning assumptions behind a diversity recommendation system for PSM. Working paper, Hans-Bredow-Institute for Media Research, Hamburg, Germany.
   Google Scholar
• Strömbäck, J. (2005). In search of a standard: Four models of democracy and their normative implications for journalism. Journalism Studies, 6(3), 331–345. doi: 10.1080/14616700500131950
   Google Scholar
• Stroud, N. J. (2011). Niche news: The politics of news choice. New York, NY: Oxford University Press.
   Google Scholar
• Sunstein, C. R. (2007). Republic.com 2.0. Princeton, NJ: Princeton University Press.
   Google Scholar
• Tulkens, S., Emmery, C., & Daelemans, W. (2016). Evaluating unsupervised Dutch word embeddings as a linguistic resource. In N. Calzolari, K. Choukri, T. Declerck, M. Grobelnik, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, & S. Piperidis (Eds.), Proceedings of the tenth international conference on language resources and evaluation (LREC 2016). European Language Resources Association (ELRA), Paris, France.
   Google Scholar
• Van Cuilenburg, J. (1999). On competition, access and diversity in media, old and new: Some remarks for communications policy in the information age. New Media & Society, 1(2), 183–207. doi: 10.1177/14614449922225555
   Google Scholar
• Willemsen, M. C., Graus, M. P., & Knijnenburg, B. P. (2016). Understanding the role of latent feature diversification on choice difficulty and satisfaction. User Modeling and User-Adapted Interaction, 26(4), 347–389. doi: 10.1007/s11257-016-9178-6
   Web of Science ®Google Scholar
• Ziegler, C.-N., McNee, S. M., Konstan, J. A., & Lausen, G. (2005). Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web (WWW '05). ACM Press, (PP. 22–32), Chiba, Japan.
   Google Scholar
• Zuiderveen Borgesius, F. J., Trilling, D., Möller, J., Bodó, B., De Vreese, C. H., & Helberger, N. (2016). Should we worry about filter bubbles?. Internet Policy Review, 5(1).
   Web of Science ®Google Scholar