Managing Digital Platforms with Robust Multi-Sided Recommender Systems

ABSTRACT

Digital platforms have replaced traditional markets in most industries and orchestrate socioeconomic aspects of our lives. We address the problem of negative direct side network effects that arise with an increased number of agents on one side of the platform. Negative effects, if unaddressed, lead to undesired long-term consequences for the platform by developing a positive vicious cycle. Addressing negative effects require dynamic solution mechanisms that adapt to the changing landscape of platforms. The recommender systems literature has proposed multi-sided recommender systems (MSR) as a dynamic solution to many problems on platforms. However, current state-of-the-art MSRs do not consider uncertainty in predicting agents’ choices, resulting in limited efficacy. We present a robust multi-sided recommender system that considers estimation errors in agents’ choice to address this concern. Extensive experiments with agent-based models—ride-pooling and education platform—provide support for the efficacy and generalizability of the robust MSR to address negative effects.

KEYWORDS:

• Digital platforms
• network effects
• negative side effects
• multi-sided platforms
• multi-sided recommenders
• robust optimization
• agent-based simulation

Acknowledgments

We thank the Editor-in-Chief, Dr. Vladimir Zwass, and the three anonymous reviewers for their constructive suggestions throughout the review process. We also thank participants of the 2019 Winter Conference on Business Analytics (WCBA) and seminar participants at the University of Wisconsin at Milwaukee and Northern Illinois University for their valuable feedback on earlier versions of this paper. Onkar Malgonde acknowledges financial support for this research from the G. Brint Ryan College of Business.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/07421222.2022.2127440

Disclosure Statement

No potential conflict of interest was reported by the authors.

Notes

¹ A positive direct side effect refers to the “positive benefits received by users when the number of users of the same kind increases—for example, the effect that arose as the number of subscribers to the Bell Telephone network grew” [40, p. 29].

² One-sided recommender systems have relied on data mining and optimization approaches [41]. For example, Adomavicius and Kwon [1] develop a candidate optimization model to balance diversity with the traditional measure of recommender quality, such as accuracy. As a relatively newer field of research within the recommender systems domain, the MSR literature has extensively relied on optimization models.

³ For brevity, this model is based on the method proposed by [45]. In our empirical study, we use the method proposed by Bertsimas and Sim [10] for its efficacy in balancing the optimality of the solution and its protection against constraint violation.

⁴ As a first step in addressing the challenges of uncertainty in data, Soyster [45] proposed a method that traded the optimality of the solution in favor of feasibility for all data. To address this limitation, Ben-Tal and Nemirovski [8] and El-Ghaoui et al. [21] proposed methods that consider robust counterparts of the nominal problem. However, the proposed methods are computationally expensive. To address these computational challenges and retain the optimality of the solution, Bertsimas and Sim [10] propose a method that allows the user to vary the conservatism of the solution. Robust optimization is applied in various fields such as finance (portfolio optimization), supply chain management (inventory control), and engineering design problems [9]. To the best of our knowledge, robust optimization has not yet been introduced to the recommender systems domain.

⁵ Popular platforms include Waze Carpool in the US, BlaBlaCar in Europe, Grab in Southeast Asia, Hitch in China, and Jrney in Africa, among others.

⁶ Similar mechanisms and dynamics of (a) making an offer, (b) accepting/rejecting an offer, (c) using two-way ratings (buyers and sellers rate each other), and (d) agents’ objectives, preferences, and constraints are at play on multiple other types of platforms such as lodging marketplaces (Airbnb), freelancing platforms (Upwork, Fiverr), and crowdsourcing platforms (Amazon Turk, Toloka), among others.

⁷ Among other differences, one of the key differences between ride-pooling and ride-hailing platforms is how matches are made: identified by agents (ride-pooling) or determined by the platform (ride-hailing).

⁸ Fitness has been used in prior agent-based simulation studies in the Information Systems domain [20, 35, 38].

⁹ We thank an anonymous reviewer for recommending this point.

¹⁰ https://www.forbes.com/sites/jeffbercovici/2014/08/14/what-are-we-actually-rating-when-we-rate-otherpeople/?sh=78550e4debca

¹¹ https://www.businessinsider.com/leaked-charts-show-how-ubers-driver-rating-system-works-2015-2

¹² https://www.uber.com/en-TW/blog/5starriders/

¹³ https://www.uber.com/newsroom/rider-ratings-breakdown/

¹⁴ An earlier version of the manuscript used flexible tuples (e.g. [1,0,0,1,0]) to represent a focal agent’s preferences. To generate the preference scores between two agents, we took the ratio of the number of common elements over the tuples’ length (e.g., for a rider [1,0,0,1,0] and driver [1,1,1,0,0], the preference match score is 2/5 = 0.4). Although we note the heterogeneity of ratings for a focal driver across all riders using this approach, the mean rating for all drivers is within the range [0.48, 0.53] and may not represent the preference dynamics on ride-pooling platforms. We appreciate the comments of an anonymous reviewer in identifying this point.

¹⁵ We choose 0.98 as the default parameter because it translates to an average rating of 4.9 (the average rating of riders and drivers based on anecdotal evidence) on a 5-point scale used by ride-pooling platforms.

¹⁶ A random value drawn from a uniform distribution; redrawn for each composite measure in a period; the same for all drivers/riders across all simulation universes for a period.

¹⁷ A random fraction of the existing fitness value; the same for all drivers/riders across all simulation universes.

¹⁸ For example, we assume that a focal driver’s fitness at the start of a period is 0.6. Also, assume that the average capacity utilization across all rides offered by the focal driver exceeds a specified value^xv, and that the focal driver’s average quality of a match is greater than a specified value^xv. If we determine that the fitness increment should be 0.25^xvii, then the fitness of the focal driver will be (0.6+(0.6*0.25)) = 0.75. We use a similar logic for riders.

¹⁹ Although unavailable for ride-pooling platforms, estimates from a related platform suggest a ratio of 20 riders per driver (https://www.earnestresearch.com/behind-the-ridesharing-wheel/; accessed: 05/10/2022)

²⁰ In Tables 3 and 4, G_k is the average fitness of agents in the k universe, where k is either no recommender, a one-sided recommender, a multi-sided recommender, or a robust multi-sided recommender system. Each cell is a Wilcoxon signed-rank test comparing agents’ fitness between the Robust Multi-sided Recommender System and the other system. When statistically significant, we conclude that the difference between the two systems’ fitness is statistically significant and that the system with greater average fitness (G_k) outperforms the other system. The Wilcoxon signed-rank test is appropriate because (a) we need to compare systems’ performance across multiple simulation runs and (b) acomparison should be paired to a simulation run—initial parameters of a simulation running across different systems are the same.

²¹ We compute Cohen’s d for each pair of compared systems. The literature suggests that the effect size can be small (d ≤ 0.2), medium (d = 0.5), or large (d ≥ 0.8), and provides broad categories that should be informed by the study’s context [46]. Each cell shows the probability of superiority and Cohen’s d for the compared systems.

²² We thank Dr. Vladimir Zwass, JMIS Editor-in-Chief, and the three anonymous reviewers for this discussion.

Additional information

Notes on contributors

Onkar S. Malgonde

Onkar S. Malgonde ([email protected]; corresponding author) is an Assistant Professor in the Information Technology & Decision Sciences Department, G. Brint Ryan College of Business, University of North Texas. He received his Ph.D. in Information Systems from the University of South Florida. Before starting his graduate studies, Dr. Malgonde was a Systems Engineer with Infosys Technologies. His research interests are at the intersection of data analytics, digital platforms, and software systems. His work has been published in such journals as MIS Quarterly, Empirical Software Engineering, and Electronic Markets, and in the proceedings of premier Information Systems conferences and workshops.

He Zhang

He Zhang ([email protected]) is an Assistant Professor in the Information Systems and Decision Sciences Department in the Muma College of Business at University of South Florida. His research interests include healthcare information management, big data, and production and inventory management. Dr. Zhang’s research has been published in several journals, including MIS Quarterly, Mathematical Programming, Decision Support Systems and ACM Transactions on Management Information Systems.

Balaji Padmanabhan

Balaji Padmanabhan ([email protected]) is the Anderson Professor of Global Management in the Information Systems Decision Sciences Department and Director of the Center for Analytics & Creativity at University of South Florida. He received his Ph.D. from the Stern School of Business of New York University. Dr. Padmanabhan’s interests include analytics and business intelligence, designing analytics algorithms for business applications, building and evaluating predictive models, patterns discovery in data, enabling citizen data science and applications of analytics in healthcare, recommender systems, fraud detection and elections. His research has been published in the premier Computer Science and Business journals and conference proceedings. He serves on the editorial boards and program committees of many leading academic journals and conferences.

Moez Limayem

Moez Limayem ([email protected]) is the President of the University of North Florida. Until June 2022, he was the Lynn Pippenger Dean of the Muma College of Business at the University of South Florida, which he joined coming from the Sam M. Walton College of Business at the University of Arkansas. Dr. Limayem’s research was published in many top journals, such as Information Systems Research, Journal of Management Information Systems, MIS Quarterly, Journal of the AIS, and Management Science.