Algorithmic versus Human Advice: Does Presenting Prediction Performance Matter for Algorithm Appreciation?

References

• Adomavicius, G.; Bockstedt, J.C.; Curley, S.P.; and Zhang, J. Reducing recommender system biases: An investigation of rating display designs. MIS Quarterly: Management Information Systems, 43, 4 (2019), 1321–1341.
   Web of Science ®Google Scholar
• Ahmad, M.I.; Bernotat, J.; Lohan, K.; and Eyssel, F. Trust and cognitive load during human-robot interaction. arXiv:1909.05160 [cs], (September 2019).
   Google Scholar
• Ahsen, M.E.; Ayvaci, M.U.S.; and Raghunathan, S. When algorithmic predictions use human-generated data: A bias-aware classification algorithm for breast cancer diagnosis. Information Systems Research, 30, 1 (March 2019), 97–116.
   Web of Science ®Google Scholar
• Banerjee, A.V. A simple model of herd behavior. The Quarterly Journal of Economics, 107, 3 (August 1992), 797–817.
   Web of Science ®Google Scholar
• Barrouillet, P.; Bernardin, S.; Portrat, S.; Vergauwe, E.; and Camos, V. Time and cognitive load in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 3 (2007), 570–585.
   PubMed Web of Science ®Google Scholar
• Benbasat, I.; and Wang, W. Trust in and adoption of online recommendation agents. Journal of the Association for Information Systems, 6, 3 (March 2005).
   Google Scholar
• Berk, R. An impact assessment of machine learning risk forecasts on parole board decisions and recidivism. Journal of Experimental Criminology, 13, 2 (June 2017), 193–216.
   Web of Science ®Google Scholar
• Bettman, J.R.; and Kakkar, P. Effects of information presentation format on consumer information acquisition strategies. Journal of Consumer Research, 3, 4 (March 1977), 233–240.
   Web of Science ®Google Scholar
• Cacioppo, J.T.; Petty, R.E.; Feinstein, J.A.; and Jarvis, W.B.G. Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119, 2 (1996), 197–253.
   Web of Science ®Google Scholar
• Cai, C.J.; Winter, S.; Steiner, D.; Wilcox, L., and Terry, M. “Hello AI”: Uncovering the onboarding needs of medical practitioners for human-ai collaborative decision-making. Proceedings of the ACM on Human-Computer Interaction, 3. CSCW, Austin, Texas, November 2019, pp. 1–24.
   Google Scholar
• Canfield, C.; Bruin, W.B. de; and Wong-Parodi, G. Perceptions of electricity-use communications: effects of information, format, and individual differences. Journal of Risk Research, 20, 9 (September 2017), 1132–1153.
   Web of Science ®Google Scholar
• Castelo, N.; Bos, M.W.; and Lehmann, D.R. Task-dependent algorithm aversion. Journal of Marketing Research, 56, 5 (October 2019), 809–825.
   Web of Science ®Google Scholar
• Chevalier, J.A.; and Mayzlin, D. The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43, 3 (August 2006), 345–354.
   Web of Science ®Google Scholar
• Chun, W.Y.; and Kruglanski, A.W. The role of task demands and processing resources in the use of base-rate and individuating information. Journal of Personality and Social Psychology, 91, 2 (2006), 205–217.
   PubMed Web of Science ®Google Scholar
• Das, T.K.; and Teng, B.-S. Trust, control, and risk in strategic alliances: An integrated framework. Organization Studies, 22, 2 (March 2001), 251–283.
   Web of Science ®Google Scholar
• Dietvorst, B.J.; Simmons, J.P.; and Massey, C. Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144, 1 (2015), 114–126.
   PubMed Web of Science ®Google Scholar
• Dietvorst, B.J.; Simmons, J.P.; and Massey, C. Overcoming algorithm aversion: people will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64, 3 (March 2018), 1155–1170.
   Web of Science ®Google Scholar
• Dove, G.; Balestra, M.; Mann, D.; and Nov, O. Good for the many or best for the few? A dilemma in the design of algorithmic advice. Proceedings of the ACM on Human-Computer Interaction, 4. CSCW2, 2020, pp. 1–22.
   Google Scholar
• Duan, J.; Xia, X.; and Van Swol, L.M. Emoticons’ influence on advice taking. Computers in Human Behavior, 79, (February 2018), 53–58.
   Google Scholar
• Dzindolet, M.T.; Peterson, S.A.; Pomranky, R.A.; Pierce, L.G.; and Beck, H.P. The role of trust in automation reliance. International Journal of Human-Computer Studies, 58, 6 (June 2003), 697–718.
   Web of Science ®Google Scholar
• Fügener, A.; Grahl, J.; Gupta, A.; and Ketter, W. Will humans-in-the-loop become borgs? Merits and pitfalls of working with AI. Management Information Systems Quarterly, 45, 3 (2021), 1527–1556.
   Web of Science ®Google Scholar
• Ganzach, Y. Predictor representation and prediction strategies. Organizational Behavior and Human Decision Processes, 56, 2 (November 1993), 190–212.
   Web of Science ®Google Scholar
• Gino, F. Do we listen to advice just because we paid for it? The impact of advice cost on its use. Organizational behavior and human decision processes, 107, 2 (2008), 234–245.
   Web of Science ®Google Scholar
• Gino, F.; and Schweitzer, M.E. Blinded by anger or feeling the love: How emotions influence advice taking. Journal of Applied Psychology, 93, 5 (2008), 1165.
   PubMed Web of Science ®Google Scholar
• Goodman, B.; and Flaxman, S. European Union regulations on algorithmic decision-making and a “right to explanation.” AI Magazine, 38, 3 (October 2017), 50–57.
   Web of Science ®Google Scholar
• Gunaratne, J.; Zalmanson, L.; and Nov, O. The persuasive power of algorithmic and crowdsourced advice. Journal of Management Information Systems, 35, 4 (October 2018), 1092–1120.
   Web of Science ®Google Scholar
• Hainc, N.; Federau, C.; Stieltjes, B.; Blatow, M.; Bink, A., and Stippich, C. The bright, artificial intelligence-augmented future of neuroimaging reading. Frontiers in Neurology, 8, (2017), 489. doi:10.3389/fneur.2017.00489.
   PubMed Web of Science ®Google Scholar
• Harvey, N.; and Fischer, I. Taking advice: Accepting help, improving judgment, and sharing responsibility. Organizational behavior and human decision processes, 70, 2 (1997), 117–133.
   Web of Science ®Google Scholar
• Hayes, A.F. Introduction to Mediation, Moderation, and Conditional Process Analysis, Second Edition: A Regression-Based Approach. New York, NY: Guilford Publications, 2017.
   Google Scholar
• de Holanda Coelho, G.L.; Hanel, P.H.; and Wolf, L.J. The very efficient assessment of need for cognition: developing a six-item version. Assessment, 1, (2018), 16.
   Google Scholar
• Hong, W.; Thong, J.Y.L.; and Tam, K.Y. The effects of information format and shopping task on consumers’ online shopping behavior: A cognitive fit perspective. Journal of Management Information Systems, 21, 3 (November 2004), 149–184.
   Web of Science ®Google Scholar
• Jarvenpaa, S.L. The effect of task demands and graphical format on information processing strategies. Management Science, 35, 3 (March 1989), 285–303.
   Web of Science ®Google Scholar
• Jussupow, E.; Spohrer, K.; Heinzl, A.; and Gawlitza, J. Augmenting medical diagnosis decisions? An investigation into physicians’ decision-making process with artificial intelligence. Information Systems Research, 32, 3 (September 2021), 713–735.
   Web of Science ®Google Scholar
• Kalyuga, S. Cognitive load theory: How many types of load does it really need? Educational Psychology Review, 23, 1 (2011), 1–19.
   Web of Science ®Google Scholar
• Komiak, S.Y.X.; and Benbasat, I. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS Quarterly, 30, 4 (2006), 941–960.
   Web of Science ®Google Scholar
• Lage, I.; Chen, E.; He, J.; Narayanan, M.; Kim, B.; Gershman, S.; and Doshi-Velez, F. An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006. (2019).
   Google Scholar
• Li, X.; and Wu, L. Herding and social media word-of-mouth: Evidence from Groupon. Management Information Systems Quarterly, 42, 4 (December 2018), 1331–1351.
   Web of Science ®Google Scholar
• Liel, Y.; and Zalmanson, L. What If an AI Told You That 2 + 2 Is 5? Conformity to Algorithmic Recommendations. (2020). ICIS 2020 Proceedings. 17. https://aisel.aisnet.org/icis2020/hci_artintel/hci_artintel/17
   Google Scholar
• Logg, J.M.; Minson, J.A.; and Moore, D.A. Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, (March 2019), 90–103.
   Google Scholar
• Longoni, C.; Bonezzi, A.; and Morewedge, C.K. Resistance to medical artificial intelligence. Journal of Consumer Research, 46, 4 (December 2019), 629–650.
   Web of Science ®Google Scholar
• McKnight, D.H.; Liu, P.; and Pentland, B.T. Trust change in information technology products. Journal of Management Information Systems, 37, 4 (October 2020), 1015–1046.
   Web of Science ®Google Scholar
• Meehl, P.E. Clinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. Minneapolis, MN: University of Minnesota Press, 1954.
   Google Scholar
• Meyer, J.; Shamo, M.K.; and Gopher, D. Information structure and the relative efficacy of tables and graphs. Human Factors, 41, 4 (December 1999), 570–587.
   PubMed Web of Science ®Google Scholar
• Mollick, E.; and Nanda, R. Wisdom or madness? Comparing crowds with expert evaluation in funding the arts. Management Science, 62, 6 (June 2016), 1533–1553.
   Web of Science ®Google Scholar
• Montazemi, A.R.; and Wang, S. The effects of modes of information presentation on decision-making: A review and meta-analysis. Journal of Management Information Systems, 5, 3 (December 1988), 101–127.
   Google Scholar
• Mousavi, S.Y.; Low, R.; and Sweller, J. Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87, 2 (1995), 319–334.
   Web of Science ®Google Scholar
• Önkal, D.; Goodwin, P.; Thomson, M.; Gönül, S.; and Pollock, A. The relative influence of advice from human experts and statistical methods on forecast adjustments. Journal of Behavioral Decision Making, 22, 4 (2009), 390–409.
   Web of Science ®Google Scholar
• Pierro, A.; Mannetti, L.; Erb, H.-P.; Spiegel, S.; and Kruglanski, A.W. Informational length and order of presentation as determinants of persuasion. Journal of Experimental Social Psychology, 41, 5 (September 2005), 458–469.
   Web of Science ®Google Scholar
• Poursabzi-Sangdeh, F.; Goldstein, D.G.; Hofman, J.M.; Wortman Vaughan, J.W.; and Wallach, H. Manipulating and Measuring Model Interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York, NY. Association for Computing Machinery, 2021, pp. 1–52.
   Google Scholar
• Promberger, M.; and Baron, J. Do patients trust computers? Journal of Behavioral Decision Making, 19, 5 (2006), 455–468.
   Web of Science ®Google Scholar
• Qiu, L.; and Benbasat, I. Evaluating anthropomorphic product recommendation agents: A social relationship perspective to designing information systems. Journal of Management Information Systems, 25, 4 (April 2009), 145–182.
   Web of Science ®Google Scholar
• Remus, W. A study of graphical and tabular displays and their interaction with environmental complexity. Management Science, 33, 9 (September 1987), 1200–1204.
   Web of Science ®Google Scholar
• Riedl, R.; Mohr, P.N.; Kenning, P.H.; Davis, F.D.; and Heekeren, H.R. Trusting humans and avatars: A brain imaging study based on evolution theory. Journal of Management Information Systems, 30, 4 (2014), 83–114.
   Web of Science ®Google Scholar
• Samson, B.P.V., and Sumi, Y. Exploring factors that influence connected drivers to (not) use or follow recommended optimal routes. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow, Scotland, UK: Association for Computing Machinery, 2019, pp. 1–14.
   Google Scholar
• Sniezek, J.A.; and Buckley, T. Cueing and cognitive conflict in judge-advisor decision making. Organizational Behavior and Human Decision Processes, 62, 2 (May 1995), 159–174.
   Web of Science ®Google Scholar
• Sniezek, J.A.; and Van Swol, L.M. Trust, confidence, and expertise in a judge-advisor system. Organizational Behavior and Human Decision Processes, 84, 2 (March 2001), 288–307.
   PubMed Web of Science ®Google Scholar
• Springer, A.; and Whittaker, S. Progressive disclosure: When, why, and how do users want algorithmic transparency information? ACM Transactions on Interactive Intelligent Systems, 10, 4 (October 2020), 29:1-29: 32.
   Web of Science ®Google Scholar
• Surowiecki, J. The Wisdom Of Crowds: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations. New York, NY: Doubleday & Co, 2004.
   Google Scholar
• Sweller, J. Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 2 (April 1988), 257–285.
   Web of Science ®Google Scholar
• Sweller, J. Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22, 2 (June 2010), 123–138.
   Web of Science ®Google Scholar
• Sweller, J.; van Merriënboer, J.J.; and Paas, F. Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31, 2 (2019), 261–292.
   Web of Science ®Google Scholar
• Sweller, J.; van Merrienboer, J.J.G.; and Paas, F.G.W.C. Cognitive architecture and instructional design. Educational Psychology Review, 10, 3 (September 1998), 251–296.
   Web of Science ®Google Scholar
• Teodorescu, M.; Morse, L.; Awwad, Y.; and Kane, G. Failures of fairness in automation require a deeper understanding of human-ML Augmentation. Management Information Systems Quarterly, 45, 3 (September 2021), 1483–1500.
   Web of Science ®Google Scholar
• Van Merriënboer, J.J.G.; and Sweller, J. Cognitive load theory and complex learning: Recent developments & future directions. Educational Psychology Review, (2005), 147177.
   Google Scholar
• Van Merriënboer, J.J.G.; and Sweller, J. Cognitive load theory in health professional education: design principles and strategies. Medical Education, 44, 1 (January 2010), 85–93.
   PubMed Web of Science ®Google Scholar
• Vessey, I. Cognitive fit: A theory-based analysis of the graphs versus tables literature. Decision Sciences, 22, 2 (1991), 219–240.
   Web of Science ®Google Scholar
• Wang, W.; and Benbasat, I. Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. Journal of Management Information Systems, 23, 4 (May 2007), 217–246.
   Web of Science ®Google Scholar
• Wang, W.; and Benbasat, I. Empirical assessment of alternative designs for enhancing different types of trusting beliefs in online recommendation agents. Journal of Management Information Systems, 33, 3 (July 2016), 744–775.
   Web of Science ®Google Scholar
• Wang, X.; Guo, Y.; and Xu, C. Recommendation algorithms for optimizing hit rate, user satisfaction and website revenue. In Proceedings of the 24th International Conference on Artificial Intelligence. Buenos Aires, Argentina: AAAI Press, 2015, pp. 1820–1826.
   Google Scholar
• Ward, A.; and Mann, T. Don’t mind if I do: Disinhibited eating under cognitive load. Journal of Personality and Social Psychology, 78, 4 (2000), 753–763.
   PubMed Web of Science ®Google Scholar
• Wolf, C.; and Blomberg, J. Evaluating the promise of human-algorithm collaborations in everyday work practices. Proceedings of the ACM on Human-Computer Interaction, 3, CSCW (November 2019), 143:1–143: 23.
   Google Scholar
• Xu, J.; Benbasat, I.; and Cenfetelli, R.T. The nature and consequences of trade-off transparency in the context of recommendation agents. MIS Quarterly, 38, 2 (February 2014), 379–406.
   Web of Science ®Google Scholar
• Yeomans, M.; Shah, A.; Mullainathan, S.; and Kleinberg, J. Making sense of recommendations. Journal of Behavioral Decision Making, 32, 4 (2019), 403–414.
   Web of Science ®Google Scholar
• Yu, S.; Chai, Y.; Chen, H.; Brown, R.A.; Sherman, S.J.; and Nunamaker Jr, J.F. Fall Detection with wearable sensors: A hierarchical attention-based convolutional neural network approach. Journal of Management Information Systems, 38, 4 (2021), 1095–1121.
   Web of Science ®Google Scholar
• Zhou, J.; Arshad, S.Z.; Luo, S., and Chen, F. Effects of uncertainty and cognitive load on user trust in predictive decision making. In R. Bernhaupt, G. Dalvi, A. Joshi, D. K. Balkrishan, J. O’Neill, and M. Winckler () (eds.), Human-Computer Interaction – INTERACT 2017. Mumbai, India: Springer International Publishing, Cham, 2017, pp. 23–39.
   Google Scholar