Decision support improves information integration and criterion placement in a multi-cue signal-identification task

References

• Arkes, H.R., R.M. Dawes, and C. Christensen. 1986. “Factors Influencing the Use of a Decision Rule in a Probabilistic Task.” Organizational Behavior and Human Decision Processes 37 (1): 93–110. doi:10.1016/0749-5978(86)90046-4.
   Web of Science ®Google Scholar
• Armitage, P. 1950. “Sequential Analysis with More than Two Alternative Hypotheses, and Its Relation to Discriminant Function Analysis.” Journal of the Royal Statistical Society Series B: Statistical Methodology 12 (1): 137–144. doi:10.1111/j.2517-6161.1950.tb00050.x.
   Web of Science ®Google Scholar
• Arnold, V., P.A. Collier, S.A. Leech, and S.G. Sutton. 2004. “Impact of Intelligent Decision Aids on Expert and Novice Decision-Makers’ Judgments.” Accounting & Finance 44 (1): 1–26. doi:10.1111/j.1467-629x.2004.00099.x.
   Google Scholar
• Aust, F., and M. Barth. 2020. “Papaja: Prepare Reproducible APA Journal Articles with R Markdown.” https://github.com/crsh/papaja.
   Google Scholar
• Bache, S.M., and H. Wickham. 2022. “Magrittr: A Forward-Pipe Operator for R.” https://CRAN.R-project.org/package=magrittr.
   Google Scholar
• Barg-Walkow, L.H., and W.A. Rogers. 2016. “The Effect of Incorrect Reliability Information on Expectations, Perceptions, and Use of Automation.” Human Factors 58 (2): 242–260. doi:10.1177/0018720815610271.
   PubMed Web of Science ®Google Scholar
• Bartlett, M.L., D.J. Carragher, P. Hancock, and J.S. McCarley. in press. “Benchmarking Automation-Aided Performance in a Forensic Face Matching Task.” Applied Eronomics.
   Google Scholar
• Bartlett, M.L., and J.S. McCarley. 2017. “Benchmarking Aided Decision Making in a Signal Detection Task.” Human Factors 59 (6): 881–900. doi:10.1177/0018720817700258.
   PubMed Web of Science ®Google Scholar
• Bartlett, M.L., and J.S. McCarley. 2019. “No Effect of Cue Format on Automation Dependence in an Aided Signal Detection Task.” Human Factors 61 (2): 169–190. doi:10.1177/0018720818802961.
   PubMed Web of Science ®Google Scholar
• Bartlett, M.L., and J.S. McCarley. 2020. “Ironic Efficiency in Automation-Aided Signal Detection.” Ergonomics 64 (1): 103–112. doi:10.1080/00140139.2020.1809716.
   PubMed Web of Science ®Google Scholar
• Berg, B.B. 1990. “Observer Efficiency and Weights in a Multiple Observation Task.” The Journal of the Acoustical Society of America 88 (1): 149–158. doi:10.1121/1.399962.
   PubMed Web of Science ®Google Scholar
• Bhatia, S., and T.L. Mullett. 2016. “The Dynamics of Deferred Decision.” Cognitive Psychology 86: 112–151. doi:10.1016/j.cogpsych.2016.02.002.
   PubMed Web of Science ®Google Scholar
• Bisantz, A.M., S.S. Marsiglio, and J. Munch. 2005. “Displaying Uncertainty: Investigating the Effects of Display Format and Specificity.” Human Factors 47 (4): 777–796. doi:10.1518/001872005775570916.
   PubMed Web of Science ®Google Scholar
• Busemeyer, J.R. 1985. “Decision Making under Uncertainty: A Comparison of Simple Scalability, Fixed-Sample, and Sequential-Sampling Models.” Journal of Experimental Psychology, Learning, Memory, and Cognition 11 (3): 538–564. doi:10.1037/0278-7393.11.3.538.
   PubMed Web of Science ®Google Scholar
• Busemeyer, J.R., and A. Rapoport. 1988. “Psychological Models of Deferred Decision Making.” Journal of Mathematical Psychology 32 (2): 91–134. doi:10.1016/0022-2496(88)90042-9.
   Web of Science ®Google Scholar
• Cadario, R., C. Longoni, and C.K. Morewedge. 2021. “Understanding, Explaining, and Utilizing Medical Artificial Intelligence.” Nature Human Behaviour 5 (12): 1636–1642. doi:10.1038/s41562-021-01146-0.
   PubMed Web of Science ®Google Scholar
• Cao, K., Y. Xia, J. Yao, X. Han, L. Lambert, T. Zhang, W. Tang, G. Jin, H. Jiang, X. Fang, I. Nogues, X. Li, W. Guo, Y. Wang, W. Fang, M. Qiu, Y. Hou, T. Kovarnik, M. Vocka, Y. Lu, Y. Chen, X. Chen, Z. Liu, J. Zhou, C. Xie, R. Zhang, H. Lu, G.D. Hager, A.L. Yuille, L. Lu, C. Shao, Y. Shi, Q. Zhang, T. Liang, L. Zhang, and J. Lu. 2023. “Large-Scale Pancreatic Cancer Detection via Non-Contrast CT and Deep Learning.” Nature Medicine 29 (12): 3033–3043. doi:10.1038/s41591-023-02640-w.
   PubMedGoogle Scholar
• Chen, J.Y.C., S.G. Lakhmani, K. Stowers, A.R. Selkowitz, J.L. Wright, and M. Barnes. 2018. “Situation Awareness-Based Agent Transparency and Human-Autonomy Teaming Effectiveness.” Theoretical Issues in Ergonomics Science 19 (3): 259–282. doi:10.1080/1463922X.2017.1315750.
   Web of Science ®Google Scholar
• Corritore, C.L., B. Kracher, and S. Wiedenbeck. 2003. “On-Line Trust: Concepts, Evolving Themes, a Model.” International Journal of Human-Computer Studies 58 (6): 737–758. doi:10.1016/S1071-5819(03)00041-7.
   Web of Science ®Google Scholar
• Dawes, R.M., and B. Corrigan. 1974. “Linear Models in Decision Making.” Psychological Bulletin 81 (2): 95–106. doi:10.1037/h0037613.
   Web of Science ®Google Scholar
• Dawes, R.M., D. Faust, and P.E. Meehl. 1989. “Clinical versus Actuarial Judgment.” Science 243 (4899): 1668–1674. doi:10.1126/science.2648573.
   PubMed Web of Science ®Google Scholar
• Dietvorst, B.J., J.P. Simmons, and C. Massey. 2015. “Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err.” Journal of Experimental Psychology. General 144 (1): 114–126. doi:10.1037/xge0000033.
   PubMed Web of Science ®Google Scholar
• Dzindolet, M.T., L.G. Pierce, H.P. Beck, L.A. Dawe, and B.W. Anderson. 2001. “Predicting Misuse and Disuse of Combat Identification Systems.” Military Psychology 13 (3): 147–164. doi:10.1207/S15327876MP1303_2.
   Web of Science ®Google Scholar
• Einhorn, H.J. 1972. “Expert Measurement and Mechanical Combination.” Organizational Behavior and Human Performance 7 (1): 86–106. doi:10.1016/0030-5073(72)90009-8.
   Web of Science ®Google Scholar
• Elvers, G.C., and P. Elrif. 1997. “The Effects of Correlation and Response Bias in Alerted Monitor Displays.” Human Factors 39 (4): 570–580. doi:10.1518/001872097778667960.
   Web of Science ®Google Scholar
• Fried, L.S., and C.R. Peterson. 1969. “Information Seeking: Optional versus Fixed Stopping.” Journal of Experimental Psychology 80 (3, Pt.1): 525–529. doi:10.1037/h0027484.
   Google Scholar
• Garnier, S., N. Ross, R. Rudis, M. Sciaini, A. P. Camargo, and C. Scherer. 2024. “viridis(Lite) – Colorblind-Friendly Color Maps for R.” viridis package version 0.6.5. doi:10.5281/zenodo.4679424.
   Google Scholar
• Gegoff, I., M. Tatasciore, V. Bowden, J.S. McCarley, and S. Loft. 2023. “Transparent Automated Advice to Mitigate the Impact of Variation in Automation Reliability.” Human Factors 66 (8): 2008–2024. doi:10.1177/00187208231196738.
   PubMed Web of Science ®Google Scholar
• Hautus, M.J., N.A. Macmillan, and C.D. Creelman. 2022. Detection Theory: A User’s Guide. Third edition. New York, NY: Routledge.
   Google Scholar
• Healy, A.F., and M. Kubovy. 1978. “The Effects of Payoffs and Prior Probabilities on Indices of Performance and Cutoff Location in Recognition Memory.” Memory & Cognition 6 (5): 544–553. doi:10.3758/BF03198243.
   PubMed Web of Science ®Google Scholar
• Healy, A.F., and M. Kubovy. 1981. “Probability Matching and the Formation of Conservative Decision Rules in a Numerical Analog of Signal Detection.” Journal of Experimental Psychology: Human Learning & Memory 7 (5): 344–354. doi:10.1037/0278-7393.7.5.344.
   Google Scholar
• Karelaia, N., and R.M. Hogarth. 2008. “Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies.” Psychological Bulletin 134 (3): 404–426. doi:10.1037/0033-2909.134.3.404.
   PubMed Web of Science ®Google Scholar
• Kunze, A., S.J. Summerskill, R. Marshall, and A.J. Filtness. 2019. “Automation Transparency: Implications of Uncertainty Communication for Human-Automation Interaction and Interfaces.” Ergonomics 62 (3): 345–360. doi:10.1080/00140139.2018.1547842.
   PubMed Web of Science ®Google Scholar
• Lee, J.D., and K.A. See. 2004. “Trust in Automation: Designing for Appropriate Reliance.” Human Factors 46 (1): 50–80. doi:10.1518/hfes.46.1.50_30392.
   PubMed Web of Science ®Google Scholar
• Mackay, J.M., and J.J. Elam. 1992. “A Comparative Study of How Experts and Novices Use a Decision Aid to Solve Problems in Complex Knowledge Domains.” Information Systems Research 3 (2): 150–172. doi:10.1287/isre.3.2.150.
   Web of Science ®Google Scholar
• Mercado, J.E., M.A. Rupp, J.Y.C. Chen, M.J. Barnes, D. Barber, and K. Procci. 2016. “Intelligent Agent Transparency in Human–Agent Teaming for Multi-Uxv Management.” Human Factors 58 (3): 401–415. doi:10.1177/0018720815621206.
   PubMed Web of Science ®Google Scholar
• Meyer, J. 2001. “Effects of Warning Validity and Proximity on Responses to Warnings.” Human Factors 43 (4): 563–572. doi:10.1518/001872001775870395.
   PubMed Web of Science ®Google Scholar
• Montgomery, D.A. 1999. “Human Sensitivity to Variability Information in Detection Decisions.” Human Factors 41 (1): 90–105. doi:10.1518/001872099779577246.
   Web of Science ®Google Scholar
• Montgomery, D.A., and R.D. Sorkin. 1996. “Observer Sensitivity to Element Reliability in a Multielement Visual Display.” Human Factors 38 (3): 484–494. doi:10.1518/001872096778702024.
   Web of Science ®Google Scholar
• Morey, R.D., and J.N. Rouder. 2018. “BayesFactor: Computation of Bayes Factors for Common Designs.” https://CRAN.R-project.org/package=BayesFactor.
   Google Scholar
• Morey, R.D., and J.N. Rouder. 2022. “BayesFactor: Computation of Bayes Factors for Common Designs.” https://CRAN.R-project.org/package=BayesFactor.
   Google Scholar
• Munoz Gomez Andrade, F., J. Duncan-Reid, and J.S. McCarley. 2022. “The Effect of Correlated Observations on Human-Automation Interaction in a Signal Detection Task.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 66 (1): 2062–2066. doi:10.1177/1071181322661290.
   Google Scholar
• Neyedli, H.F., J.G. Hollands, and G.A. Jamieson. 2011. “Beyond Identity: Incorporating System Reliability Information into an Automated Combat Identification System.” Human Factors 53 (4): 338–355. doi:10.1177/0018720811413767.
   PubMed Web of Science ®Google Scholar
• Novotny, T., R. Bond, I. Andrsova, L. Koc, M. Sisakova, D. Finlay, D. Guldenring, J. Spinar, and M. Malik. 2017. “The Role of Computerized Diagnostic Proposals in the Interpretation of the 12-Lead Electrocardiogram by Cardiology and Non-Cardiology Fellows.” International Journal of Medical Informatics 101: 85–92. doi:10.1016/j.ijmedinf.2017.02.007.
   PubMed Web of Science ®Google Scholar
• Parasuraman, R., and V. Riley. 1997. “Humans and Automation: Use, Misuse, Disuse, Abuse.” Human Factors: The Journal of the Human Factors and Ergonomics Society 39 (2): 230–253. doi:10.1518/001872097778543886.
   Web of Science ®Google Scholar
• Pedersen, T.L. 2020. “Patchwork: The Composer of Plots.” https://CRAN.R-project.org/package=patchwork.
   Google Scholar
• Pitz, G.F., H. Reinhold, and E. Scott Geller. 1969. “Strategies of Information Seeking in Deferred Decision Making.” Organizational Behavior and Human Performance 4 (1): 1–19. doi:10.1016/0030-5073(69)90028-2.
   Google Scholar
• Pleskac, T.J., and J.R. Busemeyer. 2010. “Two-Stage Dynamic Signal Detection: A Theory of Choice, Decision Time, and Confidence.” Psychological Review 117 (3): 864–901. doi:10.1037/a0019737.
   PubMed Web of Science ®Google Scholar
• R Core Team. 2020. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing.
   Google Scholar
• Rapoport, A., and G.J. Burkheimer. 1971. “Models for Deferred Decision Making.” Journal of Mathematical Psychology 8 (4): 508–538. doi:10.1016/0022-2496(71)90005-8.
   Web of Science ®Google Scholar
• Reiner, A.J., J.G. Hollands, and G.A. Jamieson. 2017. “Target Detection and Identification Performance Using an Automatic Target Detection System.” Human Factors 59 (2): 242–258. doi:10.1177/0018720816670768.
   PubMed Web of Science ®Google Scholar
• Rouder, J.N., R.D. Morey, P.L. Speckman, and J.M. Province. 2012. “Default Bayes Factors for ANOVA Designs.” Journal of Mathematical Psychology 56 (5): 356–374. doi:10.1016/j.jmp.2012.08.001.
   Web of Science ®Google Scholar
• Rouder, J.N., R.D. Morey, J. Verhagen, A.R. Swagman, and E.-J. Wagenmakers. 2017. “Bayesian Analysis of Factorial Designs.” Psychological Methods 22 (2): 304–321. doi:10.1037/met0000057.
   PubMed Web of Science ®Google Scholar
• Rouder, J.N., P.L. Speckman, D. Sun, R.D. Morey, and G. Iverson. 2009. “Bayesian t-Tests for Accepting and Rejecting the Null Hypothesis.” Psychonomic Bulletin & Review 16 (2): 225–237. doi:10.3758/PBR.16.2.225.
   PubMed Web of Science ®Google Scholar
• Schwartz, D.R., and W.C. Howell. 1985. “Optional Stopping Performance under Graphic and Numeric CRT Formatting.” Human Factors: The Journal of the Human Factors and Ergonomics Society 27 (4): 433–444. doi:10.1177/001872088502700407.
   Web of Science ®Google Scholar
• Sieck, W.R., and H.R. Arkes. 2005. “The Recalcitrance of Overconfidence and Its Contribution to Decision Aid Neglect.” Journal of Behavioral Decision Making 18 (1): 29–53. doi:10.1002/bdm.486.
   Web of Science ®Google Scholar
• Skraaning, G., and G.A. Jamieson. 2021. “Human Performance Benefits of the Automation Transparency Design Principle: Validation and Variation.” Human Factors 63 (3): 379–401. doi:10.1177/0018720819887252.
   Google Scholar
• Sorkin, R.D., T.R. Mabry, M.S. Weldon, and G. Elvers. 1991. “Integration of Information from Multiple Element Displays.” Organizational Behavior and Human Decision Processes 49 (2): 167–187. doi:10.1016/0749-5978(91)90047-W.
   Web of Science ®Google Scholar
• Stone, M. 1960. “Models for Choice-Reaction Time.” Psychometrika 25 (3): 251–260. doi:10.1007/BF02289729.
   Web of Science ®Google Scholar
• Tatasciore, M., V. Bowden, and S. Loft. 2023. “Do Concurrent Task Demands Impact the Benefit of Automation Transparency?” Applied Ergonomics 110: 104022. doi:10.1016/j.apergo.2023.104022.
   PubMed Web of Science ®Google Scholar
• Ulehla, Z.J. 1966. “Optimality of Perceptual Decision Criteria.” Journal of Experimental Psychology 71 (4): 564–569. doi:10.1037/h0023007.
   PubMedGoogle Scholar
• Wagenmakers, E.-J., M. Marsman, T. Jamil, A. Ly, J. Verhagen, J. Love, R. Selker, Q.F. Gronau, M. Šmíra, S. Epskamp, D. Matzke, J.N. Rouder, and R.D. Morey. 2018. “Bayesian Inference for Psychology. Part I: Theoretical Advantages and Practical Ramifications.” Psychonomic Bulletin & Review 25 (1): 35–57. doi:10.3758/s13423-017-1343-3.
   PubMed Web of Science ®Google Scholar
• Wang, L., G.A. Jamieson, and J.G. Hollands. 2009. “Trust and Reliance on an Automated Combat Identification System.” Human Factors 51 (3): 281–291. doi:10.1177/0018720809338842.
   PubMed Web of Science ®Google Scholar
• Wetzels, R., D. Matzke, M.D. Lee, J.N. Rouder, G.J. Iverson, and E.-J. Wagenmakers. 2011. “Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests.” Perspectives on Psychological Science 6 (3): 291–298. doi:10.1177/1745691611406923.
   Web of Science ®Google Scholar
• Wickens, C.D., and S.R. Dixon. 2007. “The Benefits of Imperfect Diagnostic Automation: A Synthesis of the Literature.” Theoretical Issues in Ergonomics Science 8 (3): 201–212. doi:10.1080/14639220500370105.
   Google Scholar
• Wickens, C.D., Helton, W.S., and Hollands. 2022. Engineering Psychology and Human Performance. 5th edition. New York, NY: Routledge.
   Google Scholar
• Wickham, H., M. Averick, J. Bryan, W. Chang, L. McGowan, R. François, G. Grolemund, A. Hayes, L. Henry, J. Hester, M. Kuhn, T. Pedersen, E. Miller, S. Bache, K. Müller, J. Ooms, D. Robinson, D. Seidel, V. Spinu, K. Takahashi, D. Vaughan, C. Wilke, K. Woo, and H. Yutani. 2019. “Welcome to the Tidyverse.” Journal of Open Source Software 4 (43): 1686. doi:10.21105/joss.01686.
   Google Scholar
• Wiegmann, D., J.S. McCarley, A.F. Kramer, and C.D. Wickens. 2006. “Age and Automation Interact to Influence Performance of a Simulated Luggage Screening Task.” Aviation, Space, and Environmental Medicine 77 (8): 825–831.
   PubMed Web of Science ®Google Scholar
• Wilke, C. 2020. “Cowplot: Streamlined Plot Theme and Plot Annotations for ‘ggplot2’.” https://CRAN.R-project.org/package=cowplot.
   Google Scholar