Towards Human Objective Real-Time Trust of Autonomy Measures for Combat Aviation

References

• Berntson, G. G., Thomas Bigger Jr, J., Eckberg, D. L., Grossman, P., Kaufmann, P. G., Malik, M., Nagaraja, H. N., Porges, S. W., Saul, J. P., Stone, P. H., & Van Der Molen, M. W. (1997). Heart rate variability: Origins methods, and interpretive caveats. Psychophysiology, 34(6), 623–648. https://doi.org/10.1111/j.1469-8986.1997.tb02140.x
   PubMed Web of Science ®Google Scholar
• Bindewald, J. M., Rusnock, C. F., & Miller, M. E. (2018). Measuring human trust behavior in human-machine teams. In Vol. 591. AHFE 2017 International Conference on Human Factors in Simulation and Modeling, 2017 (pp. 47–58). Springer Verlag.
   Google Scholar
• Blair, R. C., & Higgins, J. J. (1980). A comparison of the power of Wilcoxon’s rank-sum statistic to that of student’s t statistic under various nonnormal distributions. Journal of Educational Statistics, 5(4), 309–335.
   Google Scholar
• Borum, R. (2010). The science of interpersonal trust. Mental Health Law & Policy Faculty Publications, 574(1), 2.
   Google Scholar
• Burns, A., Harper, D., Barfield, A., Whitcomb, S., & Jurusik, B. (2011). Auto GCAS for analog flight control system. In 2011 IEEE/AIAA 30th Digital Avionics Systems Conference. IEEE Xplore.
   Google Scholar
• Chavaillaz, A., Wastell, D., & Sauer, J. (2016). System reliability, performance and trust in adaptable automation. Applied Ergonomics, 52(2016), 333–342.
   Google Scholar
• Davis, S. E., & Smith, G. A. (2019). Transcranial direct current stimulation use in warfighting: Benefits, risks, and future prospects. Frontiers in Human Neuroscience, 13. Article 114. https://doi.org/10.3389/fnhum.2019.00114
   Google Scholar
• De Visser, E. J., Cohen, M., Freedy, A., & Parasuraman, R. (2014). A design methodology for trust cue calibration in cognitive agents. In Vol. 8525 LNCS. 6th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2014 - Held as Part of 16th International Conference on Human-Computer Interaction, HCI International 2014 (pp. 251–262). Springer Verlag.
   Google Scholar
• Dideriksen, A., Reuter, C., Patry, T., Hoke, J., & Faubert, J. (2018). “Expert”: Characterizing proficiency for physiological measures of cognitive workload. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference.
   Google Scholar
• Ekbohm, G. (1982). On testing the equality of proportions in the paired case with incomplete data. Psychometrika, 47(1), 115–118.
   Web of Science ®Google Scholar
• Ekman, F., & Johansson, M. (2018). Understanding trust in an AV-context: A mixed method approach. In Proceedings of the 6th Humanist Conference, pp. 13–14. Humanist Visual Centre of Excellence.
   Google Scholar
• Foroughi, C. K., Devlin, S., Pak, R., Brown, N. L., Sibley, C., & Coyne, J. T. (2021). Near-Perfect automation: Investigating performance, trust, and visual attention allocation. Human Factors, 00187208211032889. https://doi.org/10.1177/00187208211032889
   Google Scholar
• Fu, L., Liu, J., Meng, G., & Xie, F. (Eds.) (2013). Research on beyond visual range target allocation and multi-aircraft collaborative decision-making. In 25th Chinese Control and Decision Conference, Guiyang.
   Google Scholar
• Geiselman, E. (1999). Practical considerations for fixed wing helmet-mounted display symbology design. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Air Force Research Lab.
   Google Scholar
• Habibovic, A., Andersson, J., Nilsson, M., Lundgren, V. M., & Nilsson, J. (2016). Evaluating interactions with non-existing automated vehicles: Three wizard of Oz approaches, 32–37.IEEE. https://doi.org/10.1109/ivs.2016.7535360
   Google Scholar
• Harp, D., Armstrong, R., Whiting, H., Purser, Z., Dold, S., & Nevala, S. (2020). Comparison of eye-tracking, Rhino-Pointing, and traditional human system interface (Project Have Rhino) (No. AD1113400). https://www.dtic.mil
   Google Scholar
• Hergeth, S., Lorenz, L., Vilimek, R., & Krems, J. F. (2016). Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving. Human Factors, 58(3), 509–519. https://doi.org/10.1177/0018720815625744
   PubMed Web of Science ®Google Scholar
• Highland, P., Williams, J., Yazvec, M., Dideriksen, A., Corcoran, N., Woodruff, K., Thompson, C., Kirby, L., Chun, E., Kousheh, H., & Schnell, T. (2020). Modelling of unmanned aircraft visibility for see-and-avoid operations. Journal of Unmanned Vehicle Systems, 8(4), 265–284. https://doi.org/10.1139/juvs-2020-0011
   Web of Science ®Google Scholar
• Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570
   PubMed Web of Science ®Google Scholar
• Hoke, J., Reuter, C., Romeas, T., Montariol, M., Schnell, T., & Faubert, J. (2017). Perceptual-Cognitive & physiological assessment of training effectiveness. In Proceedings of the Interservice/Industry Training, Simulation, and Education Conference.
   Google Scholar
• Jarvosek, D. (2019, August 26). Proposers day program brief. https://www.darpa.mil/news-events/2019-05-08
   Google Scholar
• Joint Chiefs of Staff. (2013). Doctrine for the armed forces of the United States. https://www.jcs.mil/Portals/36/Documents/Doctrine/pubs/jp1_ch1.pdf?ver=2019-02-11-174350-967
   Google Scholar
• Jøsang, A., & Presti, S. L. (2004). Analysing the relationship between risk and trust. In C. Jensen, S. Poslad, & T. Dimitrakos (Eds.), Trust management (p. 136).
   Google Scholar
• Khastgir, S., Birrell, S., Dhadyalla, G., & Jennings, P. (2018). Calibrating trust through knowledge: Introducing the concept of informed safety for automation in vehicles. Transportation Research Part C: Emerging Technologies, 96, 290–303. https://doi.org/10.1016/j.trc.2018.07.001
   Web of Science ®Google Scholar
• Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
   PubMed Web of Science ®Google Scholar
• Leichtenstern, K., Bee, N., André, E., Berkmüller, U., & Wagner, J. (2011). Physiological measurement of trust-related behavior in trust-neutral and trust-critical situations. Trust Management V, 358, 165–172.
   Google Scholar
• Li, R., Pei, S., Chen, B., Song, Y., Zhang, T., Yang, W., & Shaman, J. (2020). Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2). Science, 368(6490), 489–493.
   PubMed Web of Science ®Google Scholar
• Litchfield, J. J., & Wilcoxon, F. (1949). A simplified method of evaluating dose-effect experiments. The Journal of Pharmacology and Experimental Therapeutics, 96(2), 99–113.
   PubMed Web of Science ®Google Scholar
• Lyons, J. B., & Guznov, S. Y. (2019). Individual differences in human–machine trust: A multi-study look at the perfect automation schema. Theoretical Issues in Ergonomics Science, 20(4), 440–458. https://doi.org/10.1080/1463922X.2018.1491071
   Web of Science ®Google Scholar
• Lyons, J. B., Vo, T., Wynne, K. T., Mahoney, S., Nam, C. S., & Gallimore, D. (2021). Trusting autonomous security robots: The role of reliability and stated social intent. Human Factors, 63(4), 603–618. https://doi.org/10.1177/0018720820901629
   Web of Science ®Google Scholar
• Madhavan, P., & Wiegmann, D. A. (2007). Similarities and differences between human–human and human–automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8(4), 277–301.
   Google Scholar
• Martin, P., Calhoun, P., Schnell, T., & Thompson, C. (2019). Objective Measures of Pilot Workload (No. AD1088654). https://www.dtic.mil/
   Google Scholar
• McHugh, M. L. (2013). The chi-square test of independence. Biochemia Medica, 23(2), 143–149.
   PubMed Web of Science ®Google Scholar
• McKendrick, R., Parasuraman, R., Murtza, R., Formwalt, A., Baccus, W., Paczynski, M., & Ayaz, H. (2016). Into the wild: Neuroergonomic differentiation of hand-held and augmented reality wearable displays during outdoor navigation with functional near infrared spectroscopy. Frontiers in Human Neuroscience, 10, 216. https://doi.org/10.3389/fnhum.2016.00216
   PubMed Web of Science ®Google Scholar
• McKight, P. E., & Najab, J. (2010). Kruskal‐wallis test. In I. B. Weiner & W. E. Craighead (Eds.), The Corsini encyclopedia of psychology (Vol. 1). https://doi.org/10.1002/9780470479216.corpsy0491.
   Google Scholar
• Mehta, R. K., & Parasuraman, R. (2013). Neuroergonomics: A review of applications to physical and cognitive work. Frontiers in Human Neuroscience, 7. https://doi.org/10.3389/fnhum.2013.00889
   Google Scholar
• Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. The Journal of Social Issues, 56(1), 81–103.
   Web of Science ®Google Scholar
• Nass, C., Moon, Y., Fogg, B. J., Reeves, B., & Dryer, D. C. (1995). Can computer personalities be human personalities? International Journal of Human-Computer Studies, 43(2), 223–239.
   Web of Science ®Google Scholar
• Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
   Google Scholar
• Nass, C., Steuer, J., Tauber, E., & Reeder, H. (1993). Anthropomorphism, agency, and ethopoeia: Computers as social actors. INTERACT’93 and CHI’93 Conference Companion on Human factors in Computing Systems.
   Google Scholar
• Neogi, N. A. (2020). Progress in autonomy: Space robotics, satellite systems, air traffic control. Aerospace America, 58(11), 44–44.
   Web of Science ®Google Scholar
• Okamura, K., & Yamada, S. (2018). Adaptive trust calibration for supervised autonomous vehicles. In Adjunct Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Association for Computing Machinery.
   Google Scholar
• Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2008). Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2(2), 140–160.
   Google Scholar
• Reichlen, C. P. (2018). A comparison of linear and nonlinear ECG-based methods to assess pilot workload in a live-flight tactical setting [ Master’s thesis]. The University of Iowa. https://doi.org/10.17077/etd.hqx7edns
   Google Scholar
• Retzlaff, P. D., & Gibertini, M. (1987). Air force pilot personality: Hard data on the“right stuff”. Multivariate Behavioral Research, 22(4), 383–399.
   PubMed Web of Science ®Google Scholar
• Rice, S. (2009). Examining single- and multiple-process theories of trust in automation. The Journal of General Psychology, 136(3), 303–322. https://doi.org/10.3200/GENP.136.3.303-322
   PubMed Web of Science ®Google Scholar
• Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 540–547. https://doi.org/10.1518/001872008x288457
   PubMed Web of Science ®Google Scholar
• Schnell, T. (2021). Cognitive Assessment Tool Set (CATS) user manual. The University of Iowa, https://iti.uiowa.edu/our-research/centers/operator-performance-laboratory
   Google Scholar
• Schnell, T., & Engler, J. (2014). Entropic skill assessment of unmanned aerial systems (UAS) operators. Journal of Unmanned Vehicle Systems, 2(2), 53–68. https://doi.org/10.1139/juvs-2014-0001
   Google Scholar
• Schnell, T., Reichlen, C., & Reuter, C. (2017). Spatial disorientation threat characterization for F-35 representative helmet-mounted display use in the flight environment (No. AD1038022). https://www.dtic.mil/
   Google Scholar
• Schnell, T., Reuter, C., & Cover, M. (2017). Visual dominance in pilots during recovery from upset. In 2017 IEEE/AIAA 36th Digital Avionics Systems Conference (DASC). https://doi.org/www.scopus.com/inward/record.uri?eid=2-s2.0-85040999591&doi=10.1109%2fDASC.2017.8102062&partnerID=40&md5=008becab05194fbb8c2bec31a09b56f1
   Google Scholar
• Schreiber, B. T., Watz, E., & Bennett, W., Jr. (2003). Objective human performance measurement in a distributed environment: Tomorrow’s needs. In 2003 Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) Proceedings.
   Google Scholar
• Selcon, S. J., & Taylor, R. M. (1990). Evaluation of the Situational Awareness Rating Technique (SART) as a tool for aircrew systems design. AGARD, Situational Awareness in Aerospace Operations.
   Google Scholar
• Selkowitz, A. R., Lakhmani, S. G., & Chen, J. Y. C. (2017). Using agent transparency to support situation awareness of the autonomous squad member. Cognitive Systems Research, 46, 13–25. https://doi.org/10.1016/j.cogsys.2017.02.003
   Web of Science ®Google Scholar
• Sheridan, T. B., & Verplank, W. L. (1978). Human and computer control of undersea teleoperators. https://apps.dtic.mil/sti/citations/ADA057655
   Google Scholar
• Stokes, C. K., Lyons, J. B., Littlejohn, K., Natarian, J., Case, E., & Speranza, N. (2010). Accounting for the human in cyberspace: Effects of mood on trust in automation. In International Symposium on Collaborative Technologies and Systems. IEEE.
   Google Scholar
• Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285.
   Web of Science ®Google Scholar
• Tabachnick, B. G., & Fidell, L. S. (2007). Experimental designs using ANOVA (Vol. 724, p. 69). Belmont, CA: Thomson/Brooks/Cole.
   Google Scholar
• Tucker, P. (2019). SecDef: China is exporting killer robots to the Mideast. https://www.defenseone.com/technology/2019/11/secdef-china-exporting-killer-robots-mideast/161100/
   Google Scholar
• U.S. Department of Defense. (2018). Unmanned systems integrated roadmap 2017-2042. https://doi.org/https://apps.dtic.mil/dtic/tr/fulltext/u2/1059546.pdf
   Google Scholar
• Walker, F., Wang, J., Martens, M. H., & Verwey, W. B. (2019). Gaze behaviour and electrodermal activity: Objective measures of drivers’ trust in automated vehicles. Transportation Research Part F: Traffic Psychology and Behaviour, 64, 401–412. https://doi.org/10.1016/j.trf.2019.05.021
   Web of Science ®Google Scholar
• Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063.
   PubMed Web of Science ®Google Scholar