Effects of Task Proficiency and Automation Reliability on Pilot-ePilot Teaming in Intelligent Aircraft Cockpits

References

• Ahlstrom, U., & Friedman-Berg, F. J. (2006). Using eye movement activity as a correlate of cognitive workload. International Journal of Industrial Ergonomics, 36(7), 623–636. https://doi.org/10.1016/j.ergon.2006.04.002
   Web of Science ®Google Scholar
• Al-Shafei, M. (2024). Navigating Human-Chatbot Interactions: An Investigation into Factors Influencing User Satisfaction and Engagement. International Journal of Human–Computer Interaction, 1–18. https://doi.org/10.1080/10447318.2023.2301252
   Web of Science ®Google Scholar
• Andrews, S., & Veldre, A. (2020). Wrapping up sentence comprehension: The role of task demands and individual differences. Scientific Studies of Reading, 25(2), 123–140. https://doi.org/10.1080/10888438.2020.1817028
   Web of Science ®Google Scholar
• Anzanello, M. J., & Fogliatto, F. S. (2011). Learning curve models and applications: Literature review and research directions. International Journal of Industrial Ergonomics, 41(5), 573–583. https://doi.org/10.1016/j.ergon.2011.05.001
   Web of Science ®Google Scholar
• Bentivoglio, A. R., Bressman, S. B., Cassetta, E., Carretta, D., Tonali, P., & Albanese, A. (1997). Analysis of blink rate patterns in normal subjects. Movement Disorders, 12(6), 1028–1034. https://doi.org/10.1002/mds.870120629
   PubMed Web of Science ®Google Scholar
• Bhaskara, A., Duong, L., Brooks, J., Li, R., McInerney, R., Skinner, M., Pongracic, H., & Loft, S. (2021). Effect of automation transparency in the management of multiple unmanned vehicles. Applied Ergonomics, 90, 103243. https://doi.org/10.1016/j.apergo.2020.103243
   PubMed Web of Science ®Google Scholar
• Biswas, R., Veitch, B., & Power, S. D. (2019). Investigation of an EEG-based indicator of skill acquisition as novice participants practice a lifeboat maneuvering task in a simulator. International Journal of Human–Computer Interaction, 36(8), 777–787. https://doi.org/10.1080/10447318.2019.1687235
   Web of Science ®Google Scholar
• Braarud, P. Ø. (2021). Investigating the validity of subjective workload rating (NASA TLX) and subjective situation awareness rating (SART) for cognitively complex human–machine work. International Journal of Industrial Ergonomics, 86(14), 103233. https://doi.org/10.1016/j.ergon.2021.103233
   Google Scholar
• Calhoun, G. (2022). Adaptable (not adaptive) automation: Forefront of human–automation teaming. Human Factors, 64(2), 269–277. https://doi.org/10.1177/00187208211037457
   PubMed Web of Science ®Google Scholar
• Calhoun, G., Bartik, J., Ruff, H., Behymer, K., & Frost, E. (2021). Enabling human-autonomy teaming with multi-unmanned vehicle control interfaces. Human-Intelligent Systems Integration, 3(2), 155–174. https://doi.org/10.1007/s42454-020-00020-0
   Google Scholar
• Chen, Y. C., Lakhmani, S. G., Stowers, K., Selkowitz, A. R., Wright, J. L., & Barnes, M. (2018). Situation awareness-based agent transparency and human-autonomy teaming effectiveness. Theoretical Issues in Ergonomics Science, 19(3), 259–282. https://doi.org/10.1080/1463922X.2017.1315750
   Web of Science ®Google Scholar
• Choung, H., David, P., & Ross, A. (2023). Trust in AI and its role in the acceptance of AI technologies. International Journal of Human–Computer Interaction, 39(9), 1727–1739. https://doi.org/10.1080/10447318.2022.2050543
   Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge.
   Google Scholar
• Cohen, J. R. (2018). The behavioral and cognitive relevance of time-varying, dynamic changes in functional connectivity. NeuroImage, 180(Pt B), 515–525. https://doi.org/10.1016/j.neuroimage.2017.09.036
   PubMedGoogle Scholar
• Ebbatson, M., Harris, D., Huddlestone, J., & Sears, R. (2010). The relationship between manual handling performance and recent flying experience in air transport pilots. Ergonomics, 53(2), 268–277. https://doi.org/10.1080/00140130903342349
   PubMed Web of Science ®Google Scholar
• FAA. (2021). Chapter 8: Airport traffic patterns. In FAA-H-8083-3C airplane flying handbook (pp. 8.1–8.7). Federal Aviation Administration.
   Google Scholar
• Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BF03193146
   Google Scholar
• Faulhaber, A. K., Friedrich, M., & Kapol, T. (2022). Absence of pilot monitoring affects scanning behavior of pilot flying: Implications for the design of single-pilot cockpits. Human Factors, 64(2), 278–290. https://doi.org/10.1177/0018720820939691
   PubMed Web of Science ®Google Scholar
• Fedunov, B. E. (2016). Tactical-level onboard real-time advisory expert systems for manned aircraft as development and maintenance entities. Journal of Computer and Systems Sciences International, 55(4), 579–597. https://doi.org/10.1134/S1064230716040055
   Web of Science ®Google Scholar
• Flathmann, C., Schelble, B. G., Rosopa, P. J., McNeese, N. J., Mallick, R., & Madathil, K. C. (2023). Examining the impact of varying levels of AI teammate influence on human-AI teams. International Journal of Human-Computer Studies, 177(4), 103061. https://doi.org/10.1016/j.ijhcs.2023.103061
   Google Scholar
• Greiser, S., Lantzsch, R., Wolfram, J., Wartmann, J., Müllhäuser, M., Lüken, T., Döhler, H. U., & Peinecke, N. (2015). Results of the pilot assistance system ‘Assisted low-level flight and landing on unprepared landing sites’ obtained with the ACT/FHS research rotorcraft. Aerospace Science and Technology, 45(45), 215–227. https://doi.org/10.1016/j.ast.2015.05.017
   Google Scholar
• Guo, Y., Freer, D., Deligianni, F., & Yang, G. Z. (2022). Eye-tracking for performance evaluation and workload estimation in space telerobotic training. IEEE Transactions on Human-Machine Systems, 52(1), 1–11. https://doi.org/10.1109/THMS.2021.3107519
   Web of Science ®Google Scholar
• Hoff, K. A., & Bashir, M. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570
   PubMed Web of Science ®Google Scholar
• Hou, L., Chi, H. L., Tarng, W., Chai, J., Panuwatwanich, K., & Wang, X. (2017). A framework of innovative learning for skill development in complex operational tasks. Automation in Construction, 83, 29–40. https://doi.org/10.1016/j.autcon.2017.07.001
   Web of Science ®Google Scholar
• Huang, L., Cooke, N. J., Gutzwiller, R. S., Berman, S., Chiou, E. K., Demir, M., & Zhang, W. (2021). Distributed dynamic team trust in human, artificial intelligence, and robot teaming. In C. S. Nam, & J. B. Lyons (Eds.), Trust in Human-Robot Interaction (pp. 301–319). Academic Press.
   Google Scholar
• Kang, H. (2021). Sample size determination and power analysis using the G*Power software. Journal of Educational Evaluation for Health Professions, 18, 17. https://doi.org/10.3352/jeehp.2021.18.17
   PubMed Web of Science ®Google Scholar
• Kaplan, A. D., Kessler, T. T., Brill, J. C., & Hancock, P. A. (2023). Trust in Artificial Intelligence: Meta-analytic findings. Human Factors, 65(2), 337–359. https://doi.org/10.1177/00187208211013988
   PubMed Web of Science ®Google Scholar
• Kenny, B., & Power, S. D. (2021). Toward a subject-independent EEG-based neural indicator of task proficiency during training. Frontiers in Neuroergonomics, 1, 618632. https://doi.org/10.3389/fnrgo.2020.618632
   Google Scholar
• Kim, Y. S. G., Petscher, Y., & Vorstius, C. (2019). Unpacking eye movements during oral and silent reading and their relations to reading proficiency in beginning readers. Contemporary Educational Psychology, 58(6), 102–120. https://doi.org/10.1016/j.cedpsych.2019.03.002
   Google Scholar
• Kita, K., Kato, R., & Yokoi, H. (2010). Evaluation method for the proficiency level of an operating myoelectric hand using EMG signals. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference (vol. 2010, pp. 3373–3376). https://doi.org/10.1109/IEMBS.2010.5627922
   PubMedGoogle Scholar
• Kunde, S., Palmer, E., & Duncan, B. (2022). Recognizing user proficiency in piloting small unmanned aerial vehicles (sUAV). IEEE Robotics and Automation Letters, 7(2), 2345–2352. https://doi.org/10.1109/LRA.2022.3142451
   Web of Science ®Google Scholar
• Lance, C. E., Hedge, J. W., & Alley, W. E. (1989). Joint relationships of task proficiency with aptitude, experience, and task difficulty: A cross-level, interactional study. Human Performance, 2(4), 249–272. https://doi.org/10.1207/s15327043hup0204_2
   Google Scholar
• Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392
   PubMed Web of Science ®Google Scholar
• Lee, J., & Moray, N. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243–1270. https://doi.org/10.1080/00140139208967392
   PubMed Web of Science ®Google Scholar
• Li, M., Kamaraj, A. V., & Lee, J. D. (2023). Modeling trust dimensions and dynamics in human-agent conversation: A trajectory epistemic network analysis approach. International Journal of Human–Computer Interaction, 1–12. https://doi.org/10.1080/10447318.2023.2201555
   Web of Science ®Google Scholar
• Lyons, J. B., Sycara, K., Lewis, M., & Capiola, A. (2021). Human-autonomy teaming: Definitions, debates, and directions. Frontiers in Psychology, 12, 589585. https://doi.org/10.3389/fpsyg.2021.589585
   PubMed Web of Science ®Google Scholar
• Majumdar, S. (2022). U-2S dragon lady: The electronic pilot. Vayu Aerospace and Defence Review, 2, 73–74.
   Google Scholar
• Mansikka, H., Virtanen, K., Harris, D., & Simola, P. (2016). Fighter pilots’ heart rate, heart rate variation and performance during an instrument flight rules proficiency test. Applied Ergonomics, 56, 213–219. https://doi.org/10.1016/j.apergo.2016.04.006
   PubMed Web of Science ®Google Scholar
• May, J. G., Kennedy, R. S., Williams, M. C., Dunlap, W. P., & Brannan, J. R. (1990). Eye movement indices of mental workload. Acta Psychologica, 75(1), 75–89. https://doi.org/10.1016/0001-6918(90)90067-P
   PubMed Web of Science ®Google Scholar
• Moray, N., Inagaki, T., & Itoh, M. (2000). Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology Applied, 6(1), 44–58. https://doi.org/10.1037/1076-898X.6.1.44
   PubMed Web of Science ®Google Scholar
• Muir, B. M. (1987). Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies, 27(5-6), 527–539. https://doi.org/10.1016/S0020-7373(87)80013-5
   Google Scholar
• Muir, B. M. (1994). Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37(11), 1905–1922. https://doi.org/10.1080/00140139408964957
   Web of Science ®Google Scholar
• Muir, B. M., & Moray, N. (1996). Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39(3), 429–460. https://doi.org/10.1080/00140139608964474
   PubMed Web of Science ®Google Scholar
• Myers, P. L., & Starr, A. W. (2021). Single pilot operations IN commercial cockpits: Background, challenges, and options. Journal of Intelligent & Robotic Systems, 102(1), 1–15. https://doi.org/10.1007/s10846-021-01371-9
   Web of Science ®Google Scholar
• Nittala, S. K., Elkin, C. P., Kiker, J. M., Meyer, R., Curro, J., Reiter, A. K., Xu, K. S., & Devabhaktuni, V. K. (2018). Pilot skill level and workload prediction for sliding-scale autonomy. 17th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 1166–1173). https://doi.org/10.1109/ICMLA.2018.00188
   Google Scholar
• Noorunnisa, S., Jarvis, D., Jarvis, J., & Rönnquist, R. (2021). A Conceptual Model for Human-Agent Teams. In G. Jezic, J. Chen-Burger, M. Kusek, R. Sperka, R. J. Howlett, & L. C. Jain, (Eds.), Agents and multi-agent systems: Technologies and applications (pp. 17–26). Springer.
   Google Scholar
• O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2022). Human–autonomy teaming: A review and analysis of the empirical literature. Human Factors, 64(5), 904–938. https://doi.org/10.1177/0018720820960865
   PubMed Web of Science ®Google Scholar
• Olsson-Collentine, A., van Assen, M. A. L. M., & Hartgerink, C. H. J. (2019). The prevalence of marginally significant results in psychology over time. Psychological Science, 30(4), 576–586. https://doi.org/10.1177/0956797619830326
   PubMed Web of Science ®Google Scholar
• Pashakhanlou, A. H. (2019). AI, autonomy, and airpower: The end of pilots? Defence Studies, 19(4), 337–352. https://doi.org/10.1080/14702436.2019.1676156
   Google Scholar
• Ramos, M. A., Sankaran, K., Guarro, S., Mosleh, A., Ramezani, R., & Arjounilla, A. (2023). The need for and conceptual design of an AI model-based Integrated Flight Advisory System. Proceedings of the Institution of Mechanical Engineers, 237(2), 485–507. https://doi.org/10.1177/1748006X221083379
   Google Scholar
• Rodriguez, L. R., Orellana, C. B., Landfair, J., Magaldino, C., Demir, M., Amazeen, P. G., Metcalfe, J. S., Huang, L., & Kang, Y. (2021). Dynamics of trust in automation and interactive decision making during driving simulation tasks. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 65(1), 786–790. https://doi.org/10.1177/1071181321651288
   Google Scholar
• Rohr, J. J., Sater, S., Sass, A. M., Marshall-Goebel, K., Ploutz-Snyder, R. J., Ethier, C. R., Stenger, M. B., Martin, B. A., & Macias, B. R. (2020). Quantitative magnetic resonance image assessment of the optic nerve and surrounding sheath after spaceflight. Npj Microgravity. 6(1), 30. https://doi.org/10.1038/s41526-020-00119-3
   PubMed Web of Science ®Google Scholar
• Schwarz, C., & Hahn, K. U. (2011). Automated pilot assistance for wake vortex encounters. Aerospace Science and Technology, 15(5), 416–421. https://doi.org/10.1016/j.ast.2010.09.008
   Web of Science ®Google Scholar
• Shneiderman, B. (2020a). Bridging the gap between ethics and practice: guidelines for reliable, safe, and trustworthy human-centered AI systems. ACM Transactions on Interactive Intelligent Systems, 10(4), 1–31. https://doi.org/10.1145/3419764
   Web of Science ®Google Scholar
• Shneiderman, B. (2020b). Human-centered artificial intelligence: Reliable, safe & trustworthy. International Journal of Human–Computer Interaction, 36(6), 495–504. https://doi.org/10.1080/10447318.2020.1741118
   Web of Science ®Google Scholar
• Stanton, N. A., Salmon, P., Harris, D., Marshall, A., Demagalski, J., Young, M. S., Waldmann, T., & Dekker, S. (2009). Predicting pilot error: Testing a new methodology and a multi-methods and analysts approach. Applied Ergonomics, 40(3), 464–471. https://doi.org/10.1016/j.apergo.2008.10.005
   PubMed Web of Science ®Google Scholar
• Stout, R. J., Salas, E., & Carson, R. (1994). Individual task proficiency and team process behavior: What’s important for team functioning? Military Psychology, 6(3), 177–192. https://doi.org/10.1207/s15327876mp0603_3
   Google Scholar
• Taylor, R. M. (2011). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. In Salas, E. (Ed.). Situational awareness. Routledge.
   Google Scholar
• Thangavel, K., Sabatini, R., Gardi, A., Ranasinghe, K., Hilton, S., Servidia, P., & Spiller, D. (2024). Artificial intelligence for trusted autonomous satellite operations. Progress in Aerospace Sciences, 144(2), 100960. https://doi.org/10.1016/j.paerosci.2023.100960
   Google Scholar
• Tokadlı, G., Dorneich, M. C., & Matessa, M. (2021). Evaluation of playbook delegation approach in human-autonomy teaming for single pilot operations. International Journal of Human–Computer Interaction, 37(7), 703–716. https://doi.org/10.1080/10447318.2021.1890485
   Web of Science ®Google Scholar
• Van Orden, K. F., Limbert, W., Makeig, S., & Jung, T. P. (2001). Eye activity correlates of workload during a visuospatial memory task. Human Factors, 43(1), 111–121. https://doi.org/10.1518/001872001775992570
   PubMed Web of Science ®Google Scholar
• Veitch, E., & Alsos, O. A. (2022). A systematic review of human-AI interaction in autonomous ship systems. Safety Science, 152, 105778. https://doi.org/10.1016/j.ssci.2022.105778
   Web of Science ®Google Scholar
• Xiong, W., Wang, Y., Zhou, Q., Liu, Z., & Zhang, X. (2016). The research of eye movement behavior of expert and novice in flight simulation of landing. In D. Harris (Ed.), Engineering psychology and cognitive ergonomics (pp. 485–493). Springer.
   Google Scholar
• Yanıkoğlu, Ö., Kılıç, S., & Küçükönal, H. (2020). Gender in the cockpit: Challenges faced by female airline pilots. Journal of Air Transport Management, 86, 101823. https://doi.org/10.1016/j.jairtraman.2020.101823
   Web of Science ®Google Scholar