Do multimodal search cues help or hinder teleoperated search and rescue missions?

References

• Arend, M. G, and J. Müsseler. 2021. “Object Affordances from the Perspective of an Avatar.” Consciousness and Cognition 92: 103133. doi:10.1016/j.concog.2021.103133.
   PubMed Web of Science ®Google Scholar
• Bates, D., M. Mächler, B. Bolker, and S. Walker. 2015. “Fitting Linear Mixed-Effects Models Using lme4.” Journal of Statistical Software 67 (1): v067.i01. doi:10.18637/jss.v067.i01.
   Web of Science ®Google Scholar
• Benz, T. M, and V. Nitsch. 2017. “Using Multisensory Cues for Direction Information in Teleoperation: More is Not Always Better.”. In 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6541–6546). doi:10.1109/icra.2017.7989773.
   Google Scholar
• Benz, T. M. 2019. Multi-Modal User Interfaces in Teleoperation doi:10.18154/RWTH-2019-03285.
   Google Scholar
• Benz, T. M, and V. Nitsch. 2018. “Is Cross-Modal Matching Necessary? A Bayesian Analysis of Individual Reference Cues.” In Haptics: Science, Technology, and Applications. doi:10.1007/978-3-319-93445-7_2.
   Google Scholar
• Brooks, D. J., K. M. Tsui, M. Lunderville, and H. Yanco. 2015. “Methods for Evaluating and Comparing the Use of Haptic Feedback in Human-Robot Interaction with Ground-Based Mobile Robots.” Journal of Human-Robot Interaction 4 (1): 3–29. doi:10.5898/JHRI.4.1.Brooks.
   Google Scholar
• Burke, J. L., Prewett, M. S. Gray, A. A. Yang, L. Stilson, F. R. B. Coovert, M. D., and E. Redden 2006. “Comparing the Effects of Visual-Auditory and Visual-Tactile Feedback on User Performance.” In ICMI ’06: Proceedings of the 8th International Conference on Multimodal Interfaces (pp. 108–117). doi:10.1145/1180995.1181017.
   Google Scholar
• Casper, J, and R. R. Murphy. 2003. “Human-Robot Interactions during the Robot-Assisted Urban Search and Rescue Response at the World Trade Center.” IEEE Transactions on Systems, Man, and Cybernetics, Part B 33 (3): 367–385. doi:10.1109/tsmcb.
   PubMed Web of Science ®Google Scholar
• Chan, A. H. S., E. R. Hoffmann, and J. C. H. Ho. 2019. “Movement Time and Guidance Accuracy in Teleoperation of Robotic Vehicles.” Ergonomics 62 (5): 706–720. doi:10.1080/00140139.2019.1571246.
   PubMed Web of Science ®Google Scholar
• Chen, J. Y. C., E. C. Haas, and M. J. Barnes. 2007. “Human Performance Issues and User Interface Design for Teleoperated Robots.” IEEE Transactions on Systems, Man and Cybernetics, Part C 37 (6): 1231–1245. doi:10.1109/TSMCC.2007.905819.
   Web of Science ®Google Scholar
• Chen, J. Y. C, and M. J. Barnes. 2014. “Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues.” IEEE Transactions on Human-Machine Systems 44 (1): 13–29. doi:10.1109/THMS.2013.2293535.
   Web of Science ®Google Scholar
• Cohen, J. 1992. “A Power Primer.” Psychological Bulletin 112 (1): 155–159. doi:10.1037/0033-2909.112.1.155.
   PubMed Web of Science ®Google Scholar
• Condoluci, M., T. Mahmoodi, E. Steinbach, and M. Dohler. 2017. “Soft Resource Reservation for Low-Delayed Teleoperation over Mobile Networks.” IEEE Access. 5: 10445–10455. doi:10.1109/ACCESS.2017.2707319.
   Web of Science ®Google Scholar
• Cross, M., K. A. McIsaac, B. Dudley, and W. Choi. 2018. “Negotiating Corners with Teleoperated Mobile Robots with Time Delay.” IEEE Transactions on Human-Machine Systems 48 (6): 682–690. doi:10.1109/THMS.2018.2849024.
   Web of Science ®Google Scholar
• Fitts, P. M. 1954. “The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement.” Journal of Experimental Psychology 47 (6): 381–391. doi:https://doi.org/10.1037/h0055392.
   PubMedGoogle Scholar
• Fuller, R. 2005. “Towards a General Theory of Driver Behaviour.” Accident; Analysis and Prevention 37 (3): 461–472. doi:10.1016/j.aap.2004.11.003.
   PubMed Web of Science ®Google Scholar
• Glendon, A. I., S. Clarke, and E. McKenna. 2006. Human Safety and Risk Management. Boca Raton: CRC Press. doi:10.1201/9781420004687.
   Google Scholar
• Gunn, D. V., J. S. Warm, W. T. Nelson, R. S. Bolia, D. A. Schumsky, and K. J. Corcoran. 2005. “Target Acquisition with UAVs: Vigilance Displays and Advanced Cuing Interfaces.” Human Factors 47 (3): 488–497. doi:10.1518/001872005774859971.
   PubMed Web of Science ®Google Scholar
• Hancock, P. A., J. E. Mercado, J. Merlo, and J. B. F. Van Erp. 2013. “Improving Target Detection in Visual Search through the Augmenting Multi-Sensory Cues.” Ergonomics 56 (5): 729–738. doi:10.1080/00140139.2013.771219.
   PubMed Web of Science ®Google Scholar
• Helton, W. S., J. Head, and B. A. Blaschke. 2014. “Cornering Law: The Difficulty of Negotiating Corners with an Unmanned Ground Vehicle.” Human Factors 56 (2): 392–402. doi:10.1177/0018720813490952.
   PubMed Web of Science ®Google Scholar
• Hoffmann, E. R, and C. G. Drury. 2019. “Models of the Effect of Teleoperation Transmission Delay on Robot Movement Time.” Ergonomics 62 (9): 1175–1180. doi:10.1080/00140139.2019.1612954.
   PubMed Web of Science ®Google Scholar
• Hoffmann, E. R, and S. Karri. 2018. “Control Strategy in Movements with Transmission Delay.” Journal of Motor Behavior 50 (4): 398–408. doi:10.1080/00222895.2017.1363700.
   PubMed Web of Science ®Google Scholar
• Hopkins, K., S. J. Kass, L. D. Blalock, and J. C. Brill. 2017. “Effectiveness of Auditory and Tactile Crossmodal Cues in a Dual-Task Visual and Auditory Scenario.” Ergonomics 60 (5): 692–700. doi:10.1080/00140139.2016.1198495.
   PubMed Web of Science ®Google Scholar
• Jones, K. S., B. R. Johnson, and E. A. Schmidlin. 2011. “Teleoperation through Apertures.” Journal of Cognitive Engineering and Decision Making 5 (1): 10–28. doi:10.1177/1555343411399074.
   Google Scholar
• Fox, J, and S. Weisberg. 2018. An R Companion to Applied Regression. Retrieved from https://www.ebook.de/de/product/33769893/john_fox_sanford_weisberg_an_r_companion_to_applied_regression.html
   Google Scholar
• Judd, C. M., J. Westfall, and D. A. Kenny. 2017. “Experiments with More than One Random Factor: designs, Analytic Models, and Statistical Power.” Annual Review of Psychology 68: 601–625. doi:10.1146/annurev-psych-122414-033702.
   PubMed Web of Science ®Google Scholar
• Kahneman, D. 1973. Attention and Effort. New Jersey: Prentice-Hall.
   Google Scholar
• Khasawneh, A., H. Rogers, J. Bertrand, K. C. Madathil, and A. Gramopadhye. 2019. “Human Adaptation to Latency in Teleoperated Multi-Robot Human-Agent Search and Rescue Teams.” Automation in Construction 99: 265–277. doi:10.1016/j.autcon.2018.12.012.
   Web of Science ®Google Scholar
• Koch, I., E. Poljac, H. Müller, and A. Kiesel. 2018. “Cognitive Structure, Flexibility, and Plasticity in Human Multitasking—An Integrative Review of Dual-Task and Task-Switching Research.” Psychological Bulletin 144 (6): 557–583. doi:10.1037/bul0000144.
   PubMed Web of Science ®Google Scholar
• Lathan, C. E, and M. Tracey. 2002. “The Effects of Operator Spatial Perception and Sensory Feedback on Human-Robot Teleoperation Performance.” Teleoperators and Virtual Environments 11 (4): 368–377. doi:10.1162/105474602760204282.
   Web of Science ®Google Scholar
• Lenth, R. 2022. Emmeans: Estimated Marginal Means, Aka Least-Squares Means. R package version 1.8.2. https://CRAN.R-project.org/package=emmeans
   Google Scholar
• Lo, S, and S. Andrews. 2015. “To Transform or Not to Transform: Using Generalized Linear Mixed Models to Analyse Reaction Time Data.” Frontiers in Psychology 6: 1171. doi:10.3389/fpsyg.2015.01171.
   PubMed Web of Science ®Google Scholar
• Lu, S., M. Y. Zhang, T. Ersal, and X. J. Yang. 2019. “Workload Management in Teleoperation of Unmanned Ground Vehicles: Effects of a Delay Compensation Aid on Human Operators’ Workload and Teleoperation Performance.” International Journal of Human–Computer Interaction 35 (19): 1820–1830. doi:10.1080/10447318.2019.1574059.
   Web of Science ®Google Scholar
• Luo, Ruikun, Yifan Wang, Yifan Weng, Victor Paul, Mark J. Brudnak, Paramsothy Jayakumar, Matt Reed, Jeffrey L. Stein, Tulga Ersal, and X. Jessie Yang. 2019. “Toward Real-Time Assessment of Workload: A Bayesian Inference Approach.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 63 (1): 196–200. doi:https://doi.org/10.1177/1071181319631293.
   Google Scholar
• Nelles, J., M. Arend, A. Mertens, A. Henschel, C. Brandl, and V. Nitsch. 2020. “Investigating Influence of Complexity and Stressors on Human Performance during Remote Navigation of a Robot Plattform in a Virtual 3D Maze.”. In 2020 13th International Conference on Human System Interaction (HSI). doi:10.1109/hsi49210.2020.9142671.
   Google Scholar
• Nitsch, V. 2012. Haptic Human-Machine Interaction in Teleoperation Systems. Retrieved from https://www.ebook.de/de/product/19030636/verena_nitsch_haptic_human_machine_interaction_in_teleoperation_systems.html
   Google Scholar
• Nitsch, V, and B. Färber. 2013. “A Meta-Analysis of the Effects of Haptic Interfaces on Task Performance with Teleoperation Systems.” IEEE Transactions on Haptics 6 (4): 387–398. doi:10.1109/TOH.2012.62.
   PubMed Web of Science ®Google Scholar
• Opiyo, S., J. Zhou, E. Mwangi, W. Kai, and I. Sunusi. 2021. “A Review on Teleoperation of Mobile Ground Robots: Architecture and Situation Awareness.” International Journal of Control, Automation and Systems 19 (3): 1384–1407. doi:10.1007/s12555-019-0999-z.
   Web of Science ®Google Scholar
• Pastel, R., J. Champlin, M. Harper, N. Paul, W. Helton, M. Schedlbauer, and J. Heines. 2007. “The Difficulty of Remotely Negotiating Corners.” In Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting 51 (5): 489–493. doi:10.1177/154193120705100513.
   Google Scholar
• Prewett, M. S., Yang, L.Stilson, F. R. B. Gray, A. A. Coovert, M. D. Burke, J.L. R. and Elliot 2006. “The Benefits of Multimodal Information.” In ICMI ’06: Proceedings of the 8th International Conference on Multimodal Interfaces. doi:10.1145/1180995.1181057.
   Google Scholar
• Rogers, H., A. Khasawneh, J. Bertrand, and K. C. Madathil. 2017. “An Investigation of the Effect of Latency on the Operator’s Trust and Performance for Manual Multi-Robot Teleoperated Tasks. In.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 61 (1): 390–394. doi:10.1177/1541931213601579.
   Google Scholar
• Scholcover, F, and D. J. Gillan. 2018. “Using Temporal Sensitivity to Predict Performance under Latency in Teleoperation.” Human Factors 60 (1): 80–91. doi:10.1177/0018720817734727.
   PubMed Web of Science ®Google Scholar
• Shallice, T., P. McLeod, and K. Lewis. 1985. “Isolating Cognitive Modules with the Dualtask Paradigm: Are Speech Perception and Production Separate Processes?” The Quarterly Journal of Experimental Psychology 37 (4): 507–532. doi:10.1080/14640748508400917.
   Web of Science ®Google Scholar
• Storms, J, and D. Tilbury. 2018. “A New Difficulty Index for Teleoperated Robots Driving through Obstacles.” Journal of Intelligent & Robotic Systems 90 (1–2): 147–160. doi:10.1007/s10846-017-0651-1.
   Web of Science ®Google Scholar
• Summala, H. 2007. “Towards Understanding Motivational and Emotional Factors in Driver Behaviour: Comfort through Satisficing.” In Modelling Driver Behaviour in Automotive Environments 2007: 189–207. doi:10.1007/978-1-84628-618-6_11.
   Google Scholar
• Triantafyllidis, E., C. Mcgreavy, J. Gu, and Z. Li. 2020. “Study of Multimodal Interfaces and the Improvements on Teleoperation.” IEEE Access. 8: 78213–78227. doi:10.1109/ACCESS.2020.2990080.
   Web of Science ®Google Scholar
• Ulrich, R, and J. Miller. 1994. “Effects of Truncation on Reaction Time Analysis.” Journal of Experimental Psychology 123 (1): 34–80. doi:10.1037/0096-3445.123.1.34.
   Google Scholar
• Wickens, C. D. 1984. “Processing Resources in Attention.” In Varieties of Attention, edited by R. Parasuraman, 63–102. New York, NY: Academic Press.
   Google Scholar
• Wickens, C. D. 2002. “Multiple Resources and Performance Prediction.” Theoretical Issues in Ergonomics Science 3 (2): 159–177. doi:10.1080/14639220210123806.
   Google Scholar
• Wickens, C. D. 2008. “Multiple Resources and Mental Workload.” Human Factors 50 (3): 449–455. doi:10.1518/001872008x288394.
   PubMed Web of Science ®Google Scholar
• Wickens, C. D. 2021. “Attention: Theory, Principles, Models and Applications.” International Journal of Human–Computer Interaction 37 (5): 403–417. doi:10.1080/10447318.2021.1874741.
   Web of Science ®Google Scholar
• Winck, R. C., S. M. Sketch, E. W. Hawkes, D. L. Christensen, H. Jiang, M. R. Cutkosky, and A. M. Okamura. 2014. “Time-Delayed Teleoperation for Interaction with Moving Objects in Space.” In 2014 IEEE International Conference on Robotics and Automation (ICRA). doi:10.1109/icra.2014.6907736.
   Google Scholar
• Wong, C., E. Yang, X.-T. Yan, and D. Gu. 2017. “An Overview of Robotics and Autonomous Systems for Harsh Environments.” In 2017 23rd International Conference on Automation and Computing (ICAC). doi:10.23919/iconac.2017.8082020.
   Google Scholar
• Yang, E, and M. C. Dorneich. 2017. “The Emotional, Cognitive, Physiological, and Performance Effects of Variable Time Delay in Robotic Teleoperation.” International Journal of Social Robotics 9 (4): 491–508. doi:10.1007/s12369-017-0407-x.
   Web of Science ®Google Scholar
• Zulqarnain, S. Q, and S. Lee. 2021. “Selecting Remote Driving Locations for Latency Sensitive Reliable Teleoperation.” Applied Sciences 11 (21): 9799. doi:10.3390/app11219799.
   Google Scholar