Mixed-method usability investigation of ARROWS: augmented reality for roadway work zone safety

References

• CDC. Highway work zone safety [Internet]; 2019 [cited October 12, 2022]. Available from: https://www.cdc.gov/niosh/topics/highwayworkzones/default.html/.
   Google Scholar
• Nnaji C, Gambatese J, Lee HW, et al. Improving construction work zone safety using technology: a systematic review of applicable technologies. J Traffic Transp Eng. English ed. 2020;7(1):61–75. doi:10.1016/j.jtte.2019.11.001
   Google Scholar
• Sakhakarmi S, Park J, Singh A. Tactile-based wearable system for improved hazard perception of worker and equipment collision. Autom Constr. 2021;125:103613. doi:10.1016/j.autcon.2021.103613
   Web of Science ®Google Scholar
• Ahn CR, Lee S, Sun C, et al. Wearable sensing technology applications in construction safety and health. J Constr Eng Manag. 2019;145(11):03119007. doi:10.1061/(ASCE)CO.1943-7862.0001708
   Web of Science ®Google Scholar
• Li X, Yi W, Chi H-L, et al. A critical review of virtual and augmented reality (VR/AR) applications in construction safety. Autom Constr. 2018;86:150–162. doi:10.1016/j.autcon.2017.11.003
   Web of Science ®Google Scholar
• Sabeti S, Shoghli O, Tabkhi H.. Toward Wi-Fi-Enabled Real-Time Communication for Proactive Safety Systems in Highway Work Zones: A Case Study. Arlington, VA: Construction Research Congress; 2022.
   Google Scholar
• Chen Y-J, Lai Y-S, Lin Y-H. BIM-based augmented reality inspection and maintenance of fire safety equipment. Autom Constr. 2020;110:103041. doi:10.1016/j.autcon.2019.103041
   Web of Science ®Google Scholar
• Wu S, Hou L, Zhang GK, et al. Real-time mixed reality-based visual warning for construction workforce safety. Autom Constr. 2022;139:104252. doi:10.1016/j.autcon.2022.104252
   Web of Science ®Google Scholar
• Schein KE, Rauschnabel PA. Augmented reality in manufacturing: exploring workers’ perceptions of barriers. IEEE Transactions on Engineering Management. 2021;70(10):3344–3357. doi: 10.1109/TEM.2021.3093833.
   Web of Science ®Google Scholar
• Savolainen K. User-centred design without involving users: a longitudinal case study in a human-centred-design-mature company. Des J. 2021;24(6):887–905. doi:10.1080/14606925.2021.1980267
   Google Scholar
• Schumann M, Fuchs C, Kollatsch C, et al. Evaluation of augmented reality supported approaches for product design and production processes. Procedia CIRP. 2021;97:160–165. doi:10.1016/j.procir.2020.05.219
   Google Scholar
• Fischer B, Peine A, Östlund B. The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist. 2020;60(7):e513–e523. doi:10.1093/geront/gnz163
   PubMed Web of Science ®Google Scholar
• Schoonderwoerd TA, Jorritsma W, Neerincx MA, et al. Human-centered XAI: developing design patterns for explanations of clinical decision support systems. Int J Hum Comput Stud. 2021;154:102684. doi:10.1016/j.ijhcs.2021.102684
   Web of Science ®Google Scholar
• Kadir BA, Broberg O. Human-centered design of work systems in the transition to Industry 4.0. Appl Ergon. 2021;92:103334. doi:10.1016/j.apergo.2020.103334
   PubMed Web of Science ®Google Scholar
• Ármannsdóttir AL, Beckerle P, Moreno JC, et al. Assessing the involvement of users during development of lower limb wearable robotic exoskeletons: a survey study. Hum Factors. 2020;62(3):351–364. doi:10.1177/0018720819883500
   PubMed Web of Science ®Google Scholar
• Sorensen G, Dennerlein JT, Peters SE, et al. The future of research on work, safety, health and wellbeing: a guiding conceptual framework. Soc Sci Med. 2021;269:113593. doi:10.1016/j.socscimed.2020.113593
   PubMed Web of Science ®Google Scholar
• Sabeti S, Shoghli O, Baharani M, et al. Toward AI-enabled augmented reality to enhance the safety of highway work zones: feasibility, requirements, and challenges. Adv Eng Inf. 2021;50:101429. doi:10.1016/j.aei.2021.101429
   Web of Science ®Google Scholar
• Borges AF, Laurindo FJ, Spínola MM, et al. The strategic use of artificial intelligence in the digital era: systematic literature review and future research directions. Int J Inf Manage. 2021;57:102225. doi:10.1016/j.ijinfomgt.2020.102225
   Web of Science ®Google Scholar
• Ashtari N, Bunt A, McGrenere J, et al. Creating augmented and virtual reality applications: current practices, challenges, and opportunities. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, Hawaii; 2020, April 25–30. p. 1–13. https://doi.org/10.1145/3313831.3376722
   Google Scholar
• Chen H, Hou L, Zhang GK, et al. Development of BIM, IoT and AR/VR technologies for fire safety and upskilling. Autom Constr. 2021;125:103631. doi:10.1016/j.autcon.2021.103631
   Web of Science ®Google Scholar
• Bang J, Lee Y, Lee Y-T, et al. AR/VR based smart policing for fast response to crimes in safe city. In: 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), IEEE, Beijing, China; 2019. p. 470–475. doi:10.1109/ISMAR-Adjunct.2019.00126.
   Google Scholar
• Awolusi I, Marks E, Hallowell M. Wearable technology for personalized construction safety monitoring and trending: review of applicable devices. Autom Constr. 2018;85:96–106. doi:10.1016/j.autcon.2017.10.010
   Web of Science ®Google Scholar
• Akinlolu M, Haupt TC, Edwards DJ, et al. A bibliometric review of the status and emerging research trends in construction safety management technologies. Int J Constr Manag. 2020;22(14):2699–2711. doi:10.1080/15623599.2020.1819584.
   Google Scholar
• Harikrishnan A, Abdallah AS, Ayer SK, et al. Feasibility of augmented reality technology for communication in the construction industry. Adv Eng Inf. 2021;50:101363. doi:10.1016/j.aei.2021.101363
   Web of Science ®Google Scholar
• Mallela J, Gilson K, Goodrum PM, et al. Leveraging augmented reality for highway construction. McLean, VA: Federal Highway Administration; 2020.
   Google Scholar
• Brown H, Sun C, Cope T. Evaluation of mobile work zone alarm systems. Transp Res Rec: J Transp Res Board. 2015;2485:42–50. doi:10.3141/2485-06
   Google Scholar
• Gambatese JA, Lee HW, Nnaji CA, et al. Work zone intrusion alert technologies: assessment and practical guidance. Salem, OR: Oregon Dept. of Transportation Research Section; 2017.
   Google Scholar
• Sakhakarmi S, Park J. Improved intrusion accident management using haptic signals in roadway work zone. J Saf Res. 2022;80:320–329. doi:10.1016/j.jsr.2021.12.015
   PubMed Web of Science ®Google Scholar
• Alfaro-Serrano D, Balantrapu T, Chaurey R, et al. Interventions to promote technology adoption in firms: a systematic review. Campbell Systematic Reviews. 2021;17(4):e1181. doi:10.1002/cl2.1181
   PubMedGoogle Scholar
• Nnaji C, Gambatese J, Karakhan A, et al. Influential safety technology adoption predictors in construction. Eng Constr Archit Manag. 2019;26(11):2655–2681. doi:10.1108/ECAM-09-2018-0381.
   Web of Science ®Google Scholar
• Mack Z, Sharples S. The importance of usability in product choice: a mobile phone case study. Ergon. 2009;52(12):1514–1528. doi:10.1080/00140130903197446
   PubMed Web of Science ®Google Scholar
• Nielsen J. Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems, Boston, MA; 1994, April 24–28. p. 413–414.
   Google Scholar
• Lee SC, Nadri C, Sanghavi H, et al. Eliciting user needs and design requirements for user experience in fully automated vehicles. Int J Hum–Comput Int. 2022;38(3):227–239. doi:10.1080/10447318.2021.1937875
   Web of Science ®Google Scholar
• Turner CW, Lewis JR, Nielsen J. Determining usability test sample size. Int Ency Ergon Hum Factors. 2006;3(2):3084–3088.
   Google Scholar
• Kivijärvi H, Pärnänen K. Instrumental usability and effective user experience: interwoven drivers and outcomes of human–computer interaction. Int J Hum–Comput Int. 2021;39(1):34–51. doi:10.1080/10447318.2021.2016236.
   Web of Science ®Google Scholar
• Luger T, Bär M, Seibt R, et al. Using a back exoskeleton during industrial and functional tasks – effects on muscle activity, posture, performance, usability, and wearer discomfort in a laboratory trial. Hum Factors. 2023;65(1):5–21. doi:10.1177/00187208211007267.
   PubMed Web of Science ®Google Scholar
• Mueller AS, Cicchino JB, Singer J, et al. Effects of training and display content on level 2 driving automation interface usability. Transp Res Part F Traffic Psychol Behav. 2020;69:61–71. doi:10.1016/j.trf.2019.12.010
   Web of Science ®Google Scholar
• Dey A, Billinghurst M, Lindeman RW, et al. A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Front Robot AI. 2018;5:37. doi:10.3389/frobt.2018.00037
   PubMed Web of Science ®Google Scholar
• Ingrassia PL, Mormando G, Giudici E, et al. Augmented reality learning environment for basic life support and defibrillation training: usability study. J Med Internet Res. 2020;22(5):e14910. doi:10.2196/14910
   PubMed Web of Science ®Google Scholar
• Kim S, Nussbaum MA, Gabbard JL. Influences of augmented reality head-worn display type and user interface design on performance and usability in simulated warehouse order picking. Appl Ergon. 2019;74:186–193. doi:10.1016/j.apergo.2018.08.026
   PubMed Web of Science ®Google Scholar
• Tuli N, Mantri A. Evaluating usability of mobile-based augmented reality learning environments for early childhood. Int J Hum–Comput Int. 2021;37(9):815–827. doi:10.1080/10447318.2020.1843888
   Web of Science ®Google Scholar
• Mackintosh KA, Chappel SE, Salmon J, et al. Parental perspectives of a wearable activity tracker for children younger than 13 years: acceptability and usability study. JMIR Mhealth Uhealth. 2019;7(11):e13858. doi:10.2196/13858
   PubMed Web of Science ®Google Scholar
• Keogh A, Argent R, Anderson A, et al. Assessing the usability of wearable devices to measure gait and physical activity in chronic conditions: a systematic review. J Neuroeng Rehabil. 2021;18(1):1–17. doi:10.1186/s12984-020-00774-3
   PubMed Web of Science ®Google Scholar
• Riek LD. Wizard of Oz studies in HRI: a systematic review and new reporting guidelines. J Hum–Robot Interact. 2012;1(1):119–136. doi:10.5898/JHRI.1.1.Riek
   Google Scholar
• Browne JT. Wizard of Oz prototyping for machine learning experiences. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems; 2019, May 4–9, Glasgow, Scotland, UK. p. 1–6. doi:10.1145/3290607.3312877
   Google Scholar
• Schoonderwoerd TA, van Zoelen EM, van den Bosch K, et al. Design patterns for human–AI co-learning: a Wizard-of-Oz evaluation in an urban-search-and-rescue task. Int J Hum Comput Stud. 2022: 102831. doi:10.1016/j.ijhcs.2022.102831
   Web of Science ®Google Scholar
• Aburumman N, Gillies M, Ward JA, et al. Nonverbal communication in virtual reality: nodding as a social signal in virtual interactions. Int J Hum Comput Stud. 2022;164:102819. doi:10.1016/j.ijhcs.2022.102819.
   Web of Science ®Google Scholar
• Faas SM, Stange V, Baumann M. Self-driving vehicles and pedestrian interaction: does an external human–machine interface mitigate the threat of a tinted windshield or a distracted driver? Int J Hum–Comput Int. 2021;37(14):1364–1374. doi:10.1080/10447318.2021.1886483
   Web of Science ®Google Scholar
• Palmeiro AR, van der Kint S, Vissers L, et al. Interaction between pedestrians and automated vehicles: a Wizard of Oz experiment. Transportation Research Part F: Traffic Psychology and Behaviour. 2018;58:1005–1020. doi:10.1016/j.trf.2018.07.020
   Web of Science ®Google Scholar
• Alce G, Wallergård M, Hermodsson K. WozARd: a wizard of Oz method for wearable augmented reality interaction – a pilot study. Adv Hum–Comput Interact. 2015;2015:Article ID 271231. doi:10.1155/2015/271231.
   Google Scholar
• Billah SM, Ashok V, Ramakrishnan I. Write-it-yourself with the aid of smartwatches: a Wizard-of-Oz experiment with blind people. In: 23rd international Conference on Intelligent User Interfaces (IUI '18). Association for Computing Machinery, New York, NY, USA; 2018. p. 427–431. doi:10.1145/3172944.3173005
   Google Scholar
• NVIDIA Jetson Xavier [Internet]; [cited March 12, 2023]. Available from: https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-series/.
   Google Scholar
• Vuzix Blade [Internet]; 2021 [cited August 10, 2022]. Available from: https://www.vuzix.com/products/blade-smart-glasses-upgraded.
   Google Scholar
• Galaxy Active [Internet]; 2021 [cited September 18, 2022]. Available from: https://www.samsung.com/us/mobile/wearables/smartwatches/.
   Google Scholar
• Nielsen J. how-many-test-users [Internet]; 2012 [cited December 1, 2022]. Available from: https://www.nngroup.com/articles/how-many-test-users/.
   Google Scholar
• Hwang W, Salvendy G. Number of people required for usability evaluation: the 10±2 rule. Commun ACM. 2010;53(5):130–133. doi:10.1145/1735223.1735255
   Web of Science ®Google Scholar
• Tarhini A, Hone K, Liu X, et al. Examining the moderating effect of individual-level cultural values on users’ acceptance of e-learning in developing countries: a structural equation modeling of an extended technology acceptance model. Inter Learn Environ. 2017;25(3):306–328. doi:10.1080/10494820.2015.1122635
   Web of Science ®Google Scholar
• Chung JE, Park N, Wang H, et al. Age differences in perceptions of online community participation among non-users: an extension of the technology acceptance model. Comput Human Behav. 2010;26(6):1674–1684. doi:10.1016/j.chb.2010.06.016
   Web of Science ®Google Scholar
• Sun S, Lee PC, Law R, et al. An investigation of the moderating effects of current job position level and hotel work experience between technology readiness and technology acceptance. Int J Hosp Manag. 2020;90:102633. doi:10.1016/j.ijhm.2020.102633
   Web of Science ®Google Scholar
• Porter CE, Donthu N. Using the technology acceptance model to explain how attitudes determine Internet usage: the role of perceived access barriers and demographics. J Bus Res. 2006;59(9):999–1007. doi:10.1016/j.jbusres.2006.06.003
   Web of Science ®Google Scholar
• Peres SC, Pham T, Phillips R. Validation of the system usability scale (SUS) SUS in the wild. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 30–October 4, 2013, San Diego, CA. Los Angeles (CA): SAGE; 2013. Vol. 57. p. 192–196.
   Google Scholar
• Zijlstra F, Van Doorn L. The construction of a scale to measure perceived effort. Cham: University of Technology; 1985.
   Google Scholar
• Zijlstra F. Efficiency in work behaviour: a design approach for modern tools [Ph.D. thesis, Technische Universiteit Delft]; 1993.
   Google Scholar
• Peterson DA, Kozhokar D. Peak-end effects for subjective mental workload ratings. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting,  Austin, TX, USA, October 9–13, 2017; Los Angeles (CA): SAGE; 2017. Vol. 61. p. 2052–2056.
   Google Scholar
• Brooke J. Usability evaluation in industry, chap. SUS: a ‘quick and dirty’ usability scale. London: CRC Press; 1996.
   Google Scholar
• Lewis JR. Measuring user experience with 3, 5, 7, or 11 points: does it matter? Hum Factors. 2021;63(6):999–1011. doi:10.1177/0018720819881312
   PubMed Web of Science ®Google Scholar
• MUTCD T. Manual on uniform traffic control devices. Austin (TX): Texas Department of Transportation; 2006.
   Google Scholar
• Sauro J, Lewis JR. Quantifying the user experience: practical statistics for user research. Morgan Kaufmann; 2016. https://www.sciencedirect.com/book/9780128023082/quantifying-the-user-experience.
   Google Scholar
• HoloLens2. HoloLens2 [Internet]; [cited October 7, 2022]. Available from: https://fieldtech.trimble.com/en/products/mixed-reality/trimble-xr10-with-hololens-2
   Google Scholar
• Acemyan CZ, Kortum P. The relationship between trust and usability in systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Boston, MA, USA, October 22–26, 2012. Los Angeles (CA): SAGE; 2012. Vol. 56. p. 1842–1846.
   Google Scholar
• Flavián C, Guinalíu M, Gurrea R. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inf Manag. 2006;43(1):1–14. doi:10.1016/j.im.2005.01.002
   Web of Science ®Google Scholar
• Salanitri D, Hare C, Borsci S, et al. Relationship between trust and usability in virtual environments: an ongoing study. In: 17th International Conference, HCI International 2015 Los Angeles, CA, USA, August 2–7, 2015, Springer; 2015. p. 49–59.
   Google Scholar
• Ramsden R, Pit S, Colbran R, et al. Development of a framework to promote rural health workforce capability through digital solutions: a qualitative study of user perspectives. Digit Health. 2022;8:20552076221089082. doi:10.1177/20552076221089082
   Web of Science ®Google Scholar
• Yeung B. University, education, technology, and the future of work. In: Canals J, Heukamp F, editors. The Future of Management in an AI World: redefining Purpose and Strategy in the Fourth Industrial Revolution. Cham: Palgrave Macmillan; 2020. doi:10.1007/978-3-030-20680-2_6.
   Google Scholar
• Kamal SA, Shafiq M, Kakria P. Investigating acceptance of telemedicine services through an extended technology acceptance model (TAM). Technol Soc. 2020;60:101212. doi:10.1016/j.techsoc.2019.101212
   Web of Science ®Google Scholar
• Patel V, Chesmore A, Legner CM, et al. Trends in workplace wearable technologies and connected-worker solutions for next-generation occupational safety: health, and productivity. Adv Intell Syst. 2022;4(1):2100099. doi:10.1002/aisy.202100099
   Google Scholar
• Munsinger B, Quarles J. Augmented reality for children in a confirmation task: time, fatigue, and usability. In: Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, Parramatta, NSW, Australia, November 12–15; 2019 Nov. p. 1–5.
   Google Scholar