Effects of Non-Speech Auditory Cues on Control Transition Behaviors in Semi-Automated Vehicles: Empirical Study, Modeling, and Validation

ABSTRACT

In semi-automated vehicles, non-speech sounds have been prevalently used as auditory displays for control transitions since these sounds convey urgency well. However, there are no standards of specifications for warning sounds so that diverse non-speech sounds are being employed. To shed light on this, the effects of different non-speech auditory warnings on driver performance were investigated and quantified through the experimental study and human performance modeling approaches. Twenty-four young drivers drove in the driving simulator and experienced both handover and takeover transitions between manual and automated modes while performing a secondary task. The reaction times for handover and takeover, mental workload, and subjective responses were reported. Overall, a traditional warning sound with many repetitions and an indicator sound with decreasing polarity outperformed and were preferred. Additionally, a mathematical model, using the Queuing Network-Model Human Processor (QN-MHP) framework, was applied to quantify the effects of auditory warnings’ acoustic characteristics on drivers’ reaction times in response to takeover request displays. The acoustic characteristics, including the fundamental frequency, the number of repetitions, and the range of dominant frequencies were utilized in modeling. The model was able to explain 99.7% of the experimental data with a root mean square error (RMSE) of 0.148. The present study can contribute to establishing standards and design guidelines for takeover request displays in semi-automated vehicles.

Acknowledgments

This work was partially supported by a grant (code 17TLRP-B131486-01) from Transportation and Logistics R&D Program funded by Ministry of Land, Infrastructure and Transport of Korean government.

Additional information

Notes on contributors

Sangjin Ko

Sangjin Ko received his Bachelor’s degree in Electrical Engineering from Inha University and his MEng degree in Industrial and Systems Engineering from Virginia Tech. While he was at Virginia Tech, he conducted research on automated vehicles and social robots.

Kyle Kutchek

Kyle Kutcheck is an embedded software engineer in DENSO. He received his Bachelor’s degree in Computer Engineering from Michigan Tech in 2018 and his MS degree in Embedded Systems from Oakland University in 2020.

Yiqi Zhang

Yiqi Zhang is an Assistant Professor in Industrial and Manufacturing Engineering at Pennsylvania State University. She received her Ph.D. from University at Buffalo in 2017. Her Human-Technology Interaction Lab conducts research on driver behavior, human performance modeling, trust in technology, situation awareness, individual differences, and automotive interface design.

Myounghoon Jeon

Myounghoon Jeon is an Associate Professor in Industrial and Systems Engineering at Virginia Tech. He received his PhD from Georgia Tech in 2012. His Mind Music Machine Lab conducts research on HCI and HRI with a focus on auditory displays, affective computing, assistive technologies, aesthetic computing, and automotive user interfaces.