Test procedure for evaluating the human–machine interface of vehicles with automated driving systems

Figures & data

Figure 1. User-centered design process based on ISO 9241-210 (ISO 2018).

Figure 2. Steady states and transitions forming the basis of the proposed use case catalogue for evaluating ADS HMIs.

Table 1. Use case set for the assessment of minimum HMI requirements based on NHTSA’s (2017) Federal AV Policy. Note that some use cases might not be applicable if a vehicle is not equipped with a respective system.

Minimum HMI requirement	Use case	Description	Hypotheses
Functioning properly	L3	Steady driving in L3 mode	HMI is capable of communicating the properly functioning automated driving mode
Currently engaged in automated mode	L3 → L2 L2 → L3 L2	Driver voluntarily switches from L3 to L2 Driver voluntarily switches from L2 to L3 Steady driving in L2 mode	HMI is capable of differentiating between L2 and L3 (which has the highest potential for confusion)
Currently unavailable for use	L3unavailable	Driving outside the ODD, L3 is not available; this applies to all lower levels of automation (i.e., L0, L1, L2)	HMI is capable of communicating that L3 cannot be activated when driving outside the ODD
Experiencing a malfunction	L3degraded	Driving inside or outside the ODD, L3 is not available because of a malfunction such as a sensor degradation; this applies to all lower levels of automation (i.e., L0, L1, L2)	HMI is capable of communicating that L3 cannot be activated because of a malfunction
Requesting a control transition from ADS to operator	L3 → L2 L3 → L1 L3 → L0	System-initiated transition to L2 System-initiated transition to L1 either longitudinal or lateral assistance System-initiated transition to L0	HMI is capable of communicating the need for a control transition

Table 2. Checklist items.^a

#	Item
1	Unintentional activation and deactivation should be prevented
2	The system mode should be displayed continuously
3	System state changes should be effectively communicated
4	Visual interfaces used to communicate system states should be mounted at a suitable position and distance. High-priority information should be presented close to the driver’s expected line of sight
5	HMI elements should be grouped together according to their function to support the perception of mode indicators
6	Time-critical interactions with the system should not afford continuous attention
7	The visual interface should have a sufficient contrast in luminance and/or color between foreground and background
8	Texts (e.g., font types and size of characters) and symbols should be easily readable from the permitted seating position
9	Commonly accepted or standardized symbols should be used to communicate the automation mode. Use of nonstandard symbols should be supplemented by additional text explanations or vocal phrase(s)
10	The semantic of a message should be in accordance with its urgency
11	Messages should be conveyed using the language of the users (e.g., national language, avoidance of technical language, use of common syntax)
12	Text messages should be as short as possible
13	Not more than 5 colors should be consistently used to code system states (excluding white and black)
14	The colors used to communicate system states should be in accordance with common conventions and stereotypes
15	Design for color-blindness by redundant coding and avoidance of red/green and blue/yellow combinations
16	Auditory output should raise the attention of the driver without startling her or him or causing pain
17	Auditory and vibrotactile outputs should be adapted to the urgency of the message
18	High-priority messages should be multimodal
19	Warning messages should orient the user toward the source of danger
20	In case of sensor failures, their consequences and required operator steps should be displayed

^aAdapted from Naujoks et al. (2019).

Table 3. Evaluation criteria.

Minimum HMI requirement	Criteria
#1 Functioning properly	At least one mode indicator present in the HMI that shows proper L3 functioning can be identified correctly by the participants No mode indicator belonging to another level of automation is misidentified as an L3 indicator
#2 Currently engaged in ADS mode	Participants are able to correctly differentiate at least one mode indicator that separates L2 and L3 No mode indicator belonging to L2 is misidentified as an L3 indicator Participants can correctly identify when a need for monitoring is indicated
#3 Currently unavailable for use	At least one mode indicator that shows nonavailability of L3 because of driving outside the ODD can be correctly identified A mode indicator that does not show nonavailability of L3 must not be misinterpreted as such Unsuccessful activation attempts are correctly interpreted
#4 Experiencing a malfunction	At least one mode indicator that indicates a malfunction (or a consecutive nonavailability of L3) can be correctly identified A mode indicator that does not show nonavailability of L3 due to a malfunction must not be misinterpreted as such Unsuccessful activation attempts are correctly interpreted
#5 Requesting a control transition from the ADS to the operator	At least one mode indicator present in the HMI that shows the need for a control transition is correctly identified A mode indicator that does not indicate the need for a control transition must not be misinterpreted as such

Naujoks F, Wiedemann K, Schömig N, Hergeth S, Keinath A. Towards guidelines and verification methods for automated vehicle HMIs. Transp Res Part F Traffic Psychol Behav. 2019;60:121–136.

 Web of Science ®Google Scholar