Is Silent External Human–Machine Interface (eHMI) Enough? A Passenger-Centric Study on Effective eHMI for Autonomous Personal Mobility Vehicles in the Field

Abstract

Autonomous personal mobility vehicle (APMV) is a miniaturized autonomous vehicle designed for short-distance mobility to everyone. Due to its open design, APMV’s passengers are exposed to communications between the external human-machine interface (eHMI) on APMV and pedestrians. Therefore, effective eHMI designs for APMV need to consider potential impacts of APMV-pedestrian interactions on passengers’ subjective feelings. This study from the perspective of APMV passengers discussed three eHMI designs: (1) graphical user interface (GUI)-based eHMI with text message (eHMI-T), (2) multimodal user interface (MUI)-based eHMI with neutral voice (eHMI-NV), and (3) MUI-based eHMI with affective voice (eHMI-AV). In a riding field experiment (N = 24), eHMI-T made passengers feel awkward during the “silent time” when eHMI-T conveyed information exclusively to pedestrians, not passengers. MUI-based eHMIs with voice cues showed advantages, with eHMI-NV excelling in pragmatic quality and eHMI-AV in hedonic quality. Considering passengers’ personalities and genders in APMV eHMI design is also highlighted.

Keywords:

• Autonomous personal mobility vehicles
• human–AV communication
• external human–machine interface (eHMI)
• traffic psychology

1. Introduction

1.1. Autonomous personal mobility vehicle

Autonomous personal mobility vehicle (APMV) is a miniaturized vehicle with automation functions ranging from the SAE levels 3–5 automated driving systems that can decide their own driving behavior without passenger involvement. Currently, most well-known APMVs are indeed developed based on electric wheelchairs¹ or semi-open small vehicles.² It is important to note that these APMVs are not only developed for the elderly people or people with disabilities; they can be used by anyone for short-distance mobility.

As shown in Figure 1, unlike autonomous cars, APMVs can be widely used in shared spaces with other traffic participants, such areas include sidewalks, shopping centers, stations, school campuses, and other mixed traffic areas (Ali et al., 2019; Kobayashi et al., 2013; Liu et al., 2020, 2023), to facilitate the mobility of passengers. As a result, APMVs will inevitably interact with more vulnerable road users like pedestrians without any protection.

Figure 1. An APMV encounters pedestrians in a shared space.

1.2. Interaction issues of APMV in shared spaces

In the aforementioned pedestrian-rich shared spaces, pedestrians may have a low perceived safety when an APMV uses implicit communication, such as changes in maneuvers, to interact with them. For example, Liu et al. (2020, 2023) highlighted that pedestrians, when encountering an APMV, were inclined to perceive danger due to confusion in comprehending the driving intentions of the APMV. Furthermore, we consider that this confusion can result in undesirable consequences, such as lower acceptance levels, or even dangers like crashes. Therefore, it is crucial to explore effective ways of communication between APMVs and pedestrians, in order to accurately convey their intentions to each other and ensure safe and efficient interactions between them. One solution to improve APMV–pedestrian communications is to equip an external human–machine interface (eHMI) on the APMV.

1.3. External human–machine Interface and its challenge in user experience for APMV passengers

Based on the literature review, we found that current eHMI designs and research primarily focus on the interactions between autonomous vehicles (AVs) and pedestrians. These eHMIs often rely on the visual cues through light bars, icons, text, and ground projections (Bazilinskyy et al., 2021; Dey et al., 2021; Li et al., 2021, 2023; Liu et al., 2021). As mentioned in the review article (Brill et al., 2023), auditory-based eHMI has been under-studied within the eHMI literature when compared to vision-based eHMIs. A few works comparing the effects of vision-based and auditory-based eHMIs of AVs on pedestrians’ user experience (UE; Dou et al., 2021) and reactions to warning messages (Ahn et al., 2021), as well as Haimerl et al. (2022) explored the feasibility of auditory eHMI of AVs for pedestrians with intellectual disabilities. Moreover, Kreißig et al. (2023) assess the impact of vision-based and auditory-based eHMIs on the perception of surrounding pedestrians during the autonomous parking of a driver-less E-Cargo bike. To the best of our knowledge, there is presently no research addressing the design of eHMI for APMVs and investigating the UEs of vision-based and auditory-based eHMI for APMVs.

To apply eHMI designs from AVs to APMVs, we must first carefully consider the distinct characteristics that set AVs and APMVs apart. As shown in Figure 1, unlike AV–pedestrian interactions, APMVs have unique characteristics in their interactions with pedestrians: (i) APMVs are often in closer proximity to pedestrians, given their much smaller sizes and lower driving speeds than AVs, especially in indoor areas such as shopping centers and airports, where pedestrians are populated; and (ii) in addition to close proximity, passengers on the APMV are exposed to the communication between the eHMI deployed on APMVs and pedestrians by the characteristic of the open design. Therefore, to ensure an optimal passenger experience, eHMI designs for APMVs must consider the potential impact of APMV–pedestrian communications on passengers’ subjective feelings.

Based on the characteristics mentioned above, a few studies have used eHMI to improve the interactions between pedestrians and APMVs. For example, Watanabe et al. (2015) used a projector to project the trajectory of an APMV’s movement onto the ground, in order to communicate the motion intentions of the APMV to both pedestrians and passengers. Zhang et al. (2022) conducted a series of interviews with wheelchair users, and revealed the existence of two distinct interaction loops, i.e., APMV–passenger and APMV–pedestrian. Moreover, they suggested a range of design concepts, derived from a virtual workshop, for those two interactions, i.e., projecting information, such as planned path, on the ground for APMV–passenger interaction, and showing trajectory on a display and using vibrotactile feedback for APMV–passenger interactions. Furthermore, based on the related works above, we can clearly see that the eHMIs of APMVs could impact the experience of both passengers and pedestrians. However, most of the current eHMI designs for APMVs only aim to improve pedestrian experience, passengers’ experience has not yet received much attention or been widely discussed.

1.4. Purpose and research questions

To address the research gaps mentioned in the previous section, this article explores communication designs in the interactions between APMVs and pedestrians from the perspective of APMV passengers. As the APMV is for single-passenger use, which is highly associated with the passenger’s own preference, we further consider the passenger’s personality for the eHMI designs. Additionally, considering that gender differences may have an impact on user preference for eHMIs (Chang, 2020), we will further discuss the effects of gender differences among passengers on the UE of APMV’s eHMI.

In addition to commonly used graphical user interface (GUI)-based eHMI designs, voice user interface (VUI) can be a useful addition to the GUI-based eHMI (Sodnik et al., 2008). Furthermore, considering the perspective of APMV passengers, compared to only GUI-based eHMIs, the voice cues from multimodal user interface (MUI)-based eHMIs can provide confirmative feedback to passengers about the APMV’s communications with pedestrians, avoiding feelings of ignorance. The voice cues can also serve as an alternative communication channel to broadcast the APMV’s intentions to other road users in the vicinity who may be visually occupied.

Therefore, we explored MUI-based eHMIs that incorporate both GUI and verbal message-based VUI on APMVs to communicate with pedestrians. To the best of our knowledge, we are the first to conduct a field study investigating UE of APMV passengers using MUI-based eHMIs with voice cues.

To sum up, in this article, we aim to answer the following three questions through a field study:

• RQ 1: To what extent does the silent GUI-based eHMI for AVs apply to APMVs in terms of passenger’s UE?

• RQ 2: To what extent do the voice cues of MUI-based eHMI apply to APMVs in terms of passenger’s UE?

• RQ 3: Does APMV’s eHMI design need to fit the passenger’s personality?

2. Method

We conducted a passenger-centric experiment using a robotic wheelchair as the APMV. The purpose of the experiment was to investigate the impact of different visual and auditory eHMIs on passengers’ subjective feelings when the APMV communicates with pedestrians. This study has been carried out with the approval of the Research Ethics Committee of Nara Institute of Science and Technology (NAIST) [No. 2022-I-55].

2.1. Participants

A total of 24 participants (12 males and 12 females) with an age range of 22 to 29 years (mean: 23.9, standard deviation: 1.7) participated in this experiment as APMV passengers. All participants reported no previous experience with eHMI and APMVs. It took about one hour for each participant to complete the experiment.

Each participant received 1,000 Japanese Yen as reward.

2.2. Autonomous personal mobility vehicle

A robotic wheelchair WHILL Model CR with an autonomous driving system was used as the APMV. As shown in Table 1, the APMV was equipped with a LiDAR (Velodyne VLP-16) and a controlling PC allowing it to drive autonomously on a pre-designed route. In this experiment, its maximum speed was limited to 1 m/s for safety considerations.

Table 1. The basic design of APMV’s eHMI including LED lights and a display is used to show its driving behaviors.

Moving	Decelerating	Stopping	Turning left	Turning right

2.3. The eHMI for visualizing APMV’s driving behaviors

Visualizing APMV’s driving behaviors can be considered a method of transforming implicit communication into explicit communication in pedestrian–APMV interactions. This approach represents a non-directed information provision method, making it easy for enabling multiple pedestrians around APMV to comprehend the driving behaviors of the APMV. To implement this visualization method on APMVs, as shown in Table 1, the APMV is equipped with a GUI to show the vehicle’s driving behaviors via two parts: LED lights on the sides and chassis, and a display above the bodywork.

According to the eHMI design concept in study (Li et al., 2021), the chassis is equipped with LED lights that show green to move, yellow to decelerate, and red to stop by projecting the respective colored lights onto the ground. In addition, the yellow LED lights on both sides of the APMV’s body and wheels are used as turn signals.

In addition, cartoonish eyes are shown in the display to mimic the observational behaviors of a human driver driving a vehicle. The eye color is designed to be cyan, showing the autonomous driving mode is ON. In particular, when turning, the eyes look in the corresponding direction. Also, the eyes show a relaxing gaze when stopping, a serious gaze when moving forward, and a concentrated gaze when slowing down.

2.4. Three eHMIs for communicating with pedestrians

In situations where the APMV interacts with specific pedestrians, both the directionality and content of information cues are crucial for their communication and negotiation. Therefore, in the design concept of our APMV’s eHMI, we convey the direction of information delivery through the gaze direction of the eyes on the eHMI screen and convey the content of the information using voice or text to ensure clear communication.

In this study, one GUI-based eHMI and two MUI-based eHMIs are designed for the APMV to communicate with a pedestrian while they encounter each other. In particular, they are (a) GUI-based eHMI with text messages (eHMI-T), (b) MUI-based eHMI with neutral voice (eHMI-NV), and (c) MUI-based eHMI with affective voice (eHMI-AV) as shown in Table 2. The eHMI-T uses a smiley face together with a piece of text to indicate its yielding intention, and uses grateful eyes together with a piece of text to express its appreciation, i.e., thanks. Both the eHMI-NV and eHMI-AV have the same GUI design for their eHMI as that for the eHMI-T, but they differ in their VUIs. To be more specific, the eHMI-NV utilizes a neutral voice³ to say “After you” and “Thank you” to pedestrians, while the eHMI-AV utilizes an affective voice (see note 3) to express “Oh please, after you!” and “You are so kind. Thank you so much!”. Both of the voice cues are in Japanese as all participants are native Japanese speakers.

Table 2. Three types of eHMI designs to communicate with pedestrians.

Note that although the basic eHMI can be activated automatically by the APMV’s movements (see Table 1), to ensure the performance for communication with pedestrians, a trained experimenter who followed the APMV remotely controlled the eHMI and activated the designated GUI and VUIs at a certain moment.

2.5. Driving scenarios

For our experimental site, we selected an indoor 55 m × 30 m area, as shown in Figure 2. It is a walkway located on the ground floor of the Information Science Complex, at NAIST. An approximately 150 m long circular driving route has been established at the experimental site. This site was chosen based on a common usage scenario in which APMVs drove in a shared indoor area and frequently encountered pedestrians. Of these, four specific pedestrians in the interaction scenes were acted by trained actors. It should be noted that the APMV serves as the subject of communication with pedestrians. Therefore, pedestrians (actors) were instructed to face the APMV (i.e., the eyes on the eHMI display of the APMV) to communicate.

Figure 2. The experimental site is a 55 m × 30 m indoor area. In each round, the AVMP’s driving route (red line) encounters with pedestrians (actors) four times at the marked positions.

In this site, four similar encounter scenes have been designed along this route (see Figure 2). The specific interaction scenario at each scene is set up as an APMV and a pedestrian encounter at an intersection with a blind spot/corner. The APMV stops upon encountering the pedestrian, and likewise, the pedestrian stops synchronously as soon as he/she sees the APMV (see Figure 3(a)). Then, the APMV first indicates its intention to yield to the pedestrian, communicated by the eHMI (see Figure 3(b)). Afterwards, the pedestrian faces to the eye animation showing on the eHMI display and says “No, after you” to the APMV (see Figure 3(c)). Finally, the APMV expresses its thanks to the pedestrian via the eHMI (see Figure 3(d)) and then departs (see Figure 3(e)). During the whole encounter scenes, passengers (participants) were asked to ride the APMV in a natural and relaxed way and they were not required to perform non-driving related tasks (NDRTs). Furthermore, there were no restrictions on passengers while they were communicating spontaneously with pedestrians.

Figure 3. An APMV encounters a pedestrian in an indoor environment and communicates through eHMIs.

2.6. Procedure

Firstly, participants were informed about the experiment content, i.e., as a passenger in the APMV to experience the communication between the APMV and pedestrians through different eHMI designs. Meanwhile, the three eHMI designs (see Tables 1 and 2) were illustrated in detail through a demonstration. To alleviate any restlessness or nervousness among participants who are unfamiliar with the APMV, we provided an explanation of the principles behind the autonomous driving system and its sensors. Subsequently, we introduced the questionnaire (see Section 3.1) that participants would need to respond during the experiment. After signing the informed consent form, participants were instructed to complete a personality inventory before starting the experiment.

A schematic diagram of the experimental process is shown in Figure 4. During the experiment, participants (passengers) sat on the APMV, which autonomously drove around the route (see Figure 2) for three times with a different eHMI in each round, respectively. It should be noted that the experience order of the three eHMIs and gender of participants were balanced (see Table 3). Namely, there were six different orders to experience the three eHMIs, and each order was experimented with two participants. As there are four encounter scenes per round, each participant experienced a total of 12 encounter scenes while riding the APMV as a passenger. After each encounter scene, participants were required to complete a questionnaire (see Section 3.1) about their subjective feelings of the communication between the APMV (eHMI) and the pedestrians, as perceived from the passenger’s perspective. After each round (including four encounter scenes), participants were asked to complete the UE questionnaire (see Section 3.2) to assess their experience with the eHMI used. After all three rounds of experiencing all eHMIs, participants were asked to answer a final question “Which eHMI is your favorite?” and give a short open interview. In the open interview, passengers were free to share their experiential feelings about these three eHMIs and provide the reasons behind their responses.

Figure 4. A schematic diagram of the experimental process.

Table 3. Order of eHMI experience for the 24 participants in the experiment.

	Order of experience with eHMIs
Participants (N = 24)	1st eHMI		2nd eHMI		3rd eHMI
Male: N = 2, female: N = 2	eHMI-T	→	eHMI-NV	→	eHMI-AV
Male: N = 2, female: N = 2	eHMI-T	→	eHMI-AV	→	eHMI-NV
Male: N = 2, female: N = 2	eHMI-NV	→	eHMI-T	→	eHMI-AV
Male: N = 2, female: N = 2	eHMI-NV	→	eHMI-AV	→	eHMI-T
Male: N = 2, female: N = 2	eHMI-AV	→	eHMI-T	→	eHMI-NV
Male: N = 2, female: N = 2	eHMI-AV	→	eHMI-NV	→	eHMI-T

3. Measurement

3.1. Subjective evaluation after each encounter scene

After each encounter scene, passengers were asked to answer five subjective questions about how they felt in the encounter scene using the five-point Likert scale, i.e., 1 = “strongly disagree,” 2 = “disagree,” 3 = “neutral,” 4 = “agree,” and 5 = “strongly agree.” The five questions were asked to be answered from a passenger’s view which are:

• Q1: Was the information conveyed by eHMI insufficient for you?

• Q2: Was the content of the communication between the eHMI and pedestrians easy to understand?

• Q3: Did the APMV excessively communicate with pedestrians via the eHMI?

• Q4: Did you feel awkward during the APMV–pedestrian communication?

• Q5: Did you worry that the eHMI’s performance might attract too much attention from people around you?

Among them, Q1–Q3 are used to evaluate whether the information conveyed by eHMI can appropriately help passengers achieve situational awareness. Q4 and Q5 are used to evaluate the UE of passengers during the APMV–pedestrian communication.

3.2. User experience

The short version of the UE questionnaire (UEQ-S; Schrepp et al., 2017) in Japanese⁴ was used to rate the passengers’ experience with each design of the eHMIs. As shown in Figure 5, eight items in UEQ-S were analyzed in two UE domains, which were counted by: $$\text{Pragmatic Quality}=\sum _{i=1}^4(\text{UEQ} \text{item} i)/4;$$ $$\text{Hedonic Quality}=\sum _{i=5}^8(\text{UEQ} \text{Item} i)/4.$$

Figure 5. The UEQ-S. Note that the Japanese version was used.

3.3. Favorite eHMI

We analyzed the preference of the three eHMIs in relation to the participants’ personality domains. Specifically, the participants were divided into three groups based on their responses to the final question “Which eHMI is your favorite?” The mean of the five personality domains was compared across the participants in each group.

3.4. Big 5 personality domains

To find the relationship between the participants’ choice of the favorite eHMI and their personality inventory, the Japanese version (Oshio et al., 2012) of the Ten Item Personality Inventory (TIPI; Gosling et al., 2003) was used. As shown in Table 4, these ten items were rated by the participants using a seven-point scale.

Table 4. Items of TIPI (Gosling et al., 2003).

	Scores
	1	2	3	4	5	6	7
I see myself as:	Disagree strongly	Disagree moderately	Disagree a little	Neither agree nor disagree	Agree a little	Agree moderately	Agree strongly
TIPI item 1: extraverted, enthusiastic.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 2: critical, quarrelsome.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 3: dependable, self-disciplined.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 4: anxious, easily upset.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 5: open to new experiences, complex.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 6: reserved, quiet.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 7: sympathetic, warm.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 8: disorganized, careless.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 9: calm, emotionally stable.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$
TIPI item 10: conventional, uncreative.	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$	$\square$

As outlined by Oshio et al. (2012), the Big 5 personality domains were derived from the scores of TIPI items by aggregating the rating of the corresponding positive item and the residual of the negative item, as following: $$\text{Extraversion}=\text{TIPI} \text{item} \text{1+}(8-\text{TIPI} \text{item} 6),$$ $$\text{Agreeableness}=(8-\text{TIPI} \text{item} 2)+\text{TIPI} \text{item} 7,$$ $$\text{Conscientiousness}=\text{TIPI} \text{item} \text{3+}(8-\text{TIPI} \text{item} 8),$$ $$\text{Neuroticism}=\text{TIPI} \text{item} \text{4+}(8-\text{TIPI} \text{item} 9),$$ $$\text{Openness to Experience}=\text{TIPI} \text{item} \text{5+}(8-\text{TIPI} \text{item} 10).$$

Hence, the resulting rating for each personality domain is represented by a discrete value between 1 and 14.

4. Results

4.1. Subjective evaluations after each encounter scene

A summary of the answers to the five subjective evaluations after each encounter scene, i.e., Q1–Q5 described in Section 3.1, reported by the 24 participants is shown in Figure 6. As shown in Table 5, a two-way mixed analysis of variance (ANOVA) revealed that there was no a statistically significant interaction between the effects of gender and eHMI on Q1–Q5, respectively. Additionally, it showed that a main effect of gender was significant on Q5 (p < .05) only, indicating that females were significantly more concerned about eHMI attracting too much attention from people around them than males. Moreover, the two-way mixed ANOVA also reported that main effects of eHMI were significant on Q1–Q5 (p < .001), respectively.

Figure 6. Subjective evaluation results of five questions after each encounter scene, where 1 = “strongly disagree,” 2 = “disagree,” 3 = “neutral,” 4 = “agree,” and 5 = “strongly agree.”.

Table 5. Two-way mixed ANOVA for each subjective evaluation question.

Questions	Source	Sum of squares	dof1	dof2	Mean square	F-Value	p-Value	${\eta }_p^2$
Q1	Gender	0.001	1	22	0.001	0.001	.976	0.000
	eHMI	74.043	2	44	37.022	64.816	$<.001$***	0.747
	Gender × eHMI	0.366	2	44	0.183	0.321	.727	0.014
Q2	Gender	0.003	1	22	0.003	0.004	.950	0.000
	eHMI	107.220	2	44	53.610	136.115	$<.001$***	0.861
	Gender × eHMI	0.283	2	44	0.141	0.359	.700	0.016
Q3	Gender	2.084	1	22	2.084	2.517	.127	0.103
	eHMI	46.568	2	44	23.284	36.449	$<.001$***	0.624
	Gender × eHMI	2.950	2	44	1.475	2.309	.111	0.095
Q4	Gender	1.188	1	22	1.188	0.465	.502	0.021
	eHMI	19.000	2	44	9.500	9.974	$<.001$***	0.312
	Gender × eHMI	0.715	2	44	0.358	0.375	.689	0.017
Q5	Gender	11.681	1	22	11.681	6.573	.018*	0.230
	eHMI	17.382	2	44	8.691	11.170	$<.001$***	0.337
	Gender × eHMI	3.465	2	44	1.733	2.227	.120	0.092

*p < .05.

***p < .001.

Post hoc pairwise comparisons of eHMIs for Q1–Q5 using the Wilcoxon signed-rank test with BH-FDR correction are presented in Table 6. For Q1, passengers perceived that eHMI-T conveyed information significantly insufficient compared to eHMI-NV (p < .001) and eHMI-AV (p < .001). Similarly, from the results of Q2, passengers agreed that the communication contents were significantly easier to understand when using eHMI-NV (p < .001) and eHMI-AV (p < .001) compared to using eHMI-T. However, in Q3, passengers also significantly agreed that eHMI-AV exhibited excessive communication compared to eHMI-T (p < .001) and eHMI-NV (p < .001). The comparison of results from Q4 indicated that passengers felt significantly more discomfort when experiencing both eHMI-T (p < .01) and eHMI-AV (p < .01) compared to using eHMI-NV. Regarding Q5, passengers perceived that when experiencing eHMI-AV, they significantly worried about attracting too much attention from people compared to eHMI-T (p < .01) and eHMI-NV (p < .01).

Table 6. Post hoc two-sided pairwise comparisons using the Wilcoxon signed-rank test with Benjamini/Hochberg FDR correction for main effect of eHMI for each subjective evaluation question.

Questions	A	B	Mean(A)	Std(A)	Mean(B)	Std(B)	W-Value	p-Adj	Hedges’ g
Q1	eHMI-T	eHMI-NV	3.479	1.058	1.344	0.725	0.000	$<.001$***	2.316
	eHMI-T	eHMI-AV	3.479	1.058	1.312	0.586	0.000	$<.001$***	2.492
	eHMI-NV	eHMI-AV	1.344	0.725	1.312	0.586	24.500	.798	0.047
Q2	eHMI-T	eHMI-NV	2.115	0.997	4.719	0.614	0.000	$<.001$***	−3.094
	eHMI-T	eHMI-AV	2.115	0.997	4.688	0.456	0.000	$<.001$***	−3.264
	eHMI-NV	eHMI-AV	4.719	0.614	4.688	0.456	20.500	.503	0.057
Q3	eHMI-T	eHMI-NV	1.281	0.558	1.396	0.546	35.500	.505	−0.204
	eHMI-T	eHMI-AV	1.281	0.558	3.042	1.274	16.000	$<.001$***	−1.761
	eHMI-NV	eHMI-AV	1.396	0.546	3.042	1.274	0.000	$<.001$***	−1.652
Q4	eHMI-T	eHMI-NV	3.656	1.318	2.406	1.075	20.000	.003**	1.022
	eHMI-T	eHMI-AV	3.656	1.318	3.156	1.206	80.000	.134	0.389
	eHMI-NV	eHMI-AV	2.406	1.075	3.156	1.206	31.000	.009**	−0.646
Q5	eHMI-T	eHMI-NV	1.896	0.918	1.938	1.046	60.000	.696	−0.042
	eHMI-T	eHMI-AV	1.896	0.918	2.958	1.382	27.000	.006**	−0.891
	eHMI-NV	eHMI-AV	1.938	1.046	2.958	1.382	31.500	.006**	−0.819

**p < .01.

***p < .001.

4.2. User experience

The results of the 24 participants answering the UEQ-S for three eHMIs were calculated into two UE domains, i.e., pragmatic quality and hedonic quality. Cronbach’s alpha in pragmatic quality items, i.e., UEQ item 1–4, is 0.78, and Cronbach’s alpha in hedonic quality items, i.e., UEQ item 5–8, is 0.92.

The evaluation results of UE for pragmatic quality and hedonic quality of three eHMIs by male and female participants are shown in Figure 7. The average values of pragmatic quality and hedonic quality were calculated as the overall UE results, which are shown in Figure 7. Background colors of each domain in Figure 7 represent the UE benchmarks obtained from a dataset including over 400 studies that used the UEQ to evaluate different products (Hinderks et al., 2018).

Figure 7. Results of UEQ-S for three types of eHMIs with the UEQ benchmark. Error bars show their confidence intervals.

The two-way mixed ANOVA results in Table 7 indicated significant effects of eHMI on various aspects of UE. Specifically, eHMI significantly influenced the pragmatic quality (p < .001), hedonic quality (p < .001), and the overall UE (p < .001). In terms of gender, the main effect was significant only on the hedonic quality (p < .05), indicating that the influence of eHMI on the hedonic quality of UE may also depending on the gender. The two-way mixed ANOVA further revealed a significant interaction between the effects of gender and eHMI on the pragmatic quality (p < .05) and the overall UE (p < .05). However, this interaction was not significant for the hedonic quality.

Table 7. Two-way mixed ANOVA for UE domains.

UE domains	Source	Sum of squares	dof1	dof2	Mean square	F-Value	p-Value	${\eta }_p^2$
Pragmatic quality	Gender	0.170	1	22	0.170	0.128	.724	0.006
	eHMI	22.507	2	44	11.253	15.820	$<.001$***	0.418
	Gender × eHMI	6.778	2	44	3.389	4.764	.013*	0.178
Hedonic quality	Gender	14.897	1	22	14.897	4.509	.045*	0.170
	eHMI	63.630	2	44	31.815	33.822	$<.001$***	0.606
	Gender × eHMI	1.898	2	44	0.949	1.009	.373	0.044
Overall	Gender	2.971	1	22	2.971	2.814	.108	0.113
	eHMI	28.855	2	44	14.427	38.593	$<.001$***	0.637
	Gender × eHMI	3.603	2	44	1.801	4.819	.013*	0.180

*p < .05.

***p < .001.

Table 8 presents the simple main effects of gender within eHMIs on the pragmatic quality and overall UE by using two-sided t-tests with Holm correction. For all eHMIs, no significant differences were found in the pragmatic quality and overall UE due to gender differences.

Table 8. Simple main effects of gender within eHMIs by t-tests (two-sided) with Holm correction for UE domains.

UE domains	eHMIs	A	B	Mean(A)	Std(A)	Mean(B)	Std(B)	T-Value	dof	p-Adj	Cohen’s d
Pragmatic quality	eHMI-T	Female	Male	1.354	1.014	0.729	1.463	1.216	22	.474	0.496
	eHMI-NV	Female	Male	2.375	0.549	2.417	0.557	−0.185	22	.855	−0.075
	eHMI-AV	Female	Male	1.104	1.180	1.979	0.579	−2.307	22	.093	−0.942
Overall	eHMI-T	Female	Male	0.802	0.712	−0.042	1.084	1.216	22	.103	0.920
	eHMI-NV	Female	Male	1.948	0.540	1.365	0.842	−0.185	22	.111	0.825
	eHMI-AV	Female	Male	1.677	0.698	1.885	0.662	−2.307	22	.461	−0.306

Table 9 presents the simple main effects of eHMIs within gender on the pragmatic quality and overall UE by using by two-sided paired t-tests with Holm correction. For the pragmatic quality, both females and males rated eHMI-NV significantly higher than eHMI-T (female: p < .05, male: p < .01), as well as eHMI-NV significantly higher than eHMI-AV (female: p < .05, male: p < .05). However, a gender difference was observed in the pragmatic quality of eHMI-AV and eHMI-T. Only males rated eHMI-AV significantly higher than eHMI-T (p < .05) in terms of pragmatic quality, while females did not perceive a significant difference between the two. In terms of overall UE, both males and females rated eHMI-T significantly lower than eHMI-NV (female: p < .001, male: p < .001) and eHMI-AV (female: p < .05, male: p < .001). However, the significant difference in overall UE between eHMI-NV and eHMI-AV was only observed in males (p < .05), not in females.

Table 9. Simple main effects of eHMIs within gender by paired t-tests (two-sided) with Holm correction for UE domains.

UE domains	Genders	A	B	Mean(A)	Std(A)	Mean(B)	Std(B)	T-Value	dof	p-Adj	Cohen’s d
Pragmatic quality	Female	eHMI-AV	eHMI-NV	1.104	1.180	2.375	0.549	−3.302	11	.022*	−1.381
		eHMI-AV	eHMI-T	1.104	1.180	1.354	1.014	−0.504	11	.624	−0.227
		eHMI-NV	eHMI-T	2.375	0.549	1.354	1.014	3.431	11	.022*	1.252
	Male	eHMI-AV	eHMI-NV	1.979	0.579	2.417	0.557	−3.339	11	.022*	−0.770
		eHMI-AV	eHMI-T	1.979	0.579	0.729	1.463	3.927	11	.012*	1.123
		eHMI-NV	eHMI-T	2.417	0.557	0.729	1.463	5.089	11	.002**	1.524
Overall	Female	eHMI-AV	eHMI-NV	1.677	0.698	1.948	0.540	−1.309	11	.217	−0.434
		eHMI-AV	eHMI-T	1.677	0.698	0.802	0.712	2.843	11	.048*	1.241
		eHMI-NV	eHMI-T	1.948	0.540	0.802	0.712	5.714	11	$<.001$***	1.814
	Male	eHMI-AV	eHMI-NV	1.885	0.662	1.365	0.842	2.832	11	.048*	0.688
		eHMI-AV	eHMI-T	1.885	0.662	−0.042	1.084	5.856	11	$<.001$***	2.146
		eHMI-NV	eHMI-T	1.365	0.842	−0.042	1.084	6.051	11	$<.001$***	1.449

*p < .05.

**p < .01.

***p < .001.

The post hoc pairwise comparisons for main effect of eHMIs, conducted using two-sided t-tests with Holm correction across different UE domains, are detailed in Table 10. In which, eHMI-NV significantly outperforms both eHMI-T (p < .001) and eHMI-AV (p < .001) in the pragmatic quality, but there is no significant difference between eHMI-T and eHMI-AV. For hedonic quality, eHMI-T significantly underperforms both eHMI-NV (p < .001) and eHMI-AV (p < .001), while eHMI-AV significantly outperforms eHMI-NV (p < .001). When considering the overall UE, taking into account both pragmatic quality and hedonic quality, eHMI-T significantly underperforms both eHMI-NV (p < .001) and eHMI-AV (p < .001). However, there is no significant difference between eHMI-NV and eHMI-AV.

Table 10. Post hoc pairwise comparisons by paired t-tests (two-sided) with Holm correction for UE domains.

UE domains	A	B	Mean(A)	Std(A)	Mean(B)	Std(B)	T-Value	dof	p-Adj	Cohen’s d
Pragmatic quality	eHMI-AV	eHMI-NV	1.542	1.013	2.396	0.541	−3.937	23	$<.001$***	−1.052
	eHMI-AV	eHMI-T	1.542	1.013	1.042	1.272	1.525	23	.141	0.435
	eHMI-NV	eHMI-T	2.396	0.541	1.042	1.272	5.922	23	$<.001$***	1.385
Hedonic quality	eHMI-AV	eHMI-NV	2.021	1.096	0.917	1.542	4.096	23	$<.001$***	0.825
	eHMI-AV	eHMI-T	2.021	1.096	−0.281	1.453	6.683	23	$<.001$***	1.789
	eHMI-NV	eHMI-T	0.917	1.542	−0.281	1.453	5.716	23	$<.001$***	0.800
Overall	eHMI-AV	eHMI-NV	1.781	0.674	1.656	0.753	0.788	23	.438	0.175
	eHMI-AV	eHMI-T	1.781	0.674	0.380	0.995	5.693	23	$<.001$***	1.649
	eHMI-NV	eHMI-T	1.656	0.753	0.380	0.995	8.366	23	$<.001$***	1.446

***p < .001.

4.3. Favorite eHMI

After experiencing all eHMIs, participants were asked to select the favorite eHMI. Figure 8 shows the number of female, male as well as all participants who selected each type of favorite eHMI. Fisher’s exact test (two-sided) was used to determine if there was a significant association between two genders and three favorite eHMIs. Its result indicated that there was not a statistically significant association between the favorite eHMIs and genders (p = 0.680).

Figure 8. Numbers of male, female as well as all participants who selected each type of favorite eHMI.

4.4. Big 5 personality domains

The results of the Big 5 personality domains for 12 male and 12 female participants are shown in Figure 9. For each personality domain, a t-test (two-sided) was used to test for differences between male and female. Table 11 shows that there were no significant differences between males and females in all five personality domains.

Figure 9. Distribution of the Big 5 personality domains for male and female participants. Dots indicate mean scores and error bars indicate standard deviations.

Table 11. The Big 5 personality domains of 12 male and 12 female participants.

	Female		Male		t-Test (female,male)
Personality domains	Mean	Std.	Mean	Std.	T-Value	dof	p-Value	BF10	Hedges’ g
Extraversion	10.333	2.348	8.750	3.251	1.368	22	.185	0.729	0.539
Agreeableness	9.417	2.314	10.750	2.094	−1.480	22	.153	0.814	−0.583
Conscientiousness	6.500	2.680	7.583	2.678	−0.990	22	.333	0.533	−0.390
Neuroticism	7.583	2.314	7.417	3.118	0.149	22	.883	0.376	0.059
Openness to experience	9.500	3.060	10.000	2.374	−0.447	22	.659	0.402	−0.176

A two-sided t-test was used to examine each personality domain between genders.

4.5. Relation between the favorite eHMI and personality

In Sections 4.3 and 4.4, no significant differences were observed between males and females in their choice of favorite eHMI and their Big 5 personality domains. Therefore, in this section, the analysis of the relationship between participants’ choice of favorite eHMI and their personality domains did not take into account gender differences.

Figure 10 shows the differences of the Big 5 personality domains of the participants based on their favorite eHMI. In which, participants with lower scores in extraversion, openness to experience and conscientiousness tended to favor eHMI-T, while those with higher scores in those domains tended to prefer eHMI-AV.

Figure 10. Big 5 personality domains of the participants based on their favorite eHMI. The solid line represents the mean, while the dashed line represents the standard deviation.

It is worth noting that although the average neuroticism of passengers who chose eHMI-T is between the average neuroticism of people who chose eHMI-AV and eHMI-NV, its standard deviation was large. This is primarily due to the small sample size of passengers who chose eHMI-T, specifically, one male and one female.

Compared to those who favored eHMI-AV, individuals who preferred eHMI-NV exhibited lower levels of extraversion, openness to experience, and agreeableness, while demonstrating a slightly higher degree of neuroticism. This suggests that different eHMIs may appeal to different personality types.

5. Discussion

5.1. UE of APMV passengers on the silent GUI-based eHMI

The subjective evaluation results of Q1 and Q2 in Figure 6 and Table 6 show that, compared with the MUI-based eHMIs (i.e., eHMI-NV and eHMI-AV), the passengers got insufficient information from eHMI-T, which hindered their situation awareness and made them more difficult to understand the communication between APMV and pedestrians. In post-experiment interviews, participants made the same comments as “without the sound (i.e., eHMI-T), I don’t know what is being displayed. The pedestrians are talking by themselves and I don’t know what kind of communication they are having.”

Moreover, this lack of understand led passengers to feelings of awkwardness, as indicated in Q4. Also, from the interview, almost all passengers reported that the silence time when eHMI-T conveyed information to pedestrians was awkward. For example, “there was a long period of silence in the encounter, which caused a strong sense of awkwardness.” Moreover, some passengers also reported feeling “left out when eHMI-T and pedestrians communicated.”

Compared to other MUI-based eHMIs, the above issues resulted in significantly lower ratings of the eHMI-T’s pragmatic quality and hedonic quality of UE by both male and female passengers (see Figure 7). Particularly when compared to the benchmark in Hinderks et al. (2018), the hedonic quality of the eHMI-T was already categorized in the bad range. This suggests that utilizing the voice cues may offer a more engaging and enjoyable experience for passengers compared to relying solely on a silent GUI-based eHMI.

According to the above research results, the answer to RQ 1 is that from the perspective of passengers, the silent GUI-based eHMI may not be suitable for APMV because of insufficient information, difficulties in understanding and the feeling of awkwardness during the APMV–pedestrian communications.

5.2. UE of APMV passengers on the MUI-based eHMIs

The findings from Q1 and Q2, illustrated in Figure 6 and Table 6, reveal that communication with pedestrians through two MUI-based eHMIs, specifically eHMI-NV and eHMI-AV, enabled the sharing of information with passengers via voice cues, thereby facilitating passengers’ understanding of the communication.

However, as shown in Figure 6 and Table 6, the results of Q3 and Q4 indicated that passengers felt the eHMI-AV communicated excessively compared to the eHMI-NV, leading to significant feeling of awkwardness for passengers experiencing the communication between eHMI-AV and pedestrians. From the interviews, we found that there are differences in the experience of male and female passengers with the eHMI-AV. Specifically, a few male passengers felt that the eHMI-AV gave too much information compared to the eHMI-NV, e. g., “I preferred eHMI-NV over eHMI-AV because it provided the minimum necessary communication.” However, half of the female passengers said that “the eHMI-AV was too casual, especially when the encountering pedestrian who was a stranger.”

The results from Q5, as shown in Table 6, indicated that passengers felt significantly more self-conscious about attracting attention from those around them when using the eHMI-AV compared to the eHMI-NV. Interestingly, Figure 6 revealed a gender disparity in the responses to Q5 for eHMI-NV. Specifically, male passengers were not concerned about eHMI-NV drawing too much attention from people around them, whereas female passengers expressed more concern about this issue. This suggests that gender may play a role in shaping passengers’ perceptions and experiences when using these interfaces. This finding can be explained by the conclusion in Turk et al. (1998), which reported that in some public places, females were with significantly greater fear than males being the center of attention. Similarly, Zentner et al. (2023) also found that females have higher levels of social anxiety than males.

The above conclusions can also be reflected in the UEQ results described in Section 4.2. The results presented in Figure 7 and Table 10 suggest that eHMI-NV was found to have significantly better pragmatic quality than eHMI-AV. This is consistent with the results of Q3–Q5 in Figure 6 and Table 6, i.e., the eHMI-NV accomplishes negotiating with pedestrians in the most concise communication possible, without the perceived excessive communication that causes passengers to feel awkward, as is the case with the eHMI-AV. In the interview we got similar results, such as “I felt that they communicated better with voice than with text (eHMI-T), but I preferred eHMI-NV because it communicated the minimum required compared to eHMI-AV.”

However, from the perspective of hedonic quality, eHMI-AV is considered significantly better than eHMI-NV, as shown in Figure 7 and Table 10. In the interview, some passengers told that “I like the eHMI-AV better than the eHMI-NV because I feel that the eHMI-AV is more communicative,” as well as “I think the robot character (i.e., APMV’s eHMI) should have its own personality.” Above results are in line with (Lee et al., 2019), where the more anthropomorphic voice was favored and rated higher in trust, pleasure, and the sense of control than a voice-command interface. Besides, the speech embodiment of more conversational speech was evaluated as more warm and social presence (Wang et al., 2021).

Therefore, the answer to RQ 2 is that MUI-based eHMIs with voice cues, i.e., the eHMI-NV and eHMI-AV, both significantly enhance the passenger’s UE in APMV and pedestrian interactions. Moreover, it should be noted that each of these two MUI-based eHMIs has its own advantages, i.e., eHMI-NV has an advantage in pragmatic quality, while eHMI-AV has an advantage in hedonic quality. This difference in pragmatic and hedonic qualities between eHMI-NV and eHMI-AV aligns with the findings from a study on human-robot interaction by Ullrich (2017). They reported that the neutral robot personality was rated best in a goal-oriented situation, while the positive robot personality was preferred in experience-oriented scenarios. Above results highlight the importance of considering both pragmatic and hedonic quality in the design of eHMIs, and the potential benefits of using voice cues to enhance passengers’ UEs.

5.3 eHMI designs for personality

To answer the RQ 3, the relationship between participants’ personalities and their favorite eHMI was discussed. As results disrupted in Section 4.5, this study found that individuals with different personality traits tend to prefer different types of eHMI. Specifically, those with lower scores in extraversion, openness to experience, and conscientiousness were more inclined to favor eHMI-T, while those with higher scores in these domains tended to prefer eHMI-AV. This result suggests a potential correlation between personality traits and the choice of eHMI, which could have implications for designing interfaces that cater to the preferences and needs of different personality types. This result aligns with the conclusion drawn in (Sarsam & Al-Samarraie, 2018; Tapus & Matarić, 2008). In interviews, 20 out of 24 passengers thought it was important that the eHMI was designed to fit their personality. A number of representative sentiments were reported, such as “If the APMV can reflect the passenger’s personality, that’s fine. But if it deviates from the passenger’s personality, I will feel strange”, “I think it’s better to match my personality because I feel like it is a part of me” as well as “I think this is very important because if our personalities don’t match well, I might feel it’s overly polite, or like my feelings aren’t being properly responded to.” Moreover, two passengers thought that matching personalities is important, but they have concerns, such as “As a user, I think it’s better if it matches but I don’t like attracting too much attention.” The remaining two passengers made it clear that they did not think the eHMI needed to be designed to fit their personalities, for example “I don’t think so much. I think a machine is a machine. I think APMV is a separate entity from me.” and “I prioritize efficiency, so I prefer not to have unnecessary communication.”

In summary, to answer the RQ 3, it may be beneficial to take into account the personality traits of different user groups to better meet their needs and preferences when designing eHMIs. For instance, simpler and more straightforward interfaces (such as eHMI-T) might be suitable for users who are more introverted and less adventurous, while more complex and diverse interfaces (such as eHMI-AV) could be provided for users who are more extroverted, open, and conscientious.

5.4. Limitations

The preference evaluation on eHMIs under different user scenarios may be different. In this study, we used a simple but common scenario found on campus for testing, which may not fully represent real-life situations, like crowded places with lots of pedestrians. Furthermore, for some quiet scenes, e. g., library and museum, the passengers’ UE of eHMI with voice information cues may be different from the results of this study.

Since one of the objectives of this study was to verify whether the silent eHMI-T could provide a good UE for a wide range of passengers, we did not balance personality differences between participants before the experiment. It led to a small number of people choosing eHMI-T and made the standard deviation of neuroticism for passengers who chose eHMI-T too large. In this regard, we will target introverted passengers in future works to verify if they prefer to use the silent eHMI-T.

Moreover, the participants in this experiment were young Japanese people in their twenties who tend to be more easily receptive to new things than the elderly. The results may not generalize to other age groups, e. g., elderly people, non-student groups or cultural backgrounds.

It should be noted that in this study, the observation of the gender on UE is exploratory, which gives the hints that it is worth further investigation by taking gender differences into consideration for the APMV’s eHMI design. However, in this study, the sample sizes of each gender are limited. To further verify the impact of gender differences on UE, larger sample sizes are needed in future work.

5.5. Future works

In our future work, we will explore specific design guidelines tailored to APMV passengers with different personalities and genders. Additionally, we plan to pre-screen participants using the questionnaire to identify subjects with significantly different trait levels. This approach will enable us to invite individuals with diverse personalities to help validate the proposed eHMI design guidelines. We are also planning to develop an eHMI of APMV which can automatically adjust the content of the information and the tone of voice based on the user’s personality traits and genders. We will also explore the effects of passengers performing NDRTs, such as reading a book, looking at a cell phone, etc., on UEs of the eHMI while riding the APMV.

6. Conclusion

In this article, we discussed three eHMI designs, i.e., GUI-based eHMI with a text message (eHMI-T), MUI-based eHMI-NV, and MUI-based eHMI-AV used for the APMV–pedestrian communication, from the perspective of APMV passengers.

The field study suggested that the silent GUI-based eHMI-T might not be the most suitable option for APMVs. This is because when the APMV communicated with a pedestrian, its passenger might encounter difficulties in comprehending the communication, leading to a sense of awkwardness due to the lack of information. In particular, participants reported the APMV–pedestrian communication was less comprehensive and sufficient, and they felt awkward during the “silent time,” due to the lack of information cues received as passengers.

We also found that the use of voice cues can improve passengers’ understanding of the communication between the APMV and pedestrian. This enhancement contributes to an overall improved UE of eHMIs. Specifically, eHMI-NV has an advantage in pragmatic quality, while eHMI-AV has an advantage in hedonic quality.

This study also highlights that the passengers’ personalities should be considered when designing eHMI for APMVs to enhance their UE. For instance, simpler and more straightforward interfaces, (such as eHMI-T), might be better suited for passengers with lower levels of extraversion and openness to experience. Conversely, interfaces with more personality and emotion, (such as eHMI-AV), could be offered to users with higher levels of extraversion, openness to experience, and agreeableness. In addition, the eHMI-NV is also advantageous for the masses, as the eHMI-NV has also been recognized by the majority of passengers as it balances adequate functionality with acceptable hedonic quality.

We have also observed that gender differences can influence specific aspects of UE for eHMIs, such as drawing excessive attention and hedonic quality. This demonstrates the necessity of considering passenger gender when designing APMV’s eHMI.

CRediT author statement

Hailong Liu: conceptualization, investigation, methodology, software, validation, formal analysis, project administration, funding acquisition, writing—original draft. Yang Li: investigation, methodology, writing—review and editing. Zhe Zeng: methodology, writing—review and editing. Hao Cheng: methodology, writing—review and editing. Chen Peng: methodology, writing—review and editing. Takahiro Wada: methodology, writing—review and editing.

Acknowledgments

The authors would like to thank all participants who took part in the experiment. The authors appreciate MiM Mrs. Yaxi Wu, who is the wife of Hailong Liu, for creating the illustration of Figure 1, free of charge.

Disclosure statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Additional information

Funding

This work was supported by Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Numbers 20K19846 and 22H00246, Japan.

Notes on contributors

Hailong Liu

Hailong Liu is an assistant professor at Graduate School of Science and Technology, NAIST, Japan. He acquired BS (2013), MS (2015) and PhD (2018) in Engineering from Ritsumeikan University, Japan. He was a Research Fellowship for Young Scientists (DC2) of JSPS (2016–2018), and a researcher at Nagoya University (2018–2021).

Yang Li

Yang Li is a PhD candidate at KIT, Germany, specializes in the human–machine interface for communication between AVs and human traffic partners on ambiguous roads. With a 12-month visiting study at the University of Leeds and participation in the Hi-Drive Program, she also visited NAIST in 2023.

Zhe Zeng

Zhe Zeng is a postdoctoral researcher at the University of Ulm, Germany. She received her MSc degree in 2017 and PhD degree in 2022 from the Technical University of Berlin, Germany. Her research interests include eye tracking, natural gaze interaction, and human–robot interaction.

Hao Cheng

Hao Cheng is a postdoctoral researcher at the University of Twente, The Netherlands. He acquired MSc in 2017 and PhD in 2021 degrees in informatics at Leibniz University Hannover, Germany. His research focuses on exploring deep learning in modeling interactions between vehicles and VRUs to facilitate safer autonomous driving.

Chen Peng

Chen Peng is a research fellow and a PhD candidate at the Institute for Transport Studies, University of Leeds. She received her Master’s degree in Human Technology Interaction from the Eindhoven University of Technology in 2019. Her research interests include user comfort in automated driving, human–computer interaction, and aging.

Takahiro Wada

Takahiro Wada is a Full Professor at NAIST, Japan, since 2021. He obtained his PhD in Robotics from Ritsumeikan University, Japan, in 1999, and served as an Assistant and Associate Professor at Kagawa University for 12 years. He became a Full Professor at Ritsumeikan University in 2012.

Notes

1 WHILL Autonomous developed by WHILL Inc.: https://youtu.be/vJWhwNnUPRs

2 RakuRo developed by ZMP Inc.: https://youtu.be/lWhbJ0rBwjM

3 The voice is generated by Microsoft Nanami Online. The voice pitch was set to −10% for the neutral voice and +15% for the affective voice. The voices of eHMI-NV and eHMI-AV can be found at https://1drv.ms/f/s!AqdIEHyOvvX56E_YDn2VRzSeU1KK

4 The Japanese UEQ-S and the data analysis tool download from https://www.ueq-online.org