Multisensory integration effect of humanoid robot appearance and voice on users’ affective preference and visual attention

ABSTRACT

Appearance and voice are essential factors impacting users’ affective preferences for humanoid robots. However, little is known about how the appearance and voice of humanoid robots jointly influence users’ affective preferences and visual attention. We conducted a mixed-design eye-tracking experiment to examine the multisensory integration effect of humanoid robot appearances and voices on users’ affective preferences and visual attention. The results showed that the combinations of affectively preferred voices and appearances attracted more affective preferences and shorter average fixation durations. The combinations of non-preferred voices and preferred appearances captured less affective preferences and longer fixation durations. The results suggest that congruent combinations of affectively preferred voices and appearances might motivate a facilitation effect on users’ affective preference and the depth of visual attention through audiovisual complements. Incongruent combinations of non-preferred voices and preferred appearances might stimulate an attenuation effect and result in less affective preferences and a deeper retrieval of visual information. Besides, the head attracted the most amount of visual attention regardless of voice conditions. This paper contributes to deepening the understanding of the multisensory integration effect on users’ affective preferences and visual attention and providing practical implications for designing humanoid robots satisfying users’ affective preferences.

KEYWORDS:

• Multisensory integration
• appearance
• voice
• humanoid robot
• affective preference
• visual attention

Acknowledgments

We thank Vincent G. Duffy from the Purdue University, who helps us proofread the language of this paper. Also, we are grateful to all the experimental participants for this study. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by National Natural Science Foundation of China: [Grants Number 72071035, 71771045].