Character expression of a conversational robot for adapting to user personality

ABSTRACT

User adaptation is important in conversational robots to enhance the user experience and engagement. One way of user adaptation is to change the dialogue behavior. A character (e.g. extrovert or agreeable) can be defined to achieve human-like behaviors and user adaptation, and the appropriate character differs depending on each user. In this study, we investigate user adaptation by character expression of conversational robots. Our character expression model adopts the Big Five traits for controlling four dialogue behaviors: amount of utterance, backchannel frequency, filler frequency, and switching pause length. We cluster the system character and user personality into four classes based on an analysis of a speed dating dialogue corpus, and we found the best combinations between the system character and user personality. We implemented the character expression model into a laboratory guide and a chit-chat robot and conducted subjective experiments, where the subjects talked with robots with four different characters and evaluated their impressions of each character. The results shows that the Role model character was preferred for the task-oriented dialogue of the laboratory guide, but a robot character complementary to the subject personality was preferred for the non-task-oriented dialogue of the speed dating and chit-chat.

GRAPHICAL ABSTRACT

KEYWORDS:

• Conversational robot
• user adaptation
• character
• personality

Acknowledgements

These authors are grateful to Prof. Takatsune Kumada and Dr Ritsuko Iwai for helpful discussions.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 hottoSNS-w2v: https://github.com/hottolink/hottoSNS-w2v

2 Japanese dialogue transformer: https://github.com/nttcslab/japanese-dialog-transformers.

3 Open 2channel dialogue corpus: https://github.com/1never/open2ch-dialogue-corpus

Additional information

Funding

This work was supported by Moonshot R&D [grant number JPMJPS2011] and Grant-in-Aid for JSPS Fellow [grant number JP20J22284].

Notes on contributors

Kenta Yamamoto

Kenta Yamamoto received his Ph.D. in 2023 from the Graduate School of Informatics in Kyoto University, Kyoto, Japan. He was a JSPS Research Fellow (DC1) from 2020 to 2023. Currently, he is an Assistant Professor of SANKEN (The Institute of Scientific and Industrial Research), Osaka University. His research interests include spoken dialogue systems (SDSs) and character expression for SDSs.

Koji Inoue

Koji Inoue received Ph.D. in 2018 from the Graduate School of Informatics in Kyoto University, Kyoto, Japan. He was a JSPS Research Fellow (DC1) from 2015 to 2018. Currently, he is an Assistant Professor of the Graduate School of Informatics, Kyoto University. His research interests include spoken dialogue systems, speech signal processing, and human–robot interaction.

Tatsuya Kawahara

Tatsuya Kawahara received B.E. in 1987, M.E. in 1989, and Ph.D. in 1995, all in information science, from Kyoto University, Kyoto, Japan. From 1995 to 1996, he was a Visiting Researcher at Bell Laboratories, Murray Hill, NJ, USA. Currently, he is a Professor of School of Informatics, Kyoto University. From 2020 to 2023, he was the Dean of the School. Before that, he was also an Invited Researcher at ATR and NICT. He has published more than 450 academic papers on automatic speech recognition, spoken language processing, and spoken dialogue systems. He has been conducting several projects including open-source speech recognition software Julius, the automatic transcription system deployed in the Japanese Parliament (Diet), and the autonomous android ERICA. Dr. Kawahara received the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology (MEXT) in 2012. From 2003 to 2006, he was a member of IEEE SPS Speech Technical Committee. He was a General Chair of IEEE ASRU 2007 and is a General Chair of SIGdial 2024. He also served as a Tutorial Chair of INTERSPEECH 2010, a Local Arrangement Chair of ICASSP 2012, and a General Chair of APSIPA ASC 2020. He was an editorial board member of Elsevier Journal of Computer Speech and Language and IEEE/ACM Transactions on Audio, Speech, and Language Processing. From 2018 to 2021, he was the Editor-in-Chief of APSIPA Transactions on Signal and Information Processing. Dr Kawahara is the President of APSIPA, the Secretary General of ISCA, and a Fellow of IEEE.