Walking partner robot chatting about scenery

ABSTRACT

We believe that many future scenarios will exist where a partner robot will talk with people on walks. To improve the user experience, we aim to endow robots with the capability to select an appropriate conversation topics by allowing them to start chatting about a topic that matches the current scenery and to extend it based on the user's interest and involvement in it. We implemented a function to compute the similarities between utterances and scenery by comparing their topic vectors. First, we convert the scenery into a list of words by leveraging Google Cloud Vision library [Google Cloud Vision Api [Online]. Available from: https://cloud.google.com/vision/] . We form a topic vector space with the Latent Dirichlet Allocation method and transform a list of words into a topic vector. Our system uses this function to choose (from an utterance database) the utterance that best matches the current scenery. Then it estimates the user's level of involvement in the chat with a simple rule based on the length of their responses. If the user is actively involved in the chat topic, the robot continues the current topic using pre-defined derivative utterances. If the user's involvement sags, the robot selects a new topic based on the current scenery. The topic selection that is based on the current scenery was proposed in our previous work [Totsuka R, Satake S, Kanda T, et al. Is a robot a better walking partner if it associates utterances with visual scenes? ACM/IEEE Int. Conf. on Human–Robot Interaction (HRI2017), Aula der Wissenschaft, Vienna, Austria; 2017. p. 313–322]. Our main contribution is the whole chat system, which includes the user's involvement estimation. We implemented our system with a shoulder-mounted robot and conducted a user study to evaluate its effectiveness. Our experimental results show that users evaluated the robot with the proposed system as a better walking partner than one that randomly chose utterances.

GRAPHICAL ABSTRACT

KEYWORDS:

• Walking partner robot
• chatting while walking
• association of utterance and visual scene
• topic selection

Disclosure statement

The authors declare that they have no conflict of interest.

Additional information

Funding

This work was in part supported by Tateishi Science and Technology Foundation [grant number 2158001], in part supported by JST, CREST [grant number JPMJCR17A2], and in part supported by JSPS KAKENHI [grant number JP18H04121].

Notes on contributors

Taichi Sono

Taichi Sono is a Ph.D. candidate in School of Science for Open and Environmental Systems, Keio University, Yokohama, Japan. He received a B.E.degree in 2014 and Master degree in 2016 from Keio University. His research interests include autonomous robots, human–robot interaction, speech system, body extension, sense of agency and sense of ownership.

Satoru Satake

Satoru Satake received his B. Eng, M. Eng, and Ph. D. degrees from Keio University, Yokohama, Japan, in 2003, 2005, and 2009. He is currently a researcher at ATR Intelligent Robotics and Communication Laboratories, Kyoto, Japan. His current research interests include human–robot interaction, social robots, and networked robot systems.

Takayuki Kanda

Takayuki Kanda is a professor in Informatics at Kyoto University, Japan. He is also a Visiting Group Leader at ATR Intelligent Robotics and Communication Laboratories, Kyoto, Japan. He received his B. Eng, M. Eng, and Ph. D. degrees in computer science from Kyoto University, Kyoto, Japan, in 1998, 2000, and 2003, respectively. He is one of the starting members of Communication Robots project at ATR. He has developed a communication robot, Robovie, and applied it in daily situations, such as peer-tutor at elementary school and a museum exhibit guide. His research interests include human–robot interaction, interactive humanoid robots, and field trials.

Michita Imai

Michita Imai is a Professor of Faculty of science and technology at Keio university and a Researcher at ATR Intelligent Robot Laboratories. He received his Ph.D. degree in Computer Science from Keio Univ. in 2002. In 1994, he joined NTT Human Interface Laboratories. He joined the ATR Media Integration & Communications Research Laboratories in 1997. He was a visiting scholar of University of Chicago from 2009-2010. His research interests include autonomous robots, human–robot interaction, speech dialogue systems, humanoid, and spontaneous behaviors. He is a member of Information and Communication Engineers Japan (IEICE-J), the Information Processing Society of Japan, the Japanese Cognitive Science Society, the Japanese Society for Artificial Intelligence, Human Interface Society, IEEE, and ACM.