Intention-Sensing Recipe Guidance via User Accessing to Objects

ABSTRACT

Sensing the intention of a user’s forthcoming action is a necessary function for systems that assist human physical activity. In this article, a strategy for recipe guidance systems that can predict the forthcoming intended subtask in a cooking task is investigated. The focus is on user accessing objects, that is, touching and releasing objects. Touching can indicate the start of the forthcoming subtask and releasing can indicate the end of the task. The main difficulty lies in the fact that humans may move objects because they are in the way and use cooking tools that are unanticipated by an assistive system. In such cases, the accessed object should not indicate the forthcoming subtask. A method is proposed to track the progress of a task based on the object access history. This enables to eliminate object accesses that are out of context. Simultaneously, the method predicts the forthcoming subtask based on a combination of progress and materials rather than tools and materials. Then, a guidance system that runs as a web service is developed. In experiments, real cooking activities navigated by this system are observed. The Wizard of OZ method is utilized to simulate a system that detects object accesses. The experimental results show that 73.6% accuracy is achieved in the selection of the displayed information. This result supports the use of “access to objects” realize effective intention-sensing systems.

Funding

This work was supported by JSPS KAKENHI, grant numbers 24240030, 26280084 and 16K16099.

Additional information

Funding

This work was supported by JSPS KAKENHI, grant numbers 24240030, 26280084 and 16K16099.

Notes on contributors

Atsushi Hashimoto

Atsushi Hashimoto is an assistant professor at the Graduate School of Education, Kyoto University, Japan. He received his BS degree in engineering, and MS degree and PhD in informatics from Kyoto University, in 2005, 2008, and 2013, respectively. His research interests include pattern recognition, computer vision, and HCI.

Jin Inoue

Jin Inoue received his BS degree in engineering, and MS degree in informatics from Kyoto University, Japan, in 2011 and 2013, respectively.

Takuya Funatomi

Takuya Funatomi is an associate professor at Information Science Department, Nara Institute of Science and Technology, Japan. He received his MS degree and PhD in informatics from Kyoto University, in 2004 and 2007, respectively. He is a member of ACM and the IEEE Computer Society and Communication Society.

Michihiko Minoh

Michihiko Minoh is a professor at the Academic Center for Computing and Media Studies, Kyoto University, Japan. He received his BE, ME, and DE degrees in information science from Kyoto University in 1978, 1980, and 1983, respectively. He is a member of the IEEE Computer Society and Communication Society and ACM.