Behavioral learning of dish rinsing and scrubbing based on interruptive direct teaching considering assistance rate

Abstract

Robots are expected to manipulate objects in a safe and dexterous way. For example, washing dishes is a dexterous operation that involves scrubbing the dishes with a sponge and rinsing them with water. It is necessary to learn it safely without splashing water and without dropping the dishes. In this study, we propose a safe and dexterous manipulation system. The robot learns a dynamics model of the object by estimating the state of the object and the robot itself, the control input, and the amount of human assistance required (assistance rate) after the human corrects the initial trajectory of the robot's hands by interruptive direct teaching. By backpropagating the error between the estimated and the reference value using the acquired dynamics model, the robot can generate a control input that approaches the reference value, for example, so that human assistance is not required and the dish does not move excessively. This allows for adaptive rinsing and scrubbing of dishes with unknown shapes and properties. As a result, it is possible to generate safe actions that require less human assistance.

GRAPHICAL ABSTRACT

Keywords:

• Recovery manipulation
• interruptive direct teaching
• model predictive learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Shumpei Wakabayashi

Shumpei Wakabayashi received his B.E. in Mechanical and Systems Engineering from Kyoto University in 2020 and M.S. in Mechano-Informatics from the University of Tokyo in 2022. His research interests include cognitive learning and robot system integration.

Kento Kawaharazuka

Kento Kawaharazuka is a project assistant professor at JSK Robotics Laboratory in the Department of Mechano-Informatics at the University of Tokyo. He received his B.E., M.S., and Ph.D. degrees in Mechano-Informatics from the University of Tokyo in 2017, 2019, and 2022, respectively. His research interests include musculoskeletal humanoids, tendon-driven robots, machine learning, and foundation models.

Kei Okada

Kei Okada received his B.E. in Computer Science from Kyoto University in 1997. He received his M.S. and Ph.D. in Information Engineering from the University of Tokyo in 1999 and 2002, respectively. From 2002 to 2006, he participated in the Professional Programme for Strategic Software Project at the University of Tokyo. He was appointed as a lecturer in Creative Informatics at the University of Tokyo in 2006, and became an associate professor and then a professor in the Department of Mechano-Informatics in 2009 and 2018, respectively. His research interests include humanoid robots, real-time 3D computer vision, and recognition-action integrated systems.

Masayuki Inaba

Masayuki Inaba graduated from the Department of Mechanical Engineering at the University of Tokyo in 1981, and received his M.S. and Ph.D. degrees from the Graduate School of Information Engineering at the University of Tokyo in 1983 and 1986, respectively. He was appointed as a lecturer in the Department of Mechanical Engineering at the University of Tokyo in 1986, an associate professor in 1989, and a professor in the Department of Mechano-Informatics in 2000. His research interests include key technologies of robotic systems and software architectures to advance robotics research.