Can You Easily Perceive the Local Environment? A User Interface with One Stitched Live Video for Mobile Robotic Telepresence Systems

ABSTRACT

Many existing mobile robotic telepresence systems have equipped with two cameras, one is a forward-facing camera for video communication, and the other is a downward-facing camera for robot navigation. However, the two live videos from these two cameras would cause some confusion which makes it difficult for a remote operator to perceive the local environment. In this paper, we propose to use a user interface with one stitched live video instead of two live videos for mobile robotic telepresence systems. We used a video stitching algorithm to stitch the two live videos into one live video through which a remote operator can well perceive the local environment. We conducted a user study to investigate the difference between one stitched live video and two separate live videos in the user interface. The results show that the user interface with one stitched live video improves task efficiency, the number of errors, and remote operators’ feelings of presence, and enables remote operators to concentrate on the work they are doing.

Additional information

Funding

This work was supported in part by the Natural Science Foundation of China (NSFC) under Grants No. 61773062 and No. 61702037, Beijing Municipal Natural Science Foundation under Grant No. L172027.

Notes on contributors

Yanmei Dong

Yanmei Dong received the B.S. and M.S. degrees from Beijing Institute of Technology (BIT), Beijing, China, in 2013 and 2015, respectively. She is currently pursuing a Ph.D. degree in the School of Computer Science, under the supervision of Prof. Mingtao Pei and Yunde Jia.

Yunde Jia

Yunde Jia (M’11) received the B.S., M.S., and Ph.D. degrees from the Beijing Institute of Technology (BIT) in 1983, 1986, and 2000, respectively. He was a visiting scientist with the Robotics Institute, Carnegie Mellon University (CMU), from 1995 to 1997. He is currently a Professor with the School of Computer Science, BIT, and the team head of BIT innovation on vision and media computing. He serves as the director of Beijing Lab of Intelligent Information Technology. His interests include computer vision, vision-based HCI and HRI, and intelligent robotics.

Weichao Shen

Weichao Shen received the B.S. degree in control engineering from the School of Automation, Beijing Institute of Technology (BIT), Beijing, China, in 2014. He is currently pursuing a Ph.D. degree in the School of Computer Science, under the supervision of Prof. Yunde Jia. His current research interests include computer vision, unsupervised representation learning, object tracking and 3D reconstruction.

Yuwei Wu

Yuwei Wu received the Ph.D. degree in computer science from Beijing Institute of Technology (BIT), Beijing, China, in 2014. He is now an Assistant Professor at School of Computer Science, BIT. From August 2014 to August 2016, he was a post-doctoral research fellow at Rapid-Rich Object Search (ROSE) Lab, School of Electrical & Electronic Engineering (EEE), Nanyang Technological University (NTU), Singapore. He received outstanding Ph.D. Thesis award from BIT, and Distinguished Dissertation Award Nominee from China Association for Artificial Intelligence (CAAI).