Providing Dual Awareness using Multimodal Cues for Collaborative Manipulation in Virtual Environments

Abstract

Virtual reality (VR) collaborative manipulation requires a proper understanding of the user’s self, other collaborators, and shared objects in the virtual environment. Users must be aware of the movements of themselves and others to coordinate their actions when they hand over an object or move it together. The awareness of collaboration plays an essential role in virtual environments. However, previous studies have focused on providing awareness only for cooperative manipulation. This study proposes a novel framework for dual awareness that comprehensively manages the state of interactions between multiple users and shared virtual objects and provides distinct sensory cues to distinguish themselves from collaborators. The experimental results showed that participants could easily and quickly collaborate on manipulation using our feedback, and they also answered positively in the subjective evaluation.

Keywords:

• Collaborative manipulation
• awareness
• multimodal interaction

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was partly supported by the Korea Institute of Science and Technology (KIST) Institutional Program (Project No. 2E32302) and by the Future Promising Convergence Technology Pioneer Program funded by National Research Foundation of Korea grant funded by the Korean Government (MSIP) (2022M3C1A309874611).

Notes on contributors

Hyunju Kim

Hyunju Kim is a Ph.D. student at Cornell University focusing on Extended Reality, HCI, and CSCW. She earned her undergraduate degree in Electrical Engineering from Ewha Womans University and a master’s degree in Culture Technology from KAIST. Prior to Cornell, she worked as a research assistant at KIST.

Jung-Min Park

Jung-Min Park is a Principal Research Scientist at Korea Institute of Science and Technology (KIST). She received a Ph.D. degree in computer science and engineering from Ewha Womans University in 2008. Her research interests include virtual reality/extended reality, human-computer interaction, and user interface.