Evaluation of a Hybrid of Hand Gesture and Controller Inputs in Virtual Reality

ABSTRACT

Controller and hand gesture recognition are both common input mechanisms in immersive environments. We evaluate the effect of combining them as a hybrid input to perform game-like tasks. To explore the possible design space, three tests were conducted sequentially. First, a design study was conducted to investigate which of the tasks were most likely to be performed with hybrid inputs. Next, we implemented the four highest-ranked tasks and invited 22 participants to perform these tasks with different input combinations. From examining execution times and reviewing feedback from participants, we provide observations and discussions concerning the usage of a hybrid input. A follow-up experiment was designed to refine these observations. The results indicate that task execution using hybrid inputs is as efficient as bimanual controllers when assigning actions to the appropriate input. Finally, a few guidelines are provided suggesting how to consider possible scenarios for this hybrid input.

Acknowledgments

The authors would like to thank the anonymous reviewers for their constructive comments to improve the manuscript

Notes

1. Oculus Rift: https://www.oculus.com/rift

2. VIVE Focus: https://enterprise.vive.com/us/product/vive-focus

3. VIVE Cosmos: https://www.vive.com/us/product/vive-cosmos/features/

Additional information

Funding

This work was supported by the Ministry of Science and Technology in Taiwan under Grant MOST-107-2218-E-155-008-MY3 and MOST-108-2221-E-155-033-MY2.

Notes on contributors

Yi-Jheng Huang

Yi-Jheng Huang received her MS and PhD degrees in the Department of Computer Science from National Chiao Tung University, Taiwan. Now, she is an assistant professor in the Department of Information Communication, Yuan Ze University. Her research interests include human-computer interaction and computer graphics.

Kang-Yi Liu

Kang-Yi Liu received the BS degree from Department of Information Communication, Yuan Ze University, Taiwan, in 2019. Now, he is a master student in the Department of Interaction Design, National Taipei University of Technology. His research interests include human-computer interaction, natural user interface in virtual reality and multimodal interaction.

Suiang-Shyan Lee

Suiang-Shyan Lee received his PhD degree in the Department of Computer Science from National Chiao Tung University, Taiwan. He was a senior front-end engineer at QNAP Systems and is currently a technical manager at e-SOFT Corporation. His research interests include human-computer interaction, multimedia security, web application, and computer graphics.

I-Cheng Yeh

I-Cheng Yeh received the PhD degree in Computer Science and Information Engineering from National Cheng Kung University. Now, he is an assistant professor in the Department of Computer Science and Engineering, Yuan Ze University. His research interests include computer graphics, surfaces parameterization and manipulation, data visualization, object tracking and classification.