The Gesture Disagreement Problem in Free-hand Gesture Interaction

ABSTRACT

Accurately understanding a user’s intention is often essential to the success of any interactive system. An information retrieval system, for example, should address the vocabulary problem (Furnas et al., 1987) to accommodate different query terms users may choose. A system that supports natural user interaction (e.g., full-body game and immersive virtual reality) must recognize gestures that are chosen by users for an action. This article reports an experimental study on the gesture choice for tasks in three application domains. We found that the chance for users to produce the same gesture for a given task is below 0.355 on average, and offering a set of gesture candidates can improve the agreement score. We discuss the characteristics of those tasks that exhibit the gesture disagreement problem and those tasks that do not. Based on our findings, we propose some design guidelines for free-hand gesture-based interfaces.

Acknowledgments

The authors would like to thank the anonymous reviewers for their insightful comments. This work was supported by the National Natural Science Foundation of China under Grant No. 61772564, 61202344 and the funding offered by the China Scholarship Council (CSC).

Additional information

Funding

This work was supported by the National Natural Science Foundation of China: [Grant Number 61772564]; National Natural Science Foundation of China: [Grant Number 61202344].

Notes on contributors

Huiyue Wu

Huiyue Wu is an Associate Professor at Sun Yat-sen University, Guangzhou, China. His research interests include vision-based interfaces, gestural interaction, and user-centered design. He has published more than 30 papers in international journals and conference proceedings.

Shaoke Zhang

Shaoke Zhang is an Assistant Professor at Sun Yat-sen University, Guangzhou, China. His research interests include human–computer interaction, interaction design, and usability engineering. He has published more than 20 papers in international journals and conference proceedings.

Jiayi Liu

Jiayi Liu is a graduate student at Sun Yat-sen University, Guangzhou, China. Her research interests include human–computer interaction, interaction design, and usability engineering. She obtained Bachelor of Design Science from Sun Yat-sen University, Guangzhou, China, 2016.

Jiali Qiu

Jiali Qiu is a graduate student at Sun Yat-sen University, Guangzhou, China. Her research interests include human–computer interaction, interaction design, and usability engineering. She obtained Bachelor of Design Science from Sun Yat-sen University, Guangzhou, China, 2016.

Xiaolong (Luke) Zhang

Xiaolong (Luke) Zhang is an associate professor at the College of Information Sciences and Technology at the Pennsylvania State University, University Park, PA. His research interests include human–computer interaction, information visualization, and virtual reality. He has published more than 70 papers in international journals and conference proceedings.