Tripartite Effects: Exploring Users’ Mental Model of Mobile Gestures under the Influence of Operation, Handheld Posture, and Interaction Space

ABSTRACT

In this research, we conducted an elicitation study with 54 participants to evaluate user-defined mobile gestures given three types of operations, handheld postures, and interaction spaces, respectively, to investigate the effects of these factors on mental model. The results of the elicitation study revealed that each of the three factors significantly affected the users’ mental models, which could be reflected by the characteristics (e.g., nature, number of steps, and spatial utility) of the user-defined mobile gestures. In addition, we found that users tended to assign different roles to mid-air and on-screen gestures based on their motor ability given a handheld gesture and their interpretation of the characteristics and requirements of an operation. We further derived a set of gestures commonly defined in the elicitation study for a list of representative operations, and we share our insights into users’ mental models and the implications on future mobile-gesture design in different contexts.

Additional information

Notes on contributors

Kening Zhu

Kening Zhu is an assistant professor at the School of Creative Media. He received his PhD from the National University of Singapore and his bachelor degree on Computer Science from Huazhong University of Science and Technology, China. His research interests include mobile interaction design, tangible user interfaces, and rapid prototyping.

Xiaojuan Ma

Xiaojuan Ma is an assistant professor of Human-Computer Interaction at the Department of Computer Science and Engineering, Hong Kong University of Science and Technology. Her research interests include multimodal interaction, data visualization, gamification, crowdsourcing, and computational linguistic.

Haoyuan Chen

Haoyuan Chen is a student research assistant in the School of Creative Media, City University of Hong Kong, under the supervision of Dr. Kening Zhu. He is now finishing his bachelors degree on Computer Science in CityU. He has participated in various HCI projects, including mobile interface and child-computer interaction.

Miaoyin Liang

Miaoyin Liang is a research assistant in the JC School of Public Health and Primary Care, CUHK. She received her masters degree on public health from University of Hong Kong and her bachelor of science from Sun Yat Sen University, China. Her expertise lies in quantitative study on human behavior.