Working Memory Capacity for Gesture-Command Associations in Gestural Interaction

Abstract

The study addressed the working memory capacity (WMC) of gesture-command associations in gestural interaction and investigated the impact of compatibility between users’ mental models and the predefined gesture-command associations on WMC. Gestural interaction is a popular representative of natural interactions. Although gestural interaction intends to be natural, it has been criticized for not being so. One of the critical problems lies in learning and memorizing. WMC is a pivotal bottleneck that underlies learning and memorizing gesture-command associations, yet it remains unknown. Two standardized paradigms were used to estimate the WMC of gesture-command associations: change-detection task in Experiment 1, and span task in Experiment 2. Besides, we further examined the impact of compatibility on WMC. We found that two gesture-command associations can be retained in working memory under low-compatibility conditions, while three to five associations can be retained under high-compatibility conditions. The result implies that WMC of gesture-command associations is highly limited, while this cognitive limitation could be reduced by promoting the compatibility between users’ mental model and the predefined gesture-command associations. Designers for gestural interactions may require users to memorize two to five gesture-command associations, considering different application scenarios.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by Science and Technology Innovation 2030 Program [2018AAA0101605], Fundamental Research Funds for the Central Universities National, Natural Science Foundation of China Grants [31771202], and Key Program of Natural Science Foundation of Zhejiang Province [LZ20C090001].

Notes on contributors

Qi Gao

Qi Gao is a PhD candidate for psychology at Zhejiang University in Hangzhou. She received her Bachelor of Science in psychology from Nankai University and Bachelor of Management in business administration from Tianjin University in 2019.

Zheng Ma

Zheng Ma is a fourth year PhD candidate at the department of Industrial and Manufacturing Engineering in the Penn State University. He recieved a B.S. in Psychology from Zhejiang University in 2018.

Quan Gu

Quan Gu obtained a PhD of cognitive psychology and engineering psychology from Zhejiang University in 2021.

Jiaofeng Li

Jiaofeng Li received a bachelor’s degree in psychology at Sun Yat-sen University in 2018. At present, she is a PhD candidate in applied psychology at Zhejiang University.

Zaifeng Gao

Zaifeng Gao is a professor at Zhejiang University, Hangzhou, and obtained Ph.D from Zhejiang University, Hangzhou, China, 2009. His research focuses on engineering psychology and cognitive psychology.