Exploring the Potentials of Crowdsourcing for Gesture Data Collection

Abstract

Gesture data collection in a controlled lab environment often restricts participants to performing gestures in a uniform or biased manner, resulting in gesture data which may not sufficiently reflect gesture variability to build robust gesture recognition models. Crowdsourcing has been widely accepted as an efficient high-sample-size method for collecting more representative and variable data. In this study, we evaluated the effectiveness of crowdsourcing for gesture data collection, specifically for gesture variability. When compared to a controlled lab environment, crowdsourcing resulted in improved recognition performance of 8.98% and increased variability for various gesture features, eg, a 142% variation increase for Quantity of Movement. Integrating a supplemental gesture data collection methodology known as Styling Words increased recognition performance by an additional 2.94%. The study also investigated the efficacy of gesture collection methodologies and gesture memorization paradigms.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Official tutorial video (come), https://youtu.be/suFs0nftprc

2 OpenPose output format (BODY_25), https://cmu-perceptual-computing-lab.github.io/openpose/web/html/doc/md_doc_02_output.html

Additional information

Funding

This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2019-0-01842, Artificial Intelligence Graduate School Program (GIST)) (30%) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2021R1A4A1030075) (70%).

Notes on contributors

In-Taek Jung

In-Taek Jung received the BS degree in computer science from Jeonbuk National University, and the MS degree in Cultural Technology from the School of Integrated Technology, GIST, and he is currently a PhD student in the AI Graduate School, GIST. His research interests are data collection, education, AI applications for human being.

Sooyeon Ahn

Sooyeon Ahn received the BArch. degree from Dankook University, and the MSc degree in Architecture and Civil Engineering from University of Bath. She received the MS degree in Cultural Technology from the School of Integrated Technology, GIST and is currently a PhD student. Her research interests are multimodal interaction design.

JuChan Seo

Juchan Seo received the BS degrees in mechanical engineering from Handong University, Phang, Korea, He is currently integrated course with the Artificial Intelligence Graduate School, GIST Gwangju, Korea. His research interests include Explainable AI, AI applications for human beings.

Jin-Hyuk Hong

Jin-Hyuk Hong received BS, MS, and PhD degrees in computer science from Yonsei University, Korea. He is currently an assistant professor with the School of Integrated Technology and the AI Graduate School, GIST. His research interests include context awareness, pattern recognition, interaction design, focusing on the understanding of human behaviors.