An Investigation of 3D Human Pose Estimation for Learning Tai Chi: A Human Factor Perspective

ABSTRACT

In this article, we propose a Tai Chi training system based on pose estimation using Convolutional Neural Networks (CNNs) called iTai-Chi. Our system aims to overcome the disadvantages of insufficient accurate feedback in traditional teaching methods such as one-to-many tutorial and video watching. With the specially trained neural network, our iTai-Chi system can estimate learners’ poses more accurately compared to Kinect V2. In our system, user’s motion is evaluated through comparison with the template motion. The evaluated results are presented to the user to locate the error in their motions and help their correction. To verify the effectiveness of our system, we carried out a series of user studies. Results reflect that the iTai-Chi system successfully improve users’ performance in movement accuracy. Also, our system assists elder Tai Chi practitioners and students without prior knowledge to overcome learning obstacles and improve their skills. The users agreed that our system is interesting and supportive for their Tai Chi learning.

Additional information

Funding

The work is supported by the National Natural Science Foundation of China [No. 61872241, 61671290, 61572316], the Macau Science and Technology Development Fund [No. 0027/2018/A1], the National Key Research and Development Program of China [No. 2017YFE0104000, 2016YFC1300302], and the Science and Technology Commission of Shanghai Municipality [No. 17411952600, 16DZ0501100].

Notes on contributors

Aouaidjia Kamel

Aouaidjia Kamel is currently a Ph.D. candidate in Department of Computer Science and Engineering, Shanghai Jiao Tong University, China. He obtained his M.Eng. degree in computer science from the Abbes Laghrour University of Khenchela, Algeria. His research interests focus on human–machine interaction, human pose estimation, and machine learning.

Bowen Liu

Bowen Liu is currently a Ph.D. candidate at The Education University of Hong Kong. He obtained his bachelor’s degree from Jinan University, China, and had an experience as an exchange student at Aalborg University, Denmark. His interests include image processing, autonomous vehicle, and smart transportation systems.

Ping Li

Ping Li is currently an assistant professor at Macau University of Science and Technology, who obtained his Ph.D. degree from The Chinese University of Hong Kong. His research interests include creative media, virtual reality, and computer graphics. He has excellent research project reported worldwide by ACM TechNews.

Bin Sheng

Bin Sheng is currently an associate professor in Department of Computer Science and Engineering, Shanghai Jiao Tong University, who received his Ph.D. degree from the Chinese University of Hong Kong. He serves as an Associate Editor of IET Image Processing. His research interests include computer graphics and machine learning.