An Augmented Reality-Assisted Therapeutic Healthcare Exercise System Based on Bare-Hand Interaction

ABSTRACT

Augmented Reality (AR)-assisted exercises for the enhancement of finger functions have been developed for many years and have proven to be effective. Most applications tracked the motion of upper-extremity using handheld trackers, which are not convenient to put on and take off. Bare-hand tracking algorithms have been utilized to address this limitation; however, only few AR-assisted healthcare approaches have been designed to train upper-extremity skills using bare-hand tracking, and even fewer approaches detect the Range of Motion (ROM) of fingers with low-cost devices. This paper presents a low-cost and multi-modal residential-based AR-assisted therapeutic healthcare exercise system. A computer vision-based bare-hand interaction method is proposed in this system. This method is designed to estimate finger bending degrees through their projection lengths. With an accurate feature detection strategy, this method is able to detect the full ROM using two web cameras. This system also incorporates a vibration wrist band to stimulate the users with tactile feedback, and an assessment module to evaluate user performance.

Additional information

Notes on contributors

M. Y. Zhao

M. Y. Zhao received a BS degree in Biomedical and Information Engineering from Northeastern University, Shenyang, P.R.C., in 2010, specializing in lung segmentation in CT images. She is currently a PhD student in Mechanical Engineering, National University of Singapore, Singapore, specializing in augmented reality-assisted healthcare exercising system for upper-extremities.

S. K. Ong

S. K. Ong is an associate professor in the Mechanical Engineering Department, National University of Singapore. Her research interests are virtual and augmented reality applications in manufacturing, and assistive technology. She has a Google Scholar citation of over 3919 and H-Index 36. She is a Fellow of CIRP, The International Academy for Production Engineering, where she is the 4th female fellow to be elected.

A. Y. C. Nee

A. Y. C. Nee is a professor of manufacturing engineering, Department of Mechanical Engineering, National University of Singapore. He is a Fellow of the International Academy for Production Engineering (CIRP) and served as its president in 2012, a Fellow of the Society of Manufacturing Engineers (USA) and its Gold Medal recipient in 2014. He is the Editor-in-Chief of the International Journal of Advanced Manufacturing Technology (Springer).