Use of the Microsoft Kinect system to characterize balance ability during balance training

Abstract

The risk of falling increases significantly in the elderly because of deterioration of the neural musculature regulatory mechanisms. Several studies have investigated methods of preventing falling using real-time systems to evaluate balance; however, it is difficult to monitor the results of such characterizations in real time. Herein, we describe the use of Microsoft’s Kinect depth sensor system to evaluate balance in real time. Six healthy male adults (25.5±1.8 years, 173.9±6.4 cm, 71.4±6.5 kg, and 23.6±2.4 kg/m²), with normal balance abilities and with no musculoskeletal disorders, were selected to participate in the experiment. Movements of the participants were induced by controlling the base plane of the balance training equipment in various directions. The dynamic motion of the subjects was measured using two Kinect depth sensor systems and a three-dimensional motion capture system with eight infrared cameras. The two systems yielded similar results for changes in the center of body mass (P>0.05) with a large Pearson’s correlation coefficient of γ>0.60. The results for the two systems showed similarity in the mean lower-limb joint angle with flexion–extension movements, and these values were highly correlated (hip joint: within approximately 4.6°; knee joint: within approximately 8.4°) (0.40<γ<0.74) (P>0.05). Large differences with a low correlation were, however, observed for the lower-limb joint angle in relation to abduction–adduction and internal–external rotation motion (γ<0.40) (P<0.05). These findings show that clinical and dynamic accuracy can be achieved using the Kinect system in balance training by measuring changes in the center of body mass and flexion–extension movements of the lower limbs, but not abduction–adduction and internal–external rotation.

Keywords:

• balance ability
• balance training
• motion capture system
• Kinect system
• fall prevention

Introduction

Elderly persons typically exhibit poor balance due to deterioration of the regulatory mechanisms of the neuromuscular system associated with aging,¹ and are at significant risk of injury due to falling.² One in three over 65 years of age suffer a fall over the course of 1 year.^3,4 Furthermore, secondary injuries – such as fractures, joint dislocations, concussion, and severe lacerations – may result from a fall, and can worsen or diminish the ability to perform physical activities. If such injuries are severe, they can potentially result in social isolation, followed by mental health problems.⁵ Approximately 30% of those aged 65 to 80 years and approximately 40% of those over 80 years experience falling accidents.⁶ In two thirds of those over 85 years, falling accidents are directly related to secondary injuries, which in some cases result in fatalities.⁶ In USA, the annual medical costs associated with falling accidents for those aged over 65 years total more than 20 billion dollars.⁷ In Korea, it has been estimated that more than 3 billion dollars are spent annually on medical treatment related to falling accidents in the elderly and, taking into consideration forecasts of changes in demographics and income levels, these medical costs are expected to increase considerably in the near future.

Several studies have been carried out with the aim of preventing falling accidents.^8–13 Methods for improving the nervous system and musculoskeletal functions involving placing subjects on variable base planes have been widely used to prevent falling accidents by improving balance ability.^14–17 In addition, analyses of the characteristics of changes in the center of body mass (COM), center of body pressure (COP), and range of joint motion have been carried out to evaluate balance ability from a biomechanical perspective,^18–21 and advanced balance indices have been developed.^22–25 Hur et al²¹ proposed a balance index based on changes in the COM and COP that occur following a perturbation caused by application of loads in the posterior direction of a subject’s pelvis. Mayagoitia et al²² and Yang and Hsu²³ proposed a balance index that utilizes characteristic changes in the COP, as well as changes in acceleration and velocity, measured for a standing position using inertial sensors. Clark et al²⁴ proposed a balance index that utilizes weight-bearing asymmetry and COP in combination with loads on both feet, and analyzed the characteristic changes in the balance index, with the aim of evaluating balance ability. However, to evaluate balance ability, specialized test equipment is typically required, such as expensive three-dimensional (3D) motion capture systems that use infrared cameras or inertial sensors. In addition, these systems exhibit limitations related to the space required for test equipment installation and the availability of specialized human resources, as well as limitations related to the ability to present the results using real-time data analysis. Several attempts have been made in recent years to address these limitations using systems for balance training and evaluation that make use of the relatively inexpensive Microsoft Kinect depth sensor system.^26–29

Balance training using aperiodic and complex movements in the basal plane is an effective technique; however, few studies have evaluated whether balance ability can be effectively assessed using the Kinect system. Considering the range and specificity of motion required during balance training for rehabilitation, it is important to assess the accuracy of balance ability evaluations. Clark et al²⁸ reported the successful use of the Kinect system to analyze gait; however, this work was limited to simple gaits that generated periodic movements, with no movements in the basal plane. We cannot assume that the results obtained by Clark et al²⁸ apply during balance training with aperiodic and complex movements and with various movement conditions. In this study, we evaluate the use of the Kinect system to evaluate balance ability in real time during balance training with complex aperiodic movements in the basal plane.

Materials and methods

Subjects

Six male subjects with no musculoskeletal or neurological disorders aged 25.5±1.8 years participated in this study. Their mean height was 173.9±6.4 cm, mean body weight was 71.4±6.5 kg, and mean body mass index was 23.6±2.4 kg/m². The subjects were fully informed of the test methods and the risks involved, and consent was given prior to participating in the experiments. In addition, the experimental procedures used in this study and the selection of the participants were approved by Sejong University Institutional Review Board (IRB No. SJU-2015-002).

Experimental configuration and measurements to induce dynamic movements

Figure 1 shows the in-house-manufactured balance training equipment used to induce dynamic movements in the subjects. The balance training equipment could be rotated around various axes (an arbitrary axis) with a range of ±15° of the maximum on the footrest part. Six links were used to connect the footrest part, which could move ±0.1 m in the vertical direction. The tests were designed to control the basal plane in an arbitrary direction. For safety reasons, the range of rotation angles was limited to ±9° around the axis of rotation. Subjects were allowed to move dynamically within a range that allowed them to maintain their balance. Tests were repeated three times per subject to minimize measurement errors, and a 10-minute rest period introduced between tests was to minimize the effects of fatigue.

Figure 1 Experimental configuration used to generate and characterize the motion of the participants.

Two Kinect systems (Kinect for Xbox 360; Microsoft Corporation, Redmond, WA, USA) were used to measure the movements of the subjects, which had a sampling rate of 300 Hz. We also used a 3D motion capture system with eight infrared cameras (T-10s; VICON Motion System Ltd., Oxford, UK), which had a sampling rate of 100 Hz. The infrared camera system was used to provide a reference, with which we might assess the accuracy of the measurements obtained using the Kinect system. Changes in the COM and the angles of the main lower limbs (ie, the knee and hip joints) were analyzed using the information acquired from each system to quantify the movements of the subjects. Analysis of the movements captured using the Kinect system was implemented using the iPi motion capture studio (iPi soft, Moscow, Russia) with the Biomech add-on (iPi soft). Analysis of the movements recorded using the infrared cameras was implemented using VICON Nexus software, Ver. 1.8.3 (VICON Motion System Ltd.). Changes in the COM and main joint angles measured using each system were compared.

Statistical analysis

To assess the significance of the differences in the COM and joint angles measured using the Kinect system and the infrared camera system, we used SPSS Ver.19.0. (IBM Corporation, Armonk, NY, USA) to implement Student’s t-tests. The significance level was P<0.05, and an asterisk is used to indicate results that differed significantly between the two systems. The correlation between the movements measured using the two systems was evaluated using Pearson’s correlation coefficient, γ : if γ<0.40, the correlation between the two systems may be considered low; if 0.40<γ<0.74, the correlation may be considered high; and if γ>0.75, the correlation may be considered very high, so that for practical purposes the two systems may be considered to yield identical results.²⁸

Results

Changes in COM

Figure 2 shows changes in the COM determined using the Kinect system and the 3D infrared camera system during movements of the subjects, and Table 1 lists a summary of these data. The measured changes in the COM of the subjects that were required to maintain balance were almost identical using the two systems; ie, P_x=0.22, P_y=0.30, and P_z=0.13. The differences in the COM measured using the Kinect system were 135.07±22.06 mm in the medial–lateral axis (x-axis), 98.02±18.27 mm in the anterior–posterior axis (y-axis), and 67.49±32.4 mm in the cranial–caudal axis (z-axis); using the infrared camera 3D motion capture system, we found 118.82±21.22 mm in the x-axis, 118.81±42.46 mm in the y-axis, and 43.84±12.26 mm in the z-axis. The measurements using the two systems were strongly correlated, and we found γ_x=0.61, γ_y=0.59, and γ_z=0.66, which shows that the Kinect system provided a similar characterization of the movements of the subjects to the 3D infrared camera system.

Table 1 Summary of the center of body mass (COM) and joint angles measured using the Kinect system and the 3D infrared camera system

Balance index	Axis	Depth sensor-based Kinect system	Infrared camera-based motion capture system	P-value	γ
COM alteration (mm)	x (medial–lateral)	135.07±22.06	118.82±21.22	0.22	0.61
	y (anterior–posterior)	98.02±18.27	118.81±42.46	0.3	0.59
	z (cranial–caudal)	67.49±32.4	43.84±12.46	0.13	0.66
Hip joint alteration (°)	x (medial–lateral)	20.43±4.76	17.14±3.89	0.22	0.73
	y (anterior–posterior)	14.14±7.53	13.52±2.31	0.85	0.23
	z (cranial–caudal)	37.48±30.67	18.25±7.53	0.17	0.14
Knee joint alteration (°)	x (medial–lateral)	31.09±22.23	25.82±5.46	0.58	0.42
	y (anterior–posterior)	16.81±8.88	12.34±7.53	0.37	0.18
	z (cranial–caudal)	34.61±26.61	12.63±6.13	0.08	0.32

Note: Data are presented as mean ± standard deviation.

Figure 2 Changes in the center of body mass (COM).

Notes: (A) In the medial–lateral direction (x-axis). (B) In the anterior–posterior direction (y-axis). (C) In the cranial–caudal direction (z-axis). (D) A 3D representation of the alteration of COM.

Changes in the lower-limb joint angles

Figure 3 shows changes in the hip joint angles during movements of the subjects, and Figure 4 shows changes in the knee joint angles measured using both the Kinect system and the 3D infrared camera system. Table 1 lists a summary of these data. For the hip joints, the changes in the joint angles measured using the Kinect system were 15.67°–25.19° in the x-axis (flexion–extension), 6.61°–21.67° in the y-axis (abduction-adduction), and 6.81°–68.15° in the z-axis (internal–external rotation), whereas the changes measured using the infrared camera system were 13.25°–21.03° in the x-axis, 11.21°–15.83° in the y-axis, and 10.72°–25.78° in the z-axis. For knee joints, the changes in the joint angles measured using the Kinect system were 15.27°–24.37° in the x-axis, 7.93°–25.69° in the y-axis, and 8°–61.22° in the z-axis, whereas those measured using the infrared camera system were 20.36°–31.28° in the x-axis, 4.81°–19.87° in the y-axis, and 6.5°–18.76° in the z-axis. The differences in the measured lower-limb joint angles in the x- and y-axes were small (within 4.6° at the hip joint, and within 8.4° at the knee joint); however, the differences in the lower-limb joint angles in the z-axis were larger (within 19.1° for the hip joint, and within 16.3° for the knee joint). No significant changes were found between the mean lower-limb joint angles measured using the two systems (for the hip joint we found P_x=0.22, P_y=0.85, and P_z=0.17, and for the knee joint we found P_x=0.58, P_y=0.37, and P_z=0.08); however, the results of the correlation analysis for changes in the mean lower-limb joint angles measured using the two systems revealed high degrees of correlation in the x-axis (γ=0.73 for the hip joint and γ=0.42 for the knee), but low degrees of correlation in the y- and z-axes (γ<0.40 for both the hip and knee joints). Therefore, changes in the lower-limb joint angles as measured using the Kinect system were in agreement with those from the 3D infrared camera system only for flexion–extension in the x-axis.

Figure 3 Changes in the joint angles of the hip.

Notes: (A) During flexion–extension in the medial–lateral direction (x-axis). (B) During abduction–adduction in the anterior–posterior direction (y-axis). (C) During internal-external rotation in the cranial–caudal direction (z-axis).

Figure 4 Changes in the joint angle of the knee.

Notes: (A) During flexion–extension in the medial–lateral direction (x-axis). (B) During abduction–adduction in the anterior–posterior direction (y-axis). (C) Internal–external rotation in the cranial–caudal direction (z-axis).

Discussion

The Kinect system provided accurate measurements of the changes in the COM and for flexion–extension movements of the lower-limb joints; however, the measurement accuracy for abduction–adduction and cranial–caudal movements was poor. The reasons for this limitation are as follows. First, the Kinect system was originally developed for use as a game controller, with the primary goal of measuring dynamic movements of subjects in the frontal plane (ie, the x–z plane in this work), using 640×480 pixels, which is a relatively low resolution.²⁹ It is therefore difficult to measure changes in joint angles in the y-axis of the y–z plane (ie, the sagittal plane) or in the z-axis of the x–y plane (transverse plane), which differ from the frontal plane. Second, the Kinect system was developed with algorithms to determine the joint locations based on depth image data measured using cameras fixed on the ground.³⁰ For this reason, in balance training environments where the basal plane may vary, the Kinect system is inherently limited in terms of its ability to measure changes in joint angles and movements, whereas the 3D infrared camera system can measure movements of subjects using location markers attached to the subjects.³¹ The next-generation Kinect system (ie, Kinect v2.0) has recently been developed to overcome these limitations; however, the measurement accuracy should be evaluated to determine whether it can be useful for measuring balance ability.

Conclusion

Balance training environments commonly involve complex aperiodic movements produced in response to motion generated using a variable basal plane. We have investigated the use of the Microsoft Kinect depth sensor system to evaluate balance ability. We found that the Kinect system was effective in accurately characterizing changes in the COM and in flexion–extension movements of the lower limbs during balance training. However, we found that the Kinect system was not suitable for use in balance training systems that require in-depth analyses of the joint motions. The Kinect system is therefore expected to be useful for balance training systems that require characterization of the changes in the COM and the joint angles during flexion–extension movements.

Acknowledgments

This research was supported by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) and funded by the Ministry of Health and Welfare of the Republic of Korea (grant number: HI12C0089).

Disclosure

The authors report no conflicts of interest in this work.