Reliability of the head tracking measured by Microsoft Hololens during different walking conditions

1

Keywords:

• Motion capture
• biomechanics
• headset tracking
• augmented reality

1. Introduction

Augmented reality (AR) is a technology that expand the real environment by adding digital holograms into it. AR appears to be a promising field of development for serious games, especially for walking rehabilitation applications. The HoloLens device is one of the most popular and advanced AR Head Mounted Display (HMD). It includes optical and inertial sensors for position and orientation tracking. Although the algorithm combining the sensors information in the HoloLens can estimate the headset’s pose in an absolute reference frame, its reliability in walking condition is undocumented. Yet, the first step to develop high realistic application for gait rehabilitation, using position, velocity and acceleration of the headset, is to assess the reliability of the headset tracking, in comparison with a reference motion analysis tracking system, during different conditions. We hypothesis that the accuracy of the Hololens is sufficient to measure the position and the velocity of the headset, and the user by extension, without spatio-temporal drift.

2. Methods

2.1. Data collection

During 9 walking trials in 3 different conditions the 3 D raw coordinates of the HMD were measured simultaneously by the HoloLens and a VICON motion capture system (MOCAP) both at 100 Hz:

• Condition 1 – “L-shaped” walking path (5 x 2,5m) at three different speeds (self-selected, low and fast). Three repetitions.

• Condition 2 – random walking path during 2 minutes. Three repetitions.

• Condition 3 – same as condition 1 with intercalated 2 minutes walking exits outside off the laboratory between each repetition.

The Hololens tracking was performed with an application created with Unity. This app contained a script which logged the position of the HMD and Spatial Mapping prefab which provided a detailed representation of real-world surfaces in the environment around the HoloLens. A MOCAP system was used to track 5 retro-reflective markers (19 mm) attached to the headset, with 15 infrared cameras.

2.2. Data analysis

The Hololens signals were synchronized to the MOCAP signals by an automatic time shifting using a local minimum detection. An ICP algorithm (Besl and McKay 1992; Chen and Medioni 1992) was used to align the 3D positions given by the two systems minimizing the distance between them, using geometric transformations (rotations R and translations T). The ICP algorithm has two steps: The first step consists of determining the correspondence pairs ($p,\mathrm{ }m$) from two data sets $H$ and $G.$ The aim is to find for each point $p$ in $H$ its closest point in $G.$ The second step is to apply a transformation (R and T) in order to minimize the distance between the correspondence pairs: $$E\left(R,t\right)=\mathrm{ }\frac{1}{N_H}\sum _{i=1}^{N_H}{‖G_i-R{H}_i-t‖}^2$$

With $H\left\{xH\mathrm{ }yH\mathrm{ }zH\right\}$ and $G\left\{xG\mathrm{ }yG\mathrm{ }zG\right\}$ the corresponding points, from Hololens and from MOCAP respectively. These two steps are repeated until the error is below a given threshold or until the maximum number of iterations is reached.

Before components and norm velocities computations, trajectories of both devices were low pass filtered with a fourth-order Butterworth filter with a cut-off of 6 Hz.

The correlation between the position and the velocity of the headset given by the Hololens and the MOCAP system was calculated with Pearson’s correlation coefficient. The distances between the Hololens and the MOCAP measured parameters were computed by root mean squared error (RMSE).

3. Results and discussion

The ICP algorithm permitted to compare a headset position data from Hololens with a model shape gived by MOCAP system, after the shape was moved (registered, positioned) to be in best alignment (Figure 1). For trials 1 to 3, realized in condition 1, correlation coefficients between $H_i\left\{{xH}_i{\mathrm{ }yH}_i\mathrm{ }{zH}_i\right\}$ and $G_i\left\{{xG}_i{\mathrm{ }yG}_i\mathrm{ }{zG}_i\right\}$ were quite close to 1. Minimal errors between signals were about 10⁻² to 10⁻⁴. Maximal errors were of 55 to 250 mm. Maximal errors concerned antero-posterior axis. RMSE was from 9 to 53 mm. Speed variation between the different sets didn’t influence the accuracy of the Hololens tracking. There was no signal drift over the time. The error increased when there was a sudden shift of trajectory. Results obtained in condition 2 were similar. For trials 7 to 9, realized in condition 3, signal processing was more complex. Indeed, for each trial, the first laboratory exit systematically generated a “jump” on ${xH}_{\mathit{exit}}$ and ${yH}_{\mathit{exit}},$ for no apparent reason. For example, in trial 9, there was a drift of 0.45 m (medio-lateral axis) and 40 m (antero-post axis). No drift for vertical axis. However, these calibration movements were realized at the same place inside the laboratory. An hypothsis shoud be the Spatial Mapping function, which could be influence the position and orientation in space. The corridor route was not pre-mapping unlike the laboratory. Nevertheless, after this offset was eliminated, the headset tracking with Hololens presented the same good results than in condition 1 and 2. No signal drift over the time again (Table 1).

Figure 1. 3D Representation of headset trajectory with Hololens and VICON tracking for Trial 1 before ICP (top), and after ICP (bottom).

Table 1. Averaged results for all trials in each experimental conditions. Errors are given in mm.

		X	Y	Z
Cond. 1	r	1,00	1,00	0,99
	max Err	98	251	81
	min Err	21	25	6
	mean Err	24	44	6
	RMSE	32	50	9
Cond. 2	r	1,00	1,00	0,99
	max Err	75	117	105
	min Err	13	14	6
	mean Err	14	24	8
	RMSE	19	28	10
Cond. 3	r	1,00	1,00	0,95
	max Err	113	146	55
	min Err	24	27	6
	mean Err	24	42	6
	RMSE	35	53	10

For each trials, correlation coefficients for the norm of the HMD vector velocity were high (between 0.96 and 0.99) (detailed not presented in this paper).

4. Conclusions

The accuracy of the Hololens is sufficiently high to evaluate the position and the velocity of the headset, and the user by extension, without time drift. There was however uncertainty as to the exact drift of the Hololens signal, when the environment changed. This problem, with no obvious explanation, should not stop to develop gait rehabilitation application using position and velocity of the headset given by the Hololens.

Acknowledgments

The authors thank the Fondation Ellen Poidatz and the SESEP for the support to this research.