ABSTRACT
The Oculus Quest 2 (OQ2; Meta) has potential for studying and training locomotion, but data quality must be ensured. This research aimed to compare OQ2 measurements to a motion capture system and assess its test-retest reliability. Position and orientation data of the OQ2 were extracted and compared to a Vicon system as gold standard during dynamic and static conditions. The criterion validity in static measurements showed absolute maximum errors around 1 cm and 1°. Despite excellent correlation values during dynamic conditions, accuracy was worse than static with up to 19 cm and 14° absolute max error. The calibration test-retest values showed absolute max errors around 1 cm and 1°. These findings provide insights for using OQ2 for assessing locomotion.
1. Introduction
Virtual reality (VR) is a promising tool for gait and balance rehabilitation (Porras et al. Citation2018). The gaming context, level of immersion, and diversity of scenarios that can be presented, as well as the ability to provide feedback, increase the commitment of individuals with varying functional abilities (e.g. acquired brain injuries, Li et al. Citation2016); older adults, Amorim et al. (Citation2019). Locomotor skills are critical to safe and independent mobility in the community (Jahnsen et al. Citation2004) and VR can improve these abilities by immersing participants in more ecological environments or scenarios with transfer to the real world (Kim et al. Citation2019). VR training of locomotor skills remains difficult because of the required body displacement. Many current head-mounted displays (HMDs) can only be used over restrained areas due to the need to be tethered or relying on external, camera-based units. Some solutions are to use joysticks, treadmill systems or to walk in place (Mousas et al. Citation2021), however, the physical and cognitive requirements are not the same as when walking overground, even for a treadmill (Lee and Hidler Citation2008; Row Lazzarini and Kataras Citation2016).
The introduction of standalone HMDs with all components contained within the device provides the opportunity to move over greater distances. The Oculus Quest 2 (OQ2; now the Meta Quest 2; Menlo Park, USA) is currently one of the most popular standalone HMDs on the market (Needham Citation2022). In contrast with many previous models, the OQ2 tracks its position and orientation and simultaneously presents the virtual environment wirelessly. The OQ2 uses algorithms along with data from an Inertial Measurement Unit (IMU) to calculate linear acceleration and angular orientation. Four cameras help to generate a 3D map of the room and to compensate the potential drift of the IMU (Hesch Citation2019). This camera-based technology called Inside-Out (Ribo et al. Citation2001) provides a cheaper and an easier option compared to a traditional VR headset using Outside-In technology with sensors placed in specific locations around the HMD.
Getting accurate position and orientation data is essential for research and to provide measures during clinical assessments or training. To our knowledge, the psychometric properties of OQ2 data has only been tested in static conditions or over short distances (Mulvany et al. Citation2020; Holzwarth et al. Citation2021). These studies showed that the OQ2 when used with a grilled pattern on the floor has very good criterion validity compared to gold standard motion capture. Holzwarth et al. (Citation2021) and Mulvany et al. (Citation2020) found maximal differences of 0.6 cm and 0.4 cm, respectively, for static measures. For dynamic measures over short distances, Mulvany et al. (Citation2020) found an average drift of between 0.64 and 5.83 cm depending on the environmental setup. However, psychometric properties of these measurement tools must be evaluated over greater areas to ensure their functionality and the safety (calibration to space) for the participant. Therefore, the purpose of this study was to assess the criterion validity and the test-retest reliability of the position and orientation extracted by the OQ2, in comparison with a gold standard optoelectronic motion capture system, in both static and dynamic situations over several metres.
2. Methods
2.1. Data collection
All measurements were carried out in the Mobility Laboratory at the Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris). A custom platform developed at the Cirris, used the OQ2 (version 37.0) tracking system (90 Hz), a virtual environment simulating a public park (created in the UnReal Gaming Engine V4.26) and an autonomous local Wi-Fi link to an independent server using a single board computer (Raspberry Pi 4th,4 GB; Cambridge, UK). For the purpose of this study, the extracted data from the OQ2 were compared to data from a Vicon (Oxford, UK) optoelectronic motion capture system (10 cameras; 90 Hz) used as a gold standard for comparison. The Vicon system tracked four non-collinear, retro-reflective markers (14 mm) attached to the front of the OQ2. Each system’s axes were aligned with their origins separated by 5 metres. Temporal synchronisation of the two systems was achieved using an intentional, rapid, vertical movement by the participant during static measurements and a rotation movement during dynamic measurements. Due to the rendering of different parts of the scene with varying complexity (e.g. number of polygons involved for leaves in a tree, etc.), there were some irregularities in the OQ2 sampling rate. Therefore, a timeline was incorporated into the OQ2 data files that was used to resample these data at a rate of 90 Hz. Subsequently, using the ‘findsignal’ function in MatLab, the headset movement from the gold standard Vicon data was identified in the OQ2 data and the difference in time noted and then used to align both data sets in relation to these movements (vertical movement for static conditions; rotational movement for dynamic conditions). This enabled the establishment of consistent and reliable synchronisation for each trial.
2.1.1. Criterion validity assessment
A 3D-printed plastic support was fixed on a tripod, as previously reported by Mulvany et al. (Citation2020). Two plumb lines were attached on each side of the plastic support and aligned with marks on the floor to assure that the orientation and position of the HMD placed on the support were standardised for calibration and static conditions. Calibration consisted of resetting the referential orientation and position of the HMD in the laboratory, and, thus, the virtual environment within it. To validate static measures, the HMD was placed on the support and calibrated. Then data were collected for 10 s. Without recalibrating, data collection was repeated 10 times. Between each placement on the support, the HMD was donned by someone who walked at a comfortable speed from a common starting point to four different points over approximately 20 m for each path (). This constituted the dynamic trials which were also repeated 10 times for a total of 10 static and 40 walking (10 × 4 directions) trials. Due to a smaller area covered by the Vicon System (see ) compared to the size of the total walking area within the programmed scene, validation for the dynamic conditions was carried out over approximately the last 6 metres for each trajectory.
Figure 1. Schema of the laboratory setup and the trajectories used during data acquisition.
2.1.2. Calibration test-retest reliability assessment
In a separate session, the HMD calibration procedure was repeated for 10 trials to assess test-retest reliability. For each calibration collection, the HMD was placed on the tripod support to reset the position and orientation of the virtual environment within the helmet coordinates as well as with respect to the built environment, and then data were collected for 10 s. Between trials, the HMD was removed from the support and then replaced for the next collection.
2.2. Data analysis
As noted above, data extracted from the OQ2 were resampled at 90 Hz. The OQ2 and Vicon data were then filtered with a second order, dual-pass, Butterworth filter with a cut-off frequency of 9 Hz. Due to differences in the origins of the HMD and Vicon systems, the OQ2 origin was relocated to the centre of the four-marker rigid body on the front of the OQ2 which then was directly related to the Vicon origin.
For criterion validity assessment in static and dynamic conditions, errors were obtained by calculating the absolute error between the data collected with the OQ2 from those acquired with the Vicon system. Absolute mean and maximum errors were then calculated for each position and orientation. For the dynamic conditions, the relationship between the position of the headset given by the OQ2 and the Vicon system was assessed using the Pearson’s correlation coefficient (poor <0.40, modest 0.40–0.74, excellent >0.74; Fleiss Citation2011). For the dynamic condition, Friedman Repeated Measure ANOVAs (Jamovi, Australia) were performed to evaluate the differences in trajectories related to absolute mean difference and absolute maximal error as well as to assess the drift effects through time for these variables. Significance level was set at p < 0.05.
For the calibration reliability assessment, the absolute difference of the position and orientation measurements from the OQ2 between the first trial and each of the subsequent trials 2 to 10 were calculated and the absolute mean and absolute maximum test-retest errors were extracted.
3. Results
3.1. Criterion validity in static conditions
Results showed that the mean absolute errors between the measures in position ranged from 0.24 to 0.45 cm and the maximum absolute errors from 0.48 to 1.01 cm. The mean absolute difference for the orientation varied between 0.09° and 0.34° and the maximal absolute errors ranged from 0.49° to 1.22° ().
Table 1. Relationships between OQ2 and Vicon measurements in static conditions.
3.2. Criterion validity in dynamic conditions
There were no significant differences in the absolute mean and maximum errors between the different walking trajectories (p-values ranging from 0.150 to 0.905) or across repetitions (p-values ranging from 0.154 to 0.784). Therefore, the data from the four dynamic conditions were combined. Using these combined data, it was observed that the offset of the OQ2 was greater for dynamic measures compared to the static measures. Correlation coefficients (r) between OQ2 and Vicon measurements were mostly close to 1 () in the dynamic conditions showing the general ability of the OQ2 to follow the movements. However, absolute maximal position errors were as large as 13.1 cm in the longitudinal axis and 19.3 cm in the mediolateral axis. The absolute mean position error varied between 1.20 and 5.21 cm. The absolute mean error of orientation measurement for the yaw (Z) was 0.54° and the absolute maximum error was 2.69°. However, the pitch (X) and roll (Y) orientations were less accurate with an absolute mean error of, respectively, 4.52° and 4.17°, and an absolute maximum error between 11.1° and 14.1°.
Table 2. Relationships between OQ2 and Vicon measurements in dynamic conditions.
3.3. Test-retest reliability in static conditions
The offset in position for repeated calibrations had mean absolute errors between 0.76 and 1.14 cm and maximal absolute errors varying between 0.98 and 2.20 cm. The absolute mean deviation for the orientation was 0.22° or less and the maximum offset was 1.79° ().
Table 3. Descriptive statistics of the position and orientation absolute errors during test-retest reliability.
4. Discussion
The present study aimed to compare the position and orientation measurements of the OQ2 HMD with a gold standard motion capture as well as to study its test-retest reliability to understand if the OQ2 can provide accurate and reliable data for locomotor navigation. The criterion validity data of the OQ2 during the static conditions showed interesting results with offsets of around 1 cm compared to Vicon data. During the dynamic conditions, the quality of the localisation measurements dropped, and we observed offsets as large as 19 cm along the mediolateral axis.
For the static conditions, similar results were found by Holzwarth et al. (Citation2021) who showed a maximal absolute error of 0.6 cm using the Oculus Link™ which allows the HMD to be run from a separate PC computer. This use of an external PC computer may enhance accuracy of the OQ2 by freeing up CPU resources. Considering both the results of Holzwarth et al. (Citation2021) and those of the present study confirms that the OQ2 can provide good kinematic data in static conditions both with and without the Oculus Link™.
For dynamic conditions, the present study showed some substantial drift (up to 19 cm) during data collection that could potentially result in errors in anchoring the environment with the OQ2 inside-out system while walking. The current protocol actually involved walking between different rooms which had different features, and this might have reduced the accuracy of the OQ2. Yet, the excellent correlations between the two systems for most orientations (r > 0.90), except along the vertical axis (r = 0.77), showed that the Inside-Out system of the OQ2 followed the general angular movements well. These results are consistent with the study of Guinet et al. (Citation2019), who used another Inside-out technology as part of the augmented reality Microsoft HoloLens (Redmont, USA) headset, although over shorter distances, and also found excellent correlation coefficients but large absolute mean error for position validity (r around 1.00 and absolute maximum error of 25 cm). A recent study looked at the accuracy of the OQ2 helmet position and orientation data with respect to the OQ2 controllers over shorter distances for upper limb function (Carnevale et al. Citation2022). The results showed that absolute mean error increased with distance between the HMD and controller, with errors over 10 cm when the HMD and controller were 50 cm apart. This is different than the present study focusing on absolute position and orientation data of the HMD alone but highlights the caution to be taken when using the Inside-Out technology.
In order to evaluate or train the locomotor skills of people with mobility limitations, the presence of such translation measurement errors could have negative consequences on the safety of the user if the task involves walking near built environment obstacles (e.g. walls, door) (Guinet et al. Citation2021). However, the scene is visually presented in a consistent manner within the HMD, with the relative position of the person to the scene unchanged and only the HMD’s measure of displacement within absolute space is variable. Therefore, the person’s perception of the environment and their reaction to it are unaffected which makes the use of the OQ2 viable to train locomotor ability and even qualitatively assess it as long as additional measures are taken to ensure safety with respect to proximity to built environment obstacles. Moreover, as found in a previous study (Guinet et al. Citation2021) for similar technology, it is possible to extract key variables (e.g. speed, step detection) using the affordable inside-out technology, making it interesting for clinical purposes.
This is the first study to measure the test-retest reliability of the Inside-out technology, here for the OQ2. The results showed that when recalibrated multiple times, differences from the first calibration values varied around one centimetre and one degree. Consequently, the OQ2 appears to have a good test-retest reliability for resetting its position to the same point within the environment. While this is important for research, it may be particularly important within a clinical context where the HMD may have to be removed from the patient's head and recalibrated more frequently due to fatigue or discomfort (Donegan et al. Citation2020).
It should be noted that the current platform was created in order to study, train and assess daily locomotor abilities within more complex environments. Therefore, the park included features such as fountains, trees, benches, etc., to recreate an ecological environment for the participants. The level of detail related to the number of polygons required to render some objects affected the sampling rate. In other applications where less polygons are needed, sampling performances and accuracy might be improved.
Finally, the results of the present study reflect the specific configuration of the space in which the data collection was conducted. As noted by Mulvany et al. (Citation2020), the effect of the physical environment is important for tracking accuracy. Yet, the current work targeted an uncontrolled and unprepared environment for the OQ2 which is more representative of a general environment found in a laboratory or clinical settings. However, placing a grid pattern on the floor or easily distinguishable markers on the walls and using a larger room might improve the accuracy of the OQ2. This may be more easily done in a laboratory. Within clinical settings, a controlled environment may be more difficult, but using a room with several contrasting elements (e.g. different objects hung on the walls; usual marking on the floor; good lightning) might improve the validity of the OQ2 (Hesch Citation2019).
5. Conclusion
The Oculus Quest 2 alone can evaluate the head position and orientation of the participant for studying locomotor navigation. However, the results showed that the validity of orientation measurements and specifically position of the OQ2 might drift in dynamic conditions. Yet, the OQ2 can be used for training and evaluation of locomotor navigation for rehabilitation with objects far enough to ensure patient safety.
Acknowledgement
We would like to thank Jonathan Caron-Roberge and Félix Fiset for their valued input for, respectively, programming and technical assistance. We would also like to thank Mulvany et al., for sharing their 3D printing plans for the plastic support used for the calibration frame.
Disclosure statement
No potential conflict of interest was reported by the authors
Additional information
Funding
Notes on contributors
Joris Boulo
Joris Boulo Trained in biomechanics, he is a PhD Candidate within the Rehabilitation Sciences program of the Faculty of Medicine at Université Laval.
Andréanne K. Blanchette
Andréanne K. Blanchette Physotherapist and Professor within the Department of Rehabilitation at Université Laval and a researcher at the Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris) where her research focuses on the development of innovative approaches in rehabilitation for individuals with physical limitations following a lesion of the central nervous system. She works on the use of different technologies, such as robotics, virtual reality, portable captors and functional electrical stimulation to evaluate the treatment of sensorimotor deficits. Her objective is to reach a better understanding of the underlying mechanisms of sensorimotor recovery.
Alexandra Cyr
Alexandra Cyr With a Master’s in physiotherapy from Univeristé Laval, she is a practicing physiotherapist.
Bradford J. McFadyen
Bradford J. McFadyen Professor within the Department of Rehabilitation at Université Laval and a researcher at the Centre for Interdisciplinary Research in Rehabilitation and Social Integration (Cirris) where he co-leads the Centre’s research platform for immersive technology in rehabilitation (TIR). He is also a Research Fellow of the Canadian Institute for Military and Veteran Health Research. Dr. McFadyen’s research program spans from basic to applied studies about how locomotion is adapted to the environment across adulthood and following acquired brain injury (ABI). This research program has integrated virtual reality technology in order to manipulate environmental characteristics and social contexts. Clinical applications are focused on adding to the rehabilitation toolbox to better expose mobility deficits and intervene following ABI of different severities.
References
- Amorim JSCD, Leite RC, Brizola R, Yonamine CY. 2019. Virtual reality therapy for rehabilitation of balance in the elderly: a systematic review and META-analysis. Adv Rheumatol. 58(1):18. doi:10.1186/s42358-018-0013-0.
- Carnevale A, Mannocchi I, Sassi MSH, Carli M, De Luca G, Longo UG, Denaro V, Schena E. 2022. Virtual reality for shoulder rehabilitation: accuracy evaluation of Oculus Quest 2. Sensors. 22(15):5511. Article 15. doi: 10.3390/s22155511.
- Donegan T, Ryan BE, Swidrak J, Sanchez-Vives MV. 2020. Immersive virtual reality for clinical pain: considerations for effective therapy. Front Virtual Real. 1:1. doi: 10.3389/frvir.2020.00009.
- Fleiss JL. 2011. Design and analysis of clinical experiments. New York, U.S.A.: John Wiley & Sons.
- Guinet AL, Bouyer G, Otmane S, Desailly E. 2019. Reliability of the head tracking measured by Microsoft hololens during different walking conditions. Comput Methods Biomech Biomed Engin. 22(sup1):S169‑S171. doi:10.1080/10255842.2020.1714228.
- Guinet A-L, Bouyer G, Otmane S, Desailly E. 2021. Validity of hololens augmented reality head mounted display for measuring gait parameters in healthy adults and children with cerebral palsy. Sensors. 21(8):2697. Article 8. doi: 10.3390/s21082697.
- Hesch J. 2019. Powered by AI : oculus insight. 22 août. https://ai.facebook.com/blog/powered-by-ai-oculus-insight/
- Holzwarth V, Gisler J, Hirt C, Kunz A. 2021. Comparing the accuracy and precision of SteamVR tracking 2.0 and Oculus Quest 2 in a room scale setup. In the 5th International Conference on Virtual and Augmented Reality Simulations, March, Melbourne, Australia: New York, NY, USA: ACM. doi:10.1145/3463914.3463921.
- Jahnsen R, Villien L, Egeland T, Stanghelle JK. 2004. Locomotion skills in adults with cerebral palsy. Clin Rehabil. 18(3):309‑316. doi:10.1191/0269215504cr735oa.
- Kim A, Schweighofer N, Finley JM. 2019. Locomotor skill acquisition in virtual reality shows sustained transfer to the real world. J Neuroeng Rehabil. 16(1):113. doi:10.1186/s12984-019-0584-y.
- Lee SJ, Hidler J. 2008. Biomechanics of overground vs. Treadmill walking in healthy individuals. J Appl Physiol. 104(3):747‑755. doi:10.1152/japplphysiol.01380.2006.
- Li Z, Han X-G, Sheng J, Ma S-J. 2016. Virtual reality for improving balance in patients after stroke: a systematic review and meta-analysis. Clin Rehabil. 30(5):432‑440. doi:10.1177/0269215515593611.
- Mousas C, Kao D, Koilias A, Rekabdar B. 2021. Evaluating virtual reality locomotion interfaces on collision avoidance task with a virtual character. Visu Comput: Int J Comput Graph. 37(9‑11):2823‑2839. doi:10.1007/s00371-021-02202-6.
- Mulvany G, Taylor S, Greuter S. 2020. Target: zero drift: minimizing spatial drift in VR optical tracking systems such as Oculus insight. Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play, p. 321–5. doi:10.1145/3383668.3419875.
- Needham M. 2022. mars 21. AR/VR headset shipments grew dramatically in 2021, thanks largely to meta’s strong Quest 2 volumes, with growth forecast to continue, according to IDC. International Data Corporation (IDC). https://www.idc.com/getdoc.jsp?containerId=prUS48969722.
- Porras DC, Siemonsma P, Inzelberg R, Zeilig G, Plotnik M. 2018. Advantages of virtual reality in the rehabilitation of balance and gait: systematic review. Neurology. 90(22):1017‑1025. doi:10.1212/WNL.0000000000005603.
- Ribo M, Pinz A, Fuhrmann AL. 2001. A new optical tracking system for virtual and augmented reality applications. IMTC 2001. Proceedings of the 18th IEEE Instrumentation and Measurement Technology Conference. Rediscovering Measurement in the Age of Informatics (Cat. No.01CH 37188), 3, 1932‑1936. doi:10.1109/IMTC.2001.929537.
- Row Lazzarini BS, Kataras TJ. 2016. Treadmill walking is not equivalent to overground walking for the study of walking smoothness and rhythmicity in older adults. Gait Posture. 46:42‑46. doi:10.1016/j.gaitpost.2016.02.012.