Heart rate variability (HRV) during virtual reality immersion

Abstract

The goal of the study was assessment of the hour-long training involving handling virtual environment (sVR) and watching a stereoscopic 3D movie on the mechanisms of autonomic heart rate (HR) regulation among the subjects who were not predisposed to motion sickness. In order to exclude predispositions to motion sickness, all the participants (n=19) underwent a Coriolis test. During an exposure to 3D and sVR the ECG signal was continuously recorded using the Holter method. For the twelve consecutive 5-min epochs of ECG signal, the analysis of heart rate variability (HRV) in time and frequency domains was conducted. After 30 min from the beginning of the training in handling the virtual workstation a significant increase in LF spectral power was noted. The values of the sympathovagal LF/HF index while sVR indicated a significant increase in sympathetic predominance in four time intervals, namely between the 5th and the 10th minute, between the 15th and the 20th minute, between the 35th and 40th minute and between the 55th and the 60th minute of exposure.

Keywords:

• heart rate variability
• motion sickness
• stereoscopic movie
• virtual reality

1. Introduction

Virtual reality (VR) techniques are used in everyday life – for entertainment purposes, e.g., computer games or stereoscopic (3D) movies and in working environments, mainly for training purposes. Trainings using VR techniques have recently become more and more popular.[1] Taking advantage of such techniques seems particularly useful when training in real conditions threatens the life and health of the participant. Therefore, VR training is most often conducted in such fields as atomic energy (e.g., due to a decreased exposure to ionizing radiation),[2] medicine (virtual surgical procedures) [3] and different types of simulators.[4,5]

Training effectiveness may depend on the level of immersion in virtual environment (VE). Ragan et al. [6] confirmed empirically that a higher level of immersion may significantly contribute to performance improvement during abstract mental activities. Assessment of the quality of the activities performed during training involving VR immersion may be inadequate to the participant potential due to subjective sensations connected with the inclination to symptoms mimicking motion sickness and defined as simulator sickness.[7–9] In literature on the topic, the most often reported symptoms of simulator sickness include: visual disorders (hallucinations, eye strain, blurred vision), dizziness and headaches, fatigue, pallor, sweating, gastrointestinal disorders (compression of the upper abdominal cavity, nausea, emesis, loss of appetite), apathy and limited mental concentration and muscular activity.[10–12]

Also Malińska et al. [13] in their study found that immersion in VE can evoke many symptoms indicating simulator sickness including: fatigue, eye pain, excessive sweating, dizziness or nausea. It was found that the symptoms of simulator sickness are felt more frequently and are more intense during immersion in VE using the virtual helmet with the effect increased by a properly designed virtual environment than while watching a stereoscopic (3D) movie.

Figure 1. The scheme presenting the causes of discomfort due to a simulated movement during immersion in virtual reality. The question mark stands for excitation or the lack of excitation, depending on the conditions of exposure to virtual reality (own study).

The subjective sensation of discomfort resulting from poor tolerance of VE may adversely affect the result of physiological measurements and training effectiveness; they can always be the source of additional stress for people undergoing simulator training.[14]

An adequate spatial orientation requires simultaneous visual, vestibular and sensory information provided to the brain. The exposure to VE results in the conflict between the patterns of sensations connected with real movement and the stimuli received by the visual organ (the image seen) which are fixed in the brain during immersion in VR. In English language literature, this phenomenon is called visual-vestibular conflict.[15] While watching a stereoscopic movie, signals from the vestibular organ and from the sensory receptors are not received as the body remains unmoved. Only the visual information about motion (e.g., change of direction, acceleration and position), based on the image using, inter alia, ‘illusions’ analogical to those used in painting (chiaroscuro perspective) is received [16] (Figure 1). A somehow different situation can be observed during operating a virtual workstation when whole body movements are performed, e.g., bending and turn or only head and limb movements.[4]

In clinical studies, one of the methods allowing noninvasive assessment of changes in autonomic regulation is the heart rate variability (HRV) method. In healthy individuals, this method enables assessment of the response of HR to various environmental stimuli including exposure to VE.

The aim of this study was assessment of the effect of an hour long immersion in VR on the mechanisms of autonomic HR regulation among the subjects who were not predisposed to motion sickness. The study was conducted under two different types of conditions, connected with the subjects’ job, involving training in virtual workstation handling (sVR) and connected with their everyday life, involving watching a stereoscopic (3D) movie.

2. Materials and methods

2.1. Subjects

The study was carried out in a sample of 19 young, healthy males aged 19–25 years (x=21.6, SD=2.2). Eighty percent, and 20% had mean and higher educational background respectively. It is believed that the symptoms of simulator sickness, evoked by the illusion of motion in VR are more frequent in individuals predisposed to motion sickness related to real motion; hence all the participants were subjected to Coriolis test verifying individual inclinations to motion sickness. The inclusion criteria were the lack of inclination to motion sickness and symptoms of cold, such as: rhinitis, headache or sore throat.

Three hours prior to and within 24 h following the study the participants were asked not to use any stimulants. Moreover, they were asked to eat a light meal (breakfast), sleep to assure adequate rest, no consumption of alcohol at least 12 h prior to the study and taking no medications 48 h earlier.

2.2. Experimental setup

It is believed that symptoms of simulator sickness, due to illusion of movement in VR, more frequently occur in subjects prone to motion sickness connected with real movement,[7,8] hence all the participants underwent a Coriolis test verifying individual predispositions to motion sickness; next, the subjects underwent the main test involving exposure to a stereoscopic (3D) movie and training in handling a virtual workstation (sVR).

2.2.1. Stereoscopic (3D) movie

The aim of watching the stereoscopic movie was to verify whether the participants not predisposed to motion sickness might develop symptoms of simulator sickness. A question was posed whether despite ‘incomplete’ immersion in a virtual environment and passive exposure (in a seated position, without performing any activity) it is possiple to predict simulator sickness symptoms during training using a sVR.

During the experiment the participants wore shutter glasses and watched an hour long part of ‘Avatar’ (directed by James Cameron, 2009), which was selected according to the images, particularly well presenting the stereoscopic effect.

2.2.2. Training in sVR

During the second stage the participants underwent training involving operator activities on the virtual workstation. There were 19 participants as one of them refused participation in the study due to health reasons. All of them performed the same simple manual task involving arrangement of the elements in a proper place to match them with the pattern. The elements were of cylindrical shape, their height was 25 cm and their diameter – 8.5 cm. On each cylinder two red lines were drawn. In the specified places on the table also two red lines were drawn and the lines drawn on the cylinders should be matched to them (Figure 2). The participants’ task was to perform the tasks precisely within the shortest time possible.

Figure 2. The workstation presenting a simulated work performed without real objects (left side) and with real objects (right side) (Photo: Grabowski A.)

The tests were preceded by 10 min long training to get the participants acquainted with the virtual environment and the apparatus used for the study.

The study was carried out in an air-conditioned room (temperature about 23 °C) at a constant time of the day (morning hours). The total time of the study was about 2.5 h. The tests were carried out in the presence of an examiner supervising the course of the study. The tests could be interrupted at any moment if a participant wished to interrupt them.

2.3. Apparatus

The room where the study was conducted was equipped with:

• JVC DLA-X3 projector enabling display of the stereoscopic image on the screen with 1920×1080 resolution and frequency of 120 Hz, which is the most often used image refresh frequency of equipment (most of TV sets have a higher refresh frequency) (JVC, Japan)

• AVTek projection screen, type 300B (AVTek, China)

• wireless shutter glasses with a infrared (IR), synchronized with the projector by means of an IR transmitter

• a pair of DgTech info gloves equipped with five flex sensors allowing record of information at the level of flexion in all the fingers (DGTech, Italy). To the Info-glove was added a sensor from the tracking system enabling a correct reconstruction of hand avatars in the virtual environment and interactions between a trainee and the virtual environment, particularly pressing buttons and moving objects. Both gloves were equipped with a wireless data transmission system mediated by a Bluetooth interface.

• HMD (Head Mounted Display) Sony HMZ-T1 displaying a stereoscopic image with the resolution of 1280×720 points and field of view of 45° (Sony, Japan). HMD construction beclouds the entire picture of the real world. This subject has an impression of a complete immersion in virtual environment.

2.4. ECG signal recording and analysis

The ECG signal was recorded using the Holter method, enabling a noninvasive record, not burdensome to the participant. Oxford apparatus was used for this purpose. HRV analysis was carried out using the Cardioscan system program allowing determination of HRV parameters from the sequence of consecutive 5-min epochs of ECG signal, meeting the requirements of the quality of HRV analysis, established by the European Society of Cardiology and the North American Society of Pacing and Electrophysiology.[17] For the analysis the sequences of RR intervals were selected, which were free from artefacts, ventricular and supraventricular excitations. For all the sequences of RR intervals for parameters describing heart rhythm were determined. For time analysis the mRR (ms) parameter was determined – the arithmetic mean of all sinus rhythm RR intervals. HRV in the frequency domain was described using spectral power values in the low frequency [LF (ms²)] band: 0.04–0.15 Hz and high frequency [HF (ms²)] band: 0.15–0.40 Hz and LF/HF sympathovagal balance index.

2.5. Statistical analysis

For the assessment of the differences in HRV parameter values while watching an hour long stereoscopic (3D) movie or during operating the virtual workstation (sVR) Friedman's Analysis of variance (ANOVA) test was used. For comparison of the values obtained at the same moments of exposure to 3D and sVR, the Wilcoxon signed rank test was used. This test was also applied to compare the values of individual parameters determined in different time intervals of the same study. Statistical significance level was set at p≤.05. Statistical analysis was conducted using STATISTICA 6.0 PL software.

3. Results

3.1. HRV analysis in time and frequency domain

Figures 3–6 present the mean values of HR and LF, HF and LF/HF parameters determined for 12 consecutive 5-min epochs of ECG signal while watching the stereoscopic (3D) movie and during training in operating the virtual workstation (sVR). Analysis of the results indicates statistically significant differences in HR values during consecutive 5-min intervals of exposure to sVR – p<.001 (χ²=56.49), compared with the exposure to the stereoscopic (3D) movie – p<.002 (χ²=30.183).

Comparing the HR values during exposure to 3D and sVR using the Wilcoxon signed rank test within the same 5-min time interval it was found that in all cases the difference was statistically significant at the p<.001 (Figure 3). During exposure to sVR, the HR values were higher than during immersion in VR while watching a stereoscopic 3D movie.

Figure 3. Heart rate (HR) – basic statistical indexes in consecutive 5-min intervals during watching the stereoscopic (3D) movie and operating the virtual workstation (sVR) at the level p<.001.

Figure 4 presents the values of basic statistical indexes for spectral power in the low frequency band, LF:0.04–0.15 Hz during consecutive 5-min time intervals while watching the stereoscopic (3D) movie and operating the virtual workstation (sVR). For greater precision, the results obtained during watching the 3D movie and sVR training were compared.

Figure 4. Spectral Power in the low frequency (LF):0.04­0.15 Hz during the exposure to the stereoscopic (3D) movie (left side) and the virtual workstation (sVR) (right side).

Friedman's ANOVA test did not show any statistically significant differences in LF spectral power during consecutive 5-min time intervals while watching the 3D movie 3D (χ²_{(N=19, df=11)}=6.083, p<.871) and a significant difference while operating the virtual workstation sVR (χ²_{(N=19, df=11)}=22.217, p<.023). While operating the virtual workstation sVR a significant increase in LF spectral power was noted compared with the values obtained within the first 5 min of exposure, between the 30th and the 35th minute (Z=2.133, p<.033), between the 35th and the 40th minute (Z=2.415, p<.016) and between the 40th and the 45th minute (Z=2.012, p<.044). During subsequent minutes, further yet insignificant increase in LF spectral power values was observed (Figure 4).

The comparison of spectral power values in high frequency (HF) band (0.15–0.4 Hz) in the same time intervals during both exposures – to 3D film and sVR, did not show any significant differences. Only the smaller dispersion of HF power values in all time intervals during the exposure is of note (Figure 5)

Figure 5. Spectral Power in the high frequency (HF):0.15–0.4 Hz, in consecutive 5-min intervals in exposure to the stereoscopic (3D) movie (left side) and the virtual workstation (sVR) (right side).

Comparison of the parasympathetic (HF) and sympathetic (LF) component indexes for the same 5-min time interval in exposure to stereoscopic movie and while operating the virtual workstation simulator (sVR) showed statistically significant differences between the 5th and the 10th minute (Z=2.093, p<.036), between the 15th and the 20th minute (Z=2.012, p<.044), between the 35th and the 40th minute (Z=2.133, p<.033) and between the 55th and the 60th minute of exposure (Z=2.374, p<.018) (Figure 6).

Figure 6. The sympathovagal balance index (LF/HF) during consecutive 5-min intervals in exposure to the stereoscopic (3D) movie (left side) and the virtual workstation (sVR) (right side).

4. Discussion

Many authors expressed their concern about the adverse effect of using stereoscopic technologies on ailments from the visual organ and occurrence of symptoms similar to those of motion sickness.[13,18–20] Conversely, so far there have been few available reports on studies concerning the effect of long term immersion in VR on HRV in employees undergoing training in reviewed scientific journals. The scientific reports concern mainly the physiological effects of VR immersion techniques in pilots undergoing training on flight simulators.[12] Discussion of the results is difficult, also due to the lack of data on similar experimental studies using similar workstation simulators and the same equipment or with the same time of exposure to VR.

In this study, the application of a simulated operational task was a complex procedure, combining the elements of mental and physical activities. In the second case, the subjects actively participated in task performance. The volunteers actively participated in the study. Their task was to place the elements on the table during an hour long training in virtual environment (monotype activity). The operator's activities were performed under time pressure, with utmost precision. Therefore, it is impossible to compare the exposure of watching a 3D movie, during which the subjects participated in the experiment passively, i.e., they did not perform any activities or additional mental tasks including stressogenic components.

Their task was to arrange elements precisely in the shortest time possible during immersion in virtual reality. Analysis of the results showed that within the same time intervals in exposure to sVR, the HR values were significantly higher than while watching the stereoscopic movie, which suggests that ‘active’ (sVR) immersion in virtual reality is a greater burden for the trainee than passive immersion (3D).

According to Bernston et al. [21], interpretation of changes in HR can help establish the relationship between psychophysical load and the type of a specific task. Muldoom et al. [22] and Owens et al. [23] in their studies found that performance of mental tasks, such as calculations, can affect HRV. A short exposure to stressors in laboratory studies result in an increased sympathetic activity of the nervous system or withdrawal of the parasympathetic activity of the nervous system, or both of the above-mentioned phenomena. In the study carried out by Van Eekelen et al. [24], the subjects performed a memory task combined with monitor display tracking and joystick manipulation. The observed changes in HR were attributed to parasympathetic modulation.

Many authors note that HR acceleration under various conditions may be due to a different response from the autonomic nervous system (ANS) involving co-activation and co-inhibition of the sympathetic and parasympathetic branches of the ANS.[24,25] The increase in HR values may result from parasympathetic activation exceeding sympathetic activation, sympathetic activation exceeding parasympathetic activation or when the inhibition of parasympathetic nerve activity is bigger than the sympathetic inhibition.[26]

This study revealed oscillations of LF/HF ratio values, significant in four time intervals, indicating sympathetic predominance in exposure to sVR. Also significant differences were found in LF spectral power with no changes in HF spectral power during the entire exposure to sVR. These results are different from those obtained by Van Eekelen et al. A significant increase in LF spectral power during the first 5 min of exposure to sVR was observed between the 30th and the 35th minute, between the 35th and the 40th minute and between the 40th and 45th minute of exposure to sVR and further yet insignificant increase during subsequent minutes of the trial can confirm other authors’ suggestion concerning the increase in sympathetic activity in HR regulation during mental task performance.

Similar results were obtained by Japanese researchers who conducted an experiment on a sample of 10 young healthy volunteers involving immersion in a virtual environment. Their results indicate an increase in sympathetic activity (LF) without any changes in parasympathetic activity of the ANS.[27] The observed response from the ANS in exposure to sVR can be modulated by a virtual environment which, in some individuals, evokes symptoms of simulator sickness accompanied by the changes in the sympathetic and parasympathetic activity, regulating HR.[16] The study sample comprised subjects with excluded predisposition to motion sickness which, in turn predisposes to discomfort during immersion in virtual reality. In the study conducted in the same sample, aimed at the assessment of fatigue level and subjective sensations after an hour long immersion in 3D sVR environments it was found that simulator sickness symptoms more often and with a higher intensity occur after exposure to sVR as compared with the exposure to 3D.[13]

Zuzewicz et al. [4] studied the effect of an hour long exposure to VE using a virtual pallet truck simulator on the mechanisms of autonomic HR regulation (HRV). The study did not reveal any statistically significant differences in the parameters describing HRV in subjects with no predisposition to motion sickness. Statistically significant differences were observed only in the group of subjects prone to motion sickness for mRR and LF parameters between the 30th and the 60th minute from the beginning of the study. In the reported study a statistically significant increase in LF power was noted between the 30th and the 35th minute, between the 35th and the 40th minute and between the 40th and the 45th minute of exposure, compared with the values obtained for the first 5 min of exposure in subjects with no confirmed predisposition to motion sickness.

The obtained results may be also affected by fatigue – weariness in the subjects.[28] After the exposure to sVR a large part of the subjects complained of pain in the hand used for task performance (local fatigue) and of fatigue due to regularly repeated activities in the shortest time possible with utmost precision.

Researchers from the National Institute of Advanced Industrial Science and Technology in Japan carried out a study to determine the effect of the presence or absence of visual signs indicating traffic direction on autonomic HR regulation. The study was conducted in a sample of 22 volunteers participating in a virtual driving licence course. The CAVE system with four monitors (front, floor and two sides) of dimensions 3×3 m. Based on the obtained results it was concluded that sympathetic activity was higher with no signs indicating traffic direction. A sufficiently long period between the first symptoms typical for motion sickness and acceleration maintains a stable activity of ANS.[29]

Due to the growing popularity of VR techniques, both in private and occupational life and due to the reports indicating discomfort during and after immersion in VR, there is an increased need to find out how VR affects the participant's body function with particular regard to HR regulatory mechanisms.

5. Conclusions

Based on the results of the experiment we can conclude that the following changes in heart rhythm regulation occurred in the participants not predisposed to motion sickness, undergoing training in handling the virtual workstation:

1. The heart rate (HR) values were significantly higher than those obtained during watching the stereoscopic movie.

2. After 30 min from the beginning of the training in handling the virtual workstation a significant increase in LF spectral power was noted.

3. The values of LF/HF sympathovagal balance indicated a significant increase in the prevalence of sympathetic predominince in autonomic heart rhythm regulation in four time compartments, namely between the 5th and the 10th minute, between the 15th and the 20th minute, between the 35th and the-40th minute and between the 55th and the 60th minute of exposure.

We can assume that the first 30 min of training in handling the virtual workstation did not result in any statistically significant changes in the sympathetic activity of the Autonomic Nervous System in subjects with confirmed lack of predispositions to motion sickness.

Acknowledgements

This project was based on the results of a research task carried out within the scope of the statutory activity of the Central Institute for Labour Protection – National Research Institute supported by the Ministry of Science and Higher Education.

Disclosure statement

No potential conflict of interest was reported by the authors.