No pain, no gain? The effects of adding a pain stimulus in virtual training for police officers

Abstract

Virtual training systems provide highly realistic training environments for police. This study assesses whether a pain stimulus can enhance the training responses and sense of the presence of these systems. Police officers (n = 219) were trained either with or without a pain stimulus in a 2D simulator (VirTra V-300) and a 3D virtual reality (VR) system. Two (training simulator) × 2 (pain stimulus) ANOVAs revealed a significant interaction effect for perceived stress (p = .010, η_p² = .039). Post-hoc pairwise comparisons showed that VR provokes significantly higher levels of perceived stress compared to VirTra when no pain stimulus is used (p = .009). With a pain stimulus, VirTra training provokes significantly higher levels of perceived stress compared to VirTra training without a pain stimulus (p < .001). Sense of presence was unaffected by the pain stimulus in both training systems. Our results indicate that VR training appears sufficiently realistic without adding a pain stimulus.

Practitioner summary: Virtual police training benefits from highly realistic training environments. This study found that adding a pain stimulus heightened perceived stress in a 2D simulator, whereas it influenced neither training responses nor sense of presence in a VR system. VR training appears sufficiently realistic without adding a pain stimulus.

Keywords:

• Police training
• virtual reality
• virtual environment
• simulation training
• pain stimulus

1. Introduction

Training technologies, such as virtual training simulators, are becoming increasingly popular in police agencies (Arble and Arnetz 2021). Virtual training simulators allow police agencies new opportunities to advance their training. Using a virtual training simulator, complex and high-risk scenarios can be trained in a safe and controllable environment. Vulnerable populations (such as children, the elderly, mentally ill people) that cannot easily be included in real-life training can be simulated in virtual environments (Kent and Hughes 2022; Murtinger et al. 2021). Scenario locations and content can be designed and adjusted almost at will (Giessing 2021). The increased safety and flexibility of using virtual simulators in training makes these technologies particularly interesting for police agencies.

A variety of virtual training simulators exist, from basic 2D screen, video-based simulators, to interactive 2D screen simulators, to advanced, interactive virtual reality (VR) systems (de Armas, Tori, and Netto 2020). Even in their most basic form, virtual simulators have been shown to effectively induce realistic responses. For instance, by projecting pre-recorded scenarios onto a wall using a laptop and projector, Groer et al. (2010) demonstrated that virtual simulators elicit acute psychophysiological stress responses in police officers. Moreover, interactive VR systems have been used to train advanced skills, such as firearm shooting in the military (Bhagat, Liou, and Chang 2016), laparoscopic surgery in medicine (Alaker, Wynn, and Arulampalam 2016), or table tennis skills in racquet sports (Michalski et al. 2019). Thus, virtual training simulators seem able to elicit similar psychophysiological responses as real-life training does and support the development of advanced (motor) skills. These two effects of virtual training simulators, eliciting psychophysiological responses and supporting skill development, are promising and desirable for police training, particularly in the context of realistic training.

Virtual simulators offer benefits, such as flexibility and safety in training; however, there are also limitation to the application of these technologies. Because the training simulations take place in a virtual environment, the interaction with that environment offers limited multi-sensory experiences (Giessing 2021). Even in advanced interactive VR systems, the interaction with the virtual environment is generally limited to audiovisual stimuli (Melo et al. 2020). Steps are being taken to include further sensory feedback, such as tactile stimuli (for instance, using a weapon or other equipment in the virtual environment or receiving haptic feedback from virtual obstacles) and olfactory stimuli (for instance, implementing smells, such as gasoline). Despite advancements in multisensory integration in virtual simulators (e.g. Marucci et al. 2021), there are theoretical and practical limitations that make it difficult to simulate sensory experiences in a virtual environment; for instance, the simulation of localised smells that only occur in a specific area of the virtual environment or the realistic interaction with simulated objects through haptic force feedback (Gallace et al. 2012; Scarfe and Glennerster 2019).

Police agencies aim to create training situations that closely resemble real-world experiences (i.e. realistic or representative training). Current training practices rely heavily on scenario-based training—a method of designing training scenarios that replicate on-duty encounters and exposing trainees to duty-like experiences in training (Kleygrewe et al. 2022). By doing so, trainees learn to perform skills, such as situational awareness, communication, and decision-making concurrently while also experiencing psychophysiological stress (Andersen et al. 2016; Di Nota and Huhta 2019). Training in a context that representatively reflects on-duty experiences has been shown to improve the performance of police officers on duty (Beinicke and Muff 2019).

When creating training contexts that resemble on-duty situations, similar constraints as those experienced on duty should be integrated into the training environment (Brunswik 1956; Davids et al. 2013). For example, Nieuwenhuys and Oudejans (2010) included an opponent who occasionally shot back at the police officers with colored-soap cartridges in their study design. Compared to a less representative situation in which the opponent was non-threatening and did not shoot back, the police officers who could get shot by the opponent showed shooting behaviour and psychophysiological responses that resembled on-duty experiences much more closely: the police officers experienced significantly higher heart rates, had higher anxiety scores, acted faster, made themselves smaller, and blinked more often (Nieuwenhuys and Oudejans 2010). These findings (among others, see Nieuwenhuys and Oudejans 2011; Renden et al. 2014) show that including representative constraints in a training setting, for instance, by adding an opponent who can physically threaten the police officer, induces on-duty-like responses and behaviours. Experiencing these responses and their influence on behaviour in training allows police officers to effectively prepare for duty (Di Nota and Huhta 2019). Thus, designing training situations that closely resemble on-duty experiences should be as much of a goal for virtual simulation training as it is for real-life practice.

In our current study, we investigate the addition of a pain stimulus to virtual training simulators in police training. We argue that adding a pain stimulus to the virtual simulators influences the representativeness of the virtual training environment. We expect that a more representative training context provokes stronger psychophysiological responses and would lead to an increased sense of ‘being there’ (i.e. sense of presence) in the virtual environment compared to a less-representative training context (North and North 2016). Hence, our first hypothesis states that the addition of a pain stimulus to virtual training simulators increases the physical and psychological training responses of police officers. Similar to the study by Nieuwenhuys and Oudejans (2011), the pain stimulus in our study simulates the shooting back of a threatening opponent. To determine whether the addition of a pain stimulus influences the training responses of police officers, we assess physical responses, such as heart rate, and psychological responses, such as perceived stress and mental effort during the training simulation in groups that train with and without a pain stimulus. Secondly, we hypothesise that by adding a pain stimulus to the virtual training simulator (i.e. adding the sensory modality of nociceptive stimulation), the sense of presence in the virtual environment is enhanced compared to a training simulation without the pain stimulus (Gallace et al. 2012). Sense of presence refers to the experience of being or feeling physically present in a simulated environment (North and North 2016). As such, higher levels of sense of presence contribute to realistic display of behaviour in a virtual simulation and thus provide more representativeness to training contexts (Slater 2009).

We investigated whether the addition of a pain stimulus influences the training responses and sense of the presence of police officers utilising two different types of virtual training simulators: an interactive 2D simulator consisting of five screens arranged in 300 degrees (VirTra V-300) and an advanced, interactive 3D VR training systems specifically designed for police (Refense). We selected a 2D and 3D training simulator to examine if any effects of the pain stimulus rely on the type of simulator used for training. Particularly as immersion, presence, and skill acquisition appear to differ between 2D and 3D simulations (Ashraf et al. 2015; Gąsiorek et al. 2019), we aimed to explore whether the effect of a pain stimulus would differ between simulators. By investigating the influence of a pain stimulus on the training responses and sense of presence in different types of virtual training simulators, we aim to support the development of virtual training for police practice.

2. Methods

2.1. Participants

This study was conducted in collaboration with the Stadtpolizei Zürich (City Police Zurich). In total, 219 police officers of the Stadtpolizei Zürich (180 male, 35 female, and 4 other; M age = 38.22, SD = 9.16) participated in this study. The participants’ experience on the job ranged from 2 to 37 years (M years = 12.57 years, SD = 8.98). 83 of the study participants (64 male, 17 female, and 2 other) provided additional information on their technology experience: 23 participants experienced VR in the form of gaming or as a commercial product before. 60 participants had never experienced VR before the training.

Participants provided informed consent before the start of the experiment. Ethical approval was obtained from the Social and Societal Ethics Committee of the Katholieke Universiteit Leuven as part of the SHOTPROS project which is funded by the European Union’s Horizon 2020 Research and Innovation Programme (Grant number: 833672).

2.2. Design

We utilised a 2 (training simulator; within-subjects) × 2 (pain stimulus; between-subjects) mixed study design. Participants completed Virtual Reality training (VR) and training with the VirTra V-300 (VirTra) simulator on the same day. Due to the availability of training locations, all participants completed the training in the same order: VR training followed by VirTra training. Half of the participants completed the VR and VirTra training with the pain stimulus, the other half trained in both training simulators without a pain stimulus. The experiment was conducted over a span of 8 weeks as part of the annual training days for the police officers of the Stadtpolizei Zürich; thus, we were able to test and assess the training simulators as they are applied in practice.

On each training day, training groups consisting of 8 police officers participated in the two training sessions. On each day of the experiment, the use of the pain stimulus alternated (i.e. on odd training days, participants trained in VR and VirTra with the pain stimulus; on even training days, participants trained in VR and VirTra without the pain stimulus).

The objective of the training was 3-fold: (i) training of tactical procedures and movement, (ii) training of de-escalation techniques, and (iii) training of communication skills. The scenarios for the VR and VirTra training were developed and selected based on these training objectives.

Due to the set-up of the training schedule of the Stadtpolizei Zürich, some participants were unable to complete all measures at the prescribed time points without causing a delay to the schedule. Thus, we proceed with our analysis with three distinct sub-samples: 83 participants completed the sense of presence measures for both simulators, 166 participants completed the measures of psychological responses for both simulators, and 87 participants took part in the measures of physical responses for both simulators.

The VirTra V-300 training system has been used by the Stadtpolizei Zürich before this research; therefore, most of the participants had previous experience with the VirTra training. For all participants, this was their first time performing operational police training in VR.

2.3. Experimental set-up

The independent variables of the study were training simulator (VR, VirTra) and pain stimulus (with, without). The set-up and training scenarios for the training in VR and VirTra, as well as the use of the pain stimulus for both training simulators are described below.

2.3.1. Virtual reality training (VR)

The VR system used in this experiment was provided by Refense (www.refense.com). Participants were equipped with the Refense VR suit consisting of a binocular head-mounted display, microphone and audio provided via over-ear headphones, radio chatter, hand- and foot sensors for motion tracking, a computing box (backpack style; weighing 5 kg), and a replica rifle. The size of the VR training area was 15 × 15 m. Figure 1 shows the VR equipment used in this study. The training group of eight participants was split into two smaller groups of four participants to train the VR scenarios in small units. On average, the completion of the VR scenario took 13 min. Before the training scenario began, participants underwent calibration of the VR sensors and equipment and completed a short instructional tutorial in VR. The instructional tutorial contained a replication of a specific training location of the Stadtpolizei Zürich that the participants are familiar with from their regular real-life training practices. When entering this environment in VR, participants are guided through specific tasks (e.g. walking a certain distance, opening and closing doors, interacting with others, and using their replica rifles on target boards) by the instructor. The familiarisation process took, on average, 5 min. After completing the tutorial, participants had a brief break (∼30–45 s) to prepare for calibration of the actual training scenarios.

Figure 1. Refense VR equipment. Note. The VR equipment was provided by Refense.

Four police officers lined up behind each other wearing Virtual Reality gear, with a Virtual Reality replica rifle carried by the police officer in the front.

The VR training scenario depicted a three-story building, shown in Figure 2. The first floor displayed the entrance of a bank, the second floor contained multiple small offices and work areas, and the third floor depicted a residential apartment. The scenario entailed non-player characters (NPCs) that were placed throughout the virtual environment in the form of bystanders, injured bystanders, bank tellers, and perpetrators. All encounters of participants were with NPCs. To make the encounters with the perpetrators as dynamic and interactive as possible, the voice of the perpetrator was taken over by an instructor. All other NPCs that participants interacted with in the scenario were steered by experienced VR instructors of the Stadtpolizei Zürich.

Figure 2. VR training environment. Note. The picture shows the layout of the first, second, and third floors of the VR training environment. Pictures were taken from an initial run-through of the VR environment during the construction phase of the training scenario.

(a) Bird’s eye view screenshot of the ground floor of the virutal training environment, depicting the entrance hall of a bank. (b) Bird’s eye view screenshot of the second floor of the virutal training environment, depicting an office floor with several small office rooms and a large open-floor work area. (c) Bird’s eye view screenshot of the third floor of the virutal training environment, depicting a residential apartment with a living room, a separate bathroom and bedroom, and an outside terrasse.

The participants started the scenario on a large square outside of the building. The square replicated a well-known location in Zurich (Paradeplatz) and was filled with police cars with flashing blue lights and car sirens. The participants received the dispatch information that two armed perpetrators entered the bank. The participants then entered the bank in which bystanders and bank tellers were hectically running around. The bank tellers told the participants that the perpetrators had moved to a higher floor. To avoid stairs in the virtual environment, the participants were told that the stairs were blocked and had to use the elevator to enter the offices on the second floor. Exiting the elevator, the participants found injured bystanders who alerted the officers to an armed perpetrator on the floor. The participants’ task was to secure the floor and stop the perpetrator from hurting further bystanders. When participants arrested or eliminated the perpetrator, they received the dispatch information that the second perpetrator entered the apartment on the third floor. Once participants entered the apartment through the elevator, they were tasked with identifying the perpetrator. The perpetrator was taking the resident of the apartment hostage on the balcony. The task of the participants was to verbally de-escalate the situation and arrest the perpetrator without causing harm to the hostage.

2.3.2. VirTra training (VirTra)

The VirTra V-300 (https://www.virtra.com/simulator/law-enforcement-v-300/) is a shooting simulator comprised of five 2D screens that are arranged at 300 degrees to create an immersive and interactive training platform (see Figure 3). Participants can use their service weapons with a drop-in laser recoil kit during the training scenarios. The training group of eight participants was split into four groups of two participants. Each pair of participants completed a sequence of three VirTra training scenarios which, on average, accumulated to 10 min of active training time in the VirTra V-300.

Figure 3. VirTra V-300 shooting simulator. Note. This image is the property of VirTra (www.virtra.com).

Two police officers standing back to back in the middle of the fice-screen, 300° virtual training simulator onto which a video is played that the officers respond to.

VirTra V-300 comprises a range of pre-recorded videos that contain a variety of branching options. These branching options allow for the development of the scenarios to be dynamic and interactive (i.e. a variety of responses and actions of the bystanders and suspects can be chosen by the instructor at various points of the scenario). The scenario was steered by experienced instructors to react as closely as possible to the actions of the participants (i.e. when the officer shoots at a suspect, the suspect appears injured and falls to the ground without delay; the bystanders or suspects verbally respond appropriately to the requests of the officers, etc.).

The instructors selected training scenarios from the VirTra scenario menu that aligned with the overall training objectives (i.e. tactical procedures and movement, de-escalation techniques, and communication skills) and closely resembled the training tasks of the VR training. In the first scenario, participants were placed in an urban environment in which they had to identify sudden threats in the 300-degree environment, communicate their surroundings to each other, and decide whether to engage in any of the threats or call for back up (e.g. when they saw an armed perpetrator through the window of a building). In the second scenario, the participants entered a school building in which an active shooter had to be identified and eliminated. In the third scenario, the participants faced an armed suspect who threatened to kill herself with a knife. The participants had to use tactical movement, communication, and de-escalation skills to resolve the situations safely. After the participants performed a scenario, they received short feedback on their performance before they proceeded with the next scenario.

2.3.3. Pain stimulus

All participants in the pain stimulus conditions wore a pain stimulus device during the VR and VirTra training, irrespective of whether a pain stimulus was delivered or not.

In VR, the pain stimulus was delivered using the StressX PRO belt from StressVest (www.stressvest.com). Participants wore the belt around the hip so that a safe, localised electrical stimulus via six electrodes on the device would be delivered to the lower abdomen. The device features an adjustable shock level (1 = low to 5 = extreme). For this experiment, the shock level setting 2 was used for all participants who trained with the pain stimulus.

In VirTra, the pain stimulus was delivered using the V-Threat-Fire (https://www.virtra.com/tool/law-enforcement-threat-fire/), an electric feedback device that is integrated into the VirTra V-300 simulator. The device was clipped onto the participant’s belt or placed in the back pocket of the participants’ pants (depending on the participant’s preferences).

In both conditions, the localised stimuli were delivered by the trainer in accordance with the development of the scenario; for instance, when a participant was shot by a perpetrator in the scenario, the trainer would deliver the pain stimulus immediately to simulate getting shot. The electric impulses elicited by both devices were strong enough that the participants momentarily recoiled. All participants were familiar with the use and sensation of a pain stimulus from previous trainings.

Participants were given the option to refrain from wearing a pain stimulus in the training and hence be excluded from the experiment. None of the participants refused to train with the pain stimulus.

2.4. Dependent variables

2.4.1. Physical training responses

2.4.1.1. Heart rate

Average and maximum heart rate (HR) in beats per minute (bpm) were recorded using a Zephyr Bioharness 3.0 device (www.zephyranywhere.com) at a recording frequency of 1 Hz. HR recordings of the active training time in the training scenarios for VR and RL were extracted and analysed. HR Confidence (degree of validity of HR value, as a %) provided in the Zephyr output was used for data correction: data points at or below 25% HR Confidence were considered invalid and were removed from analysis (see ‘Heart Rate Confindence’, OmniSense Analysis Help 2016). Bpm values outside of a realistic HR range (e.g. 0 bpm, values >220 bpm) were removed from analysis through inspection of the data.

The Zephyr Bioharness device provides valid and reliable HR measurements (Nazari et al. 2018) and is frequently used in police research (e.g. Andersen and Gustafsberg 2016; Bertilsson et al. 2020). In police research, HR has consistently been used as a common parameter to assess police officers’ cardiovascular response to stress (Andersen and Gustafsberg 2016; Andersen et al. 2016; Anderson et al. 2019; Anderson, Litzenberger, and Plecas 2002; Baldwin et al. 2019; Bertilsson et al. 2020; Vonk 2008).

2.4.2. Psychological training responses

To assess psychological training responses, we utilised visual analogue scales (VAS) to assess mental effort and perceived stress. In police research, VAS for mental effort and perceived stress (originally anxiety) have been frequently used to quantify self-perceived psychophysiological responses to training and complex police tasks (e.g. Giessing et al. 2019; Nieuwenhuys et al. 2009; Oudejans 2008; Wilson, Smith, and Holmes 2007).

2.4.2.1. Mental effort

Subjective ratings of mental effort were obtained using the VAS ‘Rating Scale for Mental Effort’ (RSME; Zijlstra 1993) at the end of the RL and VR training (once after all VR scenarios were completed). The RSME was assessed on a VAS from 1 to 150. According to Zijlstra (1993), the RSME has adequate test-retest reliability with correlation coefficients between 0.78 in work settings and 0.88 in laboratory settings.

2.4.2.2. Perceived stress

Subjective ratings of perceived stress were obtained using the VAS for anxiety (adjusted to ‘stress’ instead of anxiety; Houtman and Bakker 1989) at the end of the RL and VR training (after all VR scenarios were completed). The stress scale was assessed on a VAS from 1 to 100. The original anxiety scale has a fair validity and test-retest reliability with correlation coefficients ranging between 0.60 and 0.78 (Houtman and Bakker 1989).

2.4.3. Sense of presence

The experiences in the virtual environment were assessed using the ITC-Sense of Presence Inventory (ITC-SOPI; Lessiter et al. 2001). Each item of the inventory is rated on a Likert-scale from 1 ‘strongly disagree’ to 5 ‘strongly agree’. The ITC-SOPI results in four factors. Spatial presence refers to the sense of being part of the virtual environment. Engagement refers to the feeling of involvement with the content and feeling psychologically engaged. Ecological validity or naturalness refers to the tendency of perceiving the virtual environment as life-like and natural. Negative effects refer to the experience of any adverse physiological experiences, such as dizziness or headaches (Lessiter et al. 2001).

According to Lessiter et al. (2001), the ITC SOPI has good internal consistency. The Cronbach alpha coefficient for spatial presence was .94, for engagement .89, for ecological validity .76, and for negative effect .77. In the current study, the Cronbach alpha coefficient for the factor spatial presence was .89, for the factor engagement .80, for the factor ecological validity .74, and for the factor negative effect .83.

2.5. Procedure

Each experimental day started at the location of the Stadtpolizei Zürich. At the start of the experiment, participants received information about the training day, the training objectives, and general information about the experiment. Participants then provided written informed consent. Next, participants were taken to the VR training location. At the VR location, participants took off their police-specific gear (weapon, belt, vest), were equipped with a Zephyr heart rate monitor (for those training with the pain stimulus, they were also equipped with the pain belt), and then got fitted into the VR gear. The first four participants completed the VR training scenarios. After the VR training was completed, participants took off the VR gear (and pain belt, if training with pain stimulus) and received a short after-action review of their training performance (∼5 min). Because the after-action review was considered an integral part of the training session, participants filled in the visual analogue scales and the ITC-SOPI questionnaire using iPads right after the review was completed (instead of immediately after the scenario execution). As soon as the first group of four participants completed the training scenario, the second group of four participants completed the same sequence. Once the second group finished the training, the 5-min after-action review, and filled in the questionnaires, participants were taken back to the training location of the Stadtpolizei Zürich where the VirTra training simulator was located. The eight participants were split into pairs. The VirTra instructors took the first pair into the training simulator to complete the training. After the VirTra scenarios were completed, the participants received feedback on their performance while still in the system (∼5 min). When the first pair finished their VirTra training, the second pair started to train. In the meantime, the first pair returned their heart rate belts and filled in the visual analogue scales and the ITC-SOPI questionnaire using iPads at the VirTra training location. This sequence was completed until all eight participants had finished the VirTra training and filled in the questionnaires. (During their waiting time, participants were given the opportunity to train medical first aid skills at various pre-prepared stations). To ensure that participant groups could not observe each other perform during the training in VR and VirTra, the training areas were set up in such a way that the participant groups who waited to perform the trainings were in a preparation room that was separated from the execution areas.

2.6. Statistical analysis

2 (training simulator: VR, VirTra) × 2 (pain stimulus: with, without) ANOVAs were performed with training simulator as the within-subjects factor and pain stimulus as the between-subjects factor. The 2 × 2 ANOVAs were performed on average heart rate (BPM), maximum heart rate (BPM), mental effort scores, perceived stress scores, and scores for spatial presence, engagement, ecological validity, and negative effects. p-Values of <0.05 were considered statistically significant. Partial eta squared was calculated as an estimate for effect size. A value of η_p² = 0.01 indicated a small effect size, a value of η_p² = 0.06 indicated a medium effect size, and value of η_p² = 0.14 indicated a large effect size (Cohen 1969). All statistical analyses were performed using IBM SPSS, version 27.

3. Results

Mixed design ANOVAs were conducted to assess the impact of a pain stimulus on participants’ training responses (heart rate, mental effort, perceived stress) and sense of presence (spatial presence, engagement, ecological validity, negative effects) during the training with virtual training simulators (VR and VirTra). Descriptive statistics are presented in Table 1. For reasons of readability in this section, we only discuss statistically significant results and present full statistics of all measures in Table 2.

Table 1. Descriptive statistics.

	Variable	Training	With pain stimulus			Without pain stimulus
			n	M	SD	n	M	SD
Training effort	Average heart rate	VR	37	104.55	18.28	50	104.90	14.41
		VirTra	37	90.10	14.55	50	89.13	13.02
	Maximum heart rate	VR	37	129.54	20.46	50	128.54	16.54
		VirTra	37	121.16	17.79	50	115.40	16.27
	Mental effort	VR	80	71.04	23.23	88	70.22	29.02
		VirTra	80	74.17	25.22	88	64.13	27.45
	Perceived stress	VR	80	61.76	20.27	88	62.11	23.32
		VirTra	80	65.38	18.75	88	53.89	20.53
Sense of presence	Spatial presence	VR	39	3.41	0.59	44	3.69	0.46
		VirTra	39	3.07	0.72	44	3.11	0.65
	Engagement	VR	39	3.55	0.57	44	3.73	0.47
		VirTra	39	3.51	0.49	44	3.51	0.58
	Ecological validity	VR	39	3.44	0.61	44	3.70	0.53
		VirTra	39	3.48	0.64	44	3.40	0.71
	Negative effects	VR	39	1.79	0.61	44	1.88	0.68
		VirTra	39	1.48	0.54	44	1.50	0.47

Note. Average and maximum heart rate in BPM. Mental effort (RSME) on a visual analogue scale from 1 to 150. Perceived stress on a visual analogue scale from 1 to 100. Sense of presence measures on a Likert-scale from 1 to 5.

Table 2. Results of the 2 (training simulator: VR, VirTra) × 2 (pain stimulus; with, without) ANOVAs on training response measures and sense of presence measures.

	Training simulator	Pain stimulus	Training simulator × pain stimulus
Average heart rate	F(1,85) = 208.455, p < .001, ηp^2 = .710	F(1,85) = .010, p = .921, ηp^2 = .000	F(1,85) = .393, p = .532, ηp^2 = .005
Maximum heart rate	F(1,85) = 47.325, p < .001, ηp^2 = .358	F(1,85) = .939, p = .335, ηp^2 = .011	F(1,85) = 2.317, p = .132, ηp^2 = .027
Mental effort	F(1,166) = .313, p = .576, ηp^2 = .002	F(1,166) = 3.048, p = .083, ηp^2 = .018	F(1,166) = 3.059, p = .082, ηp^2 = .018
Perceived stress	F(1,166) = 1.037, p = .310, ηp^2 = .006	F(1,166) = 5.928, p = .016, ηp^2 = .034	F(1,166) = 6.825, p = .010, ηp^2 = .039
Spatial presence	F(1,81) = 24.825, p < .001, ηp^2 = .235	F(1,81) = 2.834, p = .096, ηp^2 = .034	F(1,81) = 1.652, p = .202, ηp^2 = .020
Engagement	F(1,81) = 3.186, p = .078, ηp^2 = .038	F(1,81) = 1.013, p = .317, ηp^2 = .012	F(1,81) = 1.553, p = .216, ηp^2 = .019
Ecological validity	F(1,81) = 1.748, p = .190, ηp^2 = .021	F(1,81) = .787, p = .378, ηp^2 = .010	F(1,81) = 3.085, p = .083, ηp^2 = .037
Negative effects	F(1,81) = 20.650, p < .001, ηp^2 = .203	F(1,81) = .261, p = .610, ηp^2 = .003	F(1,81) = .221, p = .639, ηp^2 = .003

Note. Significant results are shown in bold.

3.1. Training responses

The ANOVAs on training response measures showed a significant interaction between the training simulator and pain stimulus for perceived stress, F(1, 166) = 6.825, p = .010, η_p² = .039 (see Figure 4). Post-hoc pairwise comparisons with Bonferroni corrections showed a significant difference in training without the pain stimulus in VR and VirTra (p = .009), indicating that when no pain stimulus was used during the training, VR provoked significantly higher levels of perceived stress compared to VirTra (mean difference in perceived stress = 8.23). In addition, post-hoc pairwise comparisons with Bonferroni corrections revealed that there was a significant difference between training with or without the pain stimulus in VirTra (p < .001), indicating that VirTra training with a pain stimulus provokes significantly higher levels of perceived stress compared to VirTra training without a pain stimulus (mean difference in perceived stress = 11.49).

Figure 4. Disordinal interaction between training simulator and pain stimulus for perceived stress. Note. Perceived stress was assessed on a visual analogue scale ranging from 1 to 100.

A line graph plotting the levels of perceived stress of training without and with a pain stimulus in VR and VirTra. Perceived stress during VirTra training significantly increases from training without a pain stimulus to training with a pain stimulus. Perceived stress in VR training remains the same without and with a pain stimulus.

There were significant main effects for the training simulator on all physical training response measures, indicating that VR training elicited higher average heart rates and maximum heart rates compared to VirTra. Note that these differences were likely caused by the inherent differences between the training simulators (e.g. larger area for movement, the additional weight of the VR gear in VR training). The ANOVAs showed no other significant interaction effects between the training simulator and pain stimulus or main effects for pain stimulus (see Table 2 for statistics).

3.2. Sense of presence

The ANOVAs revealed significant main effects for the training simulator, indicating that spatial presence and negative effects were experienced significantly higher/more frequently in VR compared to VirTra. ANOVAs on the sense of presence measures showed no statistically significant interactions between the training simulator and pain stimulus. No statistically significant main effects were found for pain stimulus (see Table 2 for statistics).

4. Discussion

In the present study, we investigated the influence of a pain stimulus on the training responses and sense of presence in two different types of training simulators: the interactive 2D simulator VirTra V-300 and the 3D police-specific VR system from Refense. We expected that adding a pain stimulus to the virtual training simulator would have an effect on representativeness as it adds (i) a physical threat that is also present in real life to the training and (ii) a sensory stimulus to the audiovisual stimuli present in virtual training simulators. Thus, we first hypothesised that training responses with a pain stimulus would be higher than without a pain stimulus. Secondly, we hypothesised that the sense of presence with a pain stimulus would be higher than without a pain stimulus.

First, to address hypothesis 1, we found that the addition of a pain stimulus influenced the perceived stress of police officers in the VirTra training whereas it did not in VR training. Police officers who trained in the VirTra simulator with the pain stimulus experienced significantly higher perceived stress compared to those who did not train with a pain stimulus in VirTra. Additionally, we found that perceived stress was significantly higher in VR compared to VirTra when no pain stimulus was used. A main difference between the two virtual training simulators seems therefore to be the psychological training responses the simulators themselves provoke. While perceived stress and mental effort were relatively low in the VirTra training without a pain stimulus, they are comparatively high in VR without a pain stimulus and do not significantly increase when adding a pain stimulus. However, adding a pain stimulus to VirTra increases psychological training responses and matches the level of responses in VR (see Figure 4). A possible explanation for this effect may be that in VR, the psychological training responses are already at such a high level that adding a pain stimulus may not influence the perceived stress or mental effort any further. Compared to recent literature assessing perceived stress in police officers in high-threat conditions (see for instance, Bélanger and Blanchette 2022, perceived induced stress = 46; Nieuwenhuys, Savelsbergh, and Oudejans 2015, perceived stress in high threat condition = 65), the elicited levels of perceived stress were relatively high in our study even without the pain stimulus (VR = 62). This supports the notion that perceived stress was already at a level at which a pain stimulus may not have added anything further. One reason for this difference in perceived stress between VR and VirTra may be due to the different training configurations. In VR, police officers performed in groups of four using rifles, which is done when police officers respond to high-threat operational situations. Comparatively, in the VirTra training, police officers use their own service weapons and performed in pairs, which is similar to their on-duty experience. Thus, it may be possible that the set-up of the VR training induced higher levels of perceived stress while in VirTra the familiarity with the training context contributed to lower levels of perceived stress. Moreover, the addition of a pain stimulus did not influence any of the physical training responses assessed in our study. Nieuwenhuys and Oudejans (2011) demonstrated that, in a real-life setting, a threatening opponent who shot back at officers with colored-soap cartridges provoked higher heart rates, and higher anxiety and mental effort scores compared to an opponent who does not shoot back. Therefore, we anticipated to see heightened physical responses in response to a pain stimulus in virtual training as well. Similar to the explanation for psychological training responses, it may be that due to the already high training load (particularly in VR), we observe a potential ceiling effect. Another possible explanation may be that the pain stimulus was seldomly used in VR (8% of participants in the pain stimulus condition received a pain stimulus in response to getting hit by an opponent) compared to the frequent use in VirTra (64% of participants in the pain stimulus condition received a pain stimulus in response to getting hit by an opponent). Note however that the presence of a physical threat alone (with just an occasional pain stimulus, just as in VR in the current study) has been found to elicit higher levels of stress and anxiety in police officers in earlier studies (Nieuwenhuys, Cañal‐Bruland, and Oudejans 2012; Nieuwenhuys and Oudejans 2010). To conclude, hypothesis 1 is not fully supported by our data—the addition of pain stimulus to virtual training simulators influenced the training responses only minimally. Only in VirTra did the addition of a pain stimulus heighten the perceived stress during the training.

Addressing the second hypothesis, we found that sense of presence measures seemed to be unaffected by the addition of a pain stimulus. Particularly regarding ecological validity (one of the four sense of presence factors assessed in our study) we hypothesised that the addition of a pain stimulus would increase the perception of the naturalness of the virtual environment. However, our findings showed that neither in VR (low-frequency use of the pain stimulus) nor in VirTra (high-frequency use of the pain stimulus) the spatial presence, engagement, or ecological validity were affected by the addition of a pain stimulus. One reason why the addition of a pain stimulus may not have influenced the sense of presence is that a nociceptive stimulus provides information about the body itself (i.e. interoceptive function) rather than information about the environment, as auditory and visual stimuli do (i.e. exteroceptive function; Craig 2002; Ogden et al. 2015). Thus, while the virtual environment is primarily experienced through exteroceptive senses like auditory and visual experiences, adding a stimulus like pain that is experienced interoceptively may not influence the sense of presence (i.e. the feeling of fully being there in the virtual environment) as much as exteroceptive stimuli would. Taken together, a pain stimulus may not be the most relevant sensory stimulus to add to the virtual environment to enhance the sense of presence, as it provides limited information about the environment itself. It should, however, be considered that the possibility of getting hit by an opponent appears to influence the behavioural responses of police officers, such as adjusting their posture to make themselves smaller to avoid getting hit (Nieuwenhuys and Oudejans 2010). Therefore, while the pain stimulus may not have influenced the feeling of ‘being there’ in the virtual environment, it may have influenced how police officers reacted to the virtual environment on a behavioural level. These behavioural responses may also give an indication of how present someone feels in the virtual environment because realistic behavioural responses are more likely to occur when presence is high (Slater 2009). Whilst not explored in this study, the addition of a pain stimulus to virtual training simulators may support eliciting duty-like behavioural responses. Future studies in VR should therefore explore the influence of a pain stimulus on behavioural responses and in conjunction with presence. To conclude, hypothesis 2 is not supported by our data—the addition of pain stimulus to virtual training simulators did not influence the sense of presence in the virtual environment. This may also be due to a ceiling effect as the sense of presence (spatial presence, engagement, ecological validity) was relatively high in both simulators.

In addition to investigating the influence of a pain stimulus on the training responses and sense of presence in virtual training simulators, our analysis also revealed differences in training responses between the two training simulators. When looking at the physical training responses between the simulators, VR provoked significantly higher average HR and max HR compared to VirTra. This difference is likely explained by the inherent lack of similarity of the training simulators: VR training offers the greater possibility for movement as trainees have more space to move about. Additionally, in VR, trainees wear VR gear that adds an increased physical load, thus, increasing cardiovascular responses to the VR training compared to the VirTra in which trainees do not have any additional gear. Further, differences in sense of presence between the simulators exist as well: police officers experienced more spatial presence in VR, while also experiencing more negative effects compared to VirTra. These findings indicate that the training environment of a VR training simulator provides a realistic and immersive training environment, yet the occurrence of cybersickness also has to be considered, particularly compared to a 2D simulator where these appear much less (Naqvi et al. 2013). Although differences between the two systems exist in terms of physical training responses and sense of presence, both virtual training simulators provoked psychological training responses similar to responses found in studies of real-life training (VR provoking higher levels without a pain stimulus; addition of a pain stimulus to VirTra could match the VR levels; Nieuwenhuys, Savelsbergh, and Oudejans 2015; Oudejans and Pijpers 2009). Conclusively, both systems, due to their inherent technological differences, elicit different training responses and sense of presence experiences. A virtual training simulator may therefore be selected based on the training goal. For instance, for trainings that aim to elicit high levels of physical training responses and require a high level of immersion, a VR system appears most fitting. For a training that requires little movement, high levels of psychological training responses, and little negative effects, a VirTra simulator (with a pain stimulus) may better support the training goal.

The current study has limitations. First, the factor of pain stimulus was a between-subjects factor, with one group of participants doing the scenarios without and one group with a pain stimulus. While we assigned participants to the pain stimulus condition randomly, a stronger design for such a factor would be to have without and with pain stimulus as a within-subject factor, implying an intra- rather than inter-person comparison of the experience of pain. Statistically, a between-subjects factor involves relatively larger standard deviations, making it more difficult to find significant differences. Second, because the training schedule of the police was arranged in such a way that participants first completed VR training followed by VirTra training, it is possible that an order effect may have influenced the results. Lastly, the design of the training scenarios in VR and VirTra training did not allow us to balance the number of hits that participants experienced during the trainings with the pain stimulus. Thus, our results provide limited insights into the influence of applying a pain stimulus in VR training (as only 8% of participants received a pain stimulus in response to getting hit by an opponent).

In conclusion, the use of a pain stimulus may have a place in virtual simulation training, particularly in situations where the experience of a pain stimulus is likely and suitable and when virtual simulation training without a pain stimulus elicits relatively low levels of stress. For virtual simulation training that already elicits representative levels of stress, the addition of a pain stimulus seems less efficacious, at least in terms of increasing stress, mental effort, and presence. In real-life training, the inclusion of a physical threat to an opponent appears to influence the behavioural responses of police officers like adjusting the posture to avoid getting hit, reacting faster, and blinking more often (Nieuwenhuys and Oudejans 2010). These behaviours are likely to occur during on-duty encounters with a threatening opponent. Therefore, it is relevant to explore whether the addition of a pain stimulus in virtual training simulators elicits similar behavioural responses; further clarifying the need or redundancy of a pain stimulus in virtual training simulators.

Acknowledgements

The authors would like to thank the Stadtpolizei Zürich for their collaboration in this study. In particular, we would like to thank Christoph Altmann for the organization and the police officers and instructors for participating in this study. We would like to thank Refense; particularly, Ronny Tobler for taking part in this project.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 833672. The content reflects only authors’ view. Research Executive Agency and European Commission are not liable for any use that may be made of the information contained herein.