Changing perspectives: enhancing learning efficacy with the after-action review in virtual reality training for police

Abstract

The After-Action Review (AAR) in Virtual Reality (VR) training for police provides new opportunities to enhance learning. We investigated whether perspectives (bird’s eye & police officer, bird’s eye & suspect, bird’s eye) and line of fire displayed in the AAR impacted the officers’ learning efficacy. A 3 x 2 ANOVA revealed a significant main effect of AAR perspectives. Post hoc pairwise comparisons showed that using a bird’s eye view in combination with the suspect perspective elicits significantly greater learning efficacy compared to using a bird’s eye view alone. Using the line of fire feature did not influence learning efficacy. Our findings show that the use of the suspect perspective during the AAR in VR training can support the learning efficacy of police officers.

Practitioner summary: VR systems possess After-Action Review tools that provide objective performance feedback. This study found that reviewing a VR police training scenario from the bird’s eye view in combination with the suspect perspective enhanced police officers’ learning efficacy. Designing and applying the After-Action Review effectively can improve learning efficacy in VR.

Keywords:

• Police training
• virtual reality
• after-action review
• learning efficacy

1. Introduction

Virtual Reality (VR) is becoming an increasingly popular training tool for police agencies (Uhl et al. 2022; Zechner et al. 2023). Utilising high fidelity VR tools, police instructors are able to simulate various complex situations (e.g. Fejdyś et al. 2022). They can do so by utilising a wide variety of locations, select a range of avatars to appear the virtual environment, and include different sounds to create realistic training scenarios (Kleygrewe, Hutter, Koedijk, et al. 2023; Zechner et al. 2023). VR therefore provides police agencies the opportunity to train their officers for complex tasks in a safe and versatile environment (Giessing 2021). In the field of policing, VR has been used to teach police officers a variety of skills. For instance, VR training was conducted to investigate police officers’ helping behaviour towards victims of police racial aggression (Kishore et al. 2019). Police officers trained firearms skills in VR under light, moderate and frustrating difficulties (Muñoz et al. 2020) and were taught breathing regulation techniques in stressful situations (Brammer et al. 2021). Thus, VR appears to be a versatile training tool in which police officers can train and perform relevant police-specific tasks while being exposed to stress-inducing environments (Caserman et al. 2022; Garcia, Ware, and Baker 2019).

VR offers the opportunity to provide objective, technology-enhanced feedback on training performance (Cosman et al. 2002). By recording the virtual training scenarios, VR systems enable instructors and trainees to review the training using the VR after-action review (AAR). AAR is a common debrief procedure in military and other fields that rely on performance reviews to enhance learning (Raemer et al. 2011). While originally designed as a debrief tool for real-life combat training exercises, the AAR has been adapted to simulation-based environments in different fields (e.g. health care education, Sawyer and Deering 2013). In a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis of VR for police training, Giessing (2021) points out that the VR AAR provides police instructors and trainees with unique feedback capabilities to foster the learning experience of virtual training sessions. For instance, technical features such as perspective taking may foster learning by providing concrete visual performance feedback and thus make the debrief with the VR AAR less abstract than a verbal debrief after real-life training. Similar prospects of the AAR to enhance learning through the use of objective performance feedback have been discussed in a SWOT analysis of CBRNe training (i.e. training to prepare for Chemical, Biological, Radiological, Nuclear, and explosives incidents) in virtual environments (Murtinger et al. 2021). Whilst the potential of the VR AAR to enhance learning has been noticed, empirical research on the influence of feedback through simulation-based debriefing in police training is sparse.

In practice, police instructors can use the VR AAR tool to share training experiences on an external screen where instructors and trainees can review the training performance. The VR AAR offers the opportunity to replay the recorded training scenarios from various perspectives, for instance, from the birds’ eye view, from the view of the suspect, officer, or a bystander. During the review, the instructor and trainees can jump to specific incidents of the scenario and replay the scene using a variety of feedback features. Instructors can select to show the line of fire of the weapon, the viewing field of the trainees and various avatars, and the snake lines of the walking routes that trainees took during the VR scenario. In addition, the AAR may provide key performance indicators (e.g. shots fired, shots missed, target hits, number of bystanders flagged) that provide objective information on the training performance. Having a large range of options, police instructors can deliver visually supported and objective feedback using the AAR of the VR system and combine it with their personal expertise and the self-assessment of trainees.

Providing feedback effectively enhances learning (Hattie and Timperley 2007). Following the ecological dynamics framework, learning refers to the emergence of an adaptive, functional relationship between a trainee and their environment (Araújo and Davids 2011). In the context of police work, learning ensues that police officers can transfer knowledge and skills acquired in training to novel operational situations (Di Nota and Huhta 2019). Learning is therefore ideally assessed as performance of the taught knowledge and skills in a relevant on-duty context (Staller and Koerner 2022). Whilst the present study design did not allow for the assessment of learning outcomes in on-duty situations, this study examined the police officers’ learning efficacy. Learning efficacy refers to the trainees’ level of confidence in the application of their acquired knowledge and skills in real-life situations (Srivastava, Babu, and Shetye 2019). Learning efficacy has been shown to enhance engagement and motivation (Linnenbrink and Pintrich 2003), and appears to be correlated with higher levels of performance (Klobas, Renzi, and Nigrelli 2007; Schunk 1996). We assessed learning efficacy with a self-designed questionnaire aimed to capture the officers’ efficacy of applying the skills practiced during the virtual training to relevant on-duty situations.

In the current study, we investigate whether features of the VR AAR enhance the learning efficacy in police officers. In particular, we examined a) whether review perspectives ([i] combination of bird’s eye view and suspect perspective, [ii] combination of bird’s eye view and officer perspective, [iii] bird’s eye view) influence learning efficacy and b) whether using the line of fire of the weapon during the AAR influences learning efficacy.

• Our first hypothesis states that reviewing the VR training scenario from the suspect perspective elicits the highest learning efficacy in police officers.

Reviewing one’s performance from a perspective where one can see themselves act improves learning (Fukkink, Trienekens, and Kramer 2011; Guadagnoli, Holcomb, and Davis 2002). Because police officers are able to review their performance from the perspective of the suspect, they are able to see how effective their performance and behaviours are from the perspective of the person they engage. They seldomly have access to the suspect perspective—as compared to their own perspective, from which they view, albeit normally not review, their actions on duty and in training all the time—, and therefore we expect most ‘added value’ to learning from the suspect perspective. To investigate whether reviewing one’s performance from the suspect or officer perspective during engagements with suspects influences the learning efficacy, we used the bird’s eye view only throughout the entire training scenario as a control condition.

• Our second hypothesis states that using the line of fire of the weapon during the AAR review enhances learning efficacy compared to not using the line of fire.

The line of fire provides important performance feedback on the handling of the service weapon and thus informs police officers on how well they used their service weapon. For instance, reviewing the VR scenario with the line of fire, officers can see if or when they flag (i.e. point their weapon at something they should not) a bystander or colleague. In addition, particularly when performing in a team, police officers can see how well they cover each other with their line of fire when they make contact with a suspect. This objective feedback on flagging is not routinely available to officers; thus, having this information available during the AAR is expected to be of added value for police officers.

Investigating the effectiveness of the various AAR features on the learning efficacy of police officers may improve the development and implementation of VR training for police agencies. VR providers can utilise and implement perspectives and features that enhance learning efficacy in their technical developments and provide VR training tools that benefit learning. Moreover, knowing which perspectives and features provide the highest learning efficacy, police agencies can utilise these in their feedback procedure and provide targeted performance feedback. Therefore, this research aims to enhance development and delivery of virtual training in the police sector.

2. Methods

2.1. Participants

413 police officers of the City Police Zurich, Switzerland (Stadtpolizei Zürich) (342 male, 66 female, and 5 other; M age = 37.54, SD = 8.96) participated in this study. The participants’ experience on the job ranged from 2 to 37 years (M years = 11.50 years, SD = 8.50). Participants provided informed consent before the start of the experiment. Ethical approval was obtained from the Social and Societal Ethics Committee of the Katholieke Universiteit Leuven as part of the SHOTPROS project which is funded by the European Union’s Horizon 2020 Research and Innovation Programme (Grant number: 833672).

2.2. Design

We utilised a 3 (AAR perspectives: bird’s eye & police officer, bird’s eye & suspect, bird’s eye) x 2 (AAR line of fire: Off, On) between-subjects study design. Each participant completed the VR training and received the AAR from one of the three perspectives with the line of fire of their weapon either turned on or off. The experiment was conducted over a span of 7 weeks as part of the yearly training of the City Police Zurich. The training coordinator scheduled for police officers to attend the training on a particular training day in advance based on the officers’ availability. Eight instructors were selected for the experiment and rotated on a daily basis (two instructors were present daily, the pairings of instructors were also rotated).

The AAR perspectives changed biweekly (e.g. Week 2 & 3 = bird’s eye & police officer, Week 4 & 5 = bird’s eye & suspect, Week 6 & 7 = bird’s eye only). During Week 1, police instructors involved in this study received special training to familiarise themselves with the steering of the VR scenario and of the AAR and its features (participants of Week 1 were not included in this study). The AAR line of fire alternated on a daily basis: participants of the first training day trained with the line of fire off, participants on the second training day trained with the line of fire turned on, etc. In this way, the participants were randomly assigned to the experimental groups.

2.2.1. Virtual reality training (VR)

The VR system used in this experiment was provided by Refense (www.refense.com). Participants were equipped with the Refense VR suit consisting of a binocular head-mounted display, microphone and audio provided via over-ear headphones, radio chatter, hand- and foot sensors for motion tracking, a computing box (backpack style), and a replica rifle. The size of the VR training area was 15 × 15 metres. Figure 1 shows the VR equipment used in this study. Participants completed the VR scenario in groups of four. The completion of the VR scenario took on average 13 minutes. Before the start of the training scenario, participants underwent calibration of the VR sensors and equipment and completed a short instructional tutorial in VR. The same VR set-up and training scenario was used in a previous study by Kleygrewe, Hutter, and Oudejans (2023).

Figure 1. Refense VR equipment. Note. The VR equipment was provided by Refense.

The objective of the VR training consisted of the training of tactical procedures and movements, training of de-escalation techniques, and training of communication skills. The VR scenario contained three different layers in which participants trained these skills. The scenario was spread over a three-story building that contained a bank on the first floor, an office area on the second floor, and a residential apartment on the third floor. Participants were tasked with identifying and arresting two armed suspects that were located on the second and third floor. The suspects threatened bystanders or themselves. All encounters in the virtual environment were with non-player characters (NPCs). These NPCs were steered by experienced VR instructors of the City Police Zurich via an external control station consisting of a computer, an external screen, and a specialised keyboard to control the behaviours of the NPCs. From this control station positioned right next to the VR training pitch, the instructors could steer the virtual scenario and observe the participants on the physical training ground. During encounters with an NPC suspect, the instructors took over the voice of the NPC to create dynamic interactions. To keep the interactions with NPCs as standardised as possible, the VR instructors were provided with a script. During the dynamic voice-over interactions, the VR instructors could slightly deviate from the script if the interaction with the police officers necessitated it. For more information on the scenario, we refer readers to Kleygrewe, Hutter, and Oudejans (2023).

2.3. Independent variable

2.3.1. AAR perspective

The AAR perspectives used after the VR training consisted of (i) bird’s eye view in combination with police officer perspective, (ii) bird’s eye view in combination with suspect perspective, and (iii) bird’s eye view (as control condition). For perspectives (i) and (ii), the instructors were tasked with switching from a bird’s eye view, adopted during general movement through the virtual environment, to the specific perspective (i.e. police officer or suspect) whenever the participants made contact with a suspect. For perspective (i) this meant that the instructor selected the most relevant police officer perspective from the four team members during the engagement with the suspect (most often, instructors selected the officer who engaged in verbal communication). Perspective (iii), the bird’s eye view, was utilised as a control condition: participants reviewed their performance from the bird’s eye view for both engagement with the suspect and general movement through the environment. Figure 2 shows the different perspectives used in this study.

Figure 2. AAR perspectives: police officer perspective (top), suspect perspective (middle), bird’s eye view (bottom).

2.3.2. Line of fire

The AAR line of fire of the weapon the participants carried during the VR training was turned off or turned on during the entire AAR. The line of fire provided participants with information about the positioning and pointing of their own and their team member’s weapons. Because participants were equipped with a rifle-type weapon, participants were tasked with handling their weapon intentionally throughout the scenario (i.e. pointing the weapon only in high-threat situations). The weapons were never holstered (as might be the case with service pistols), therefore, the line of fire of the weapon is visible throughout the entire scenario, also when the weapon was safely pointing to the ground.

2.4. Dependent variable

2.4.1. Learning efficacy

We assessed learning efficacy using a self-developed questionnaire specific for police training in virtual training systems. The questionnaire contains three items: (i) ‘how confident are you that you can put into practice what you have learned in this training?’ (ii) ‘if one of the situations trained with this system occurs on-duty, I will be better able to master it’, (iii) ‘thanks to the training in the virtual system, I will be able to handle demanding operational situations more safely in the future’. These items were selected in accordance with the definition of learning efficacy—the participants’ level of confidence in the application of their acquired knowledge and skills in real-life situations (Srivastava, Babu, and Shetye 2019). Participants assessed these items on a 5-point Likert-type scale where 1 = extremely uncertain/strongly disagree and 5 = extremely certain/strongly agree. The average score of the three items was used for data analysis.

To assess the internal consistency reliability of the scale, we computed McDonald (1999) omega using RStudio 2022.07.02 which returned an ω value of .75, implying good internal consistency based on the minimum standard of reliability .70 (Nunnally and Bernstein 1994).

2.5. Procedure

Each data collection day started at the location of the City Police Zurich. Participants were scheduled to attend on specific training days by the training coordinator of the City Police Zurich in advance. At the start of the experiment, participants received information about the training day, the training objectives, and general information about the experiment. Participants then provided written informed consent. Next, participants were taken to the VR training location. At the VR location, participants took off their police-specific gear (weapon, belt, vest) and got fitted into the VR gear. Teams of four participants completed the VR training scenario. After the VR training was completed, participants took off the VR gear and received the AAR of their training performance in their teams of four. The police instructor who steered the training scenario provided the AAR. Instructors were equipped with points of references for the AAR feedback. The points of reference related to the three training objectives of tactical procedures and movements, training of de-escalation techniques, and training of communication skills Once completed, participants filled in the learning efficacy questionnaire using iPads.

2.6. Statistical analysis

A two-way ANOVA was conducted on the influence of two VR AAR features (AAR perspective, AAR line of fire) on learning efficacy. AAR perspective included three levels (bird’s eye & police officer, bird’s eye & suspect, bird’s eye) and AAR line of fire consisted of two levels (line of fire off, on). Where appropriate, post hoc pairwise comparisons were conducted with Bonferroni-adjusted p-values and 95% confidence intervals. P-values < 0.05 were considered statistically significant. Partial eta squared was ${\eta }_p^2$ calculated as an estimate for effect size. A value of ${\eta }_p^2$= 0.01 indicated a small effect size, a value of ${\eta }_p^2$= 0.06 indicated a medium effect size and value of ${\eta }_p^2$ = 0.14 indicated a large effect size (Cohen 1969). The statistical analysis was performed using IBM SPSS, version 27.

3. Results

Descriptive statistics of learning efficacy of all groups are reported in Table 1.

Table 1. Descriptive statistics of learning efficacy of the different groups.

AAR Perspective	Line of Fire	Mean	SD	N
Bird’s Eye View & Police Officer	Off	3.98	0.52	83
	On	4.01	0.50	59
Bird’s Eye View & Suspect	Off	4.03	0.47	91
	On	4.05	0.51	60
Bird’s Eye View	Off	3.89	0.56	46
	On	3.82	0.57	74

The two-way ANOVA revealed a statistically significant main effect of AAR perspective on learning efficacy, F(2, 407) = 4.284, p = .014, ${\eta }_p^2$= .021. Pairwise comparisons on AAR perspective revealed a statistically significant difference between bird’s eye view & suspect perspective and bird’s eye view alone, p = .013, 95% CI [.030, .343], mean difference on learning efficacy = .186. There was no statistically significant difference between bird’s eye view & police officer perspective and bird’s eye view & suspect perspective, p = 1.000, 95% CI [-.197, .100], mean difference on learning efficacy = −.049. Similarly, the difference between bird’s eye view & police officer perspective and bird’s eye view alone did not reach significance, p = .112, 95% CI [-.021, .296], mean difference on learning efficacy = .137.

There was no statistically significant main effect of line of fire on learning efficacy, F(1, 407) = .026, p = .871, ${\eta }_p^2$ = .000, nor a significant interaction between AAR perspective and AAR line of fire, F(2, 407) = 0.387, p = .679, ${\eta }_p^2$= .002. For a graphical output, see Figure 3.

Figure 3. Learning efficacy according to AAR perspective and line of fire feature.

4. Discussion

In the present study, we investigated the influence of AAR features in VR on learning efficacy of police officers. We defined learning efficacy as the trainees’ level of confidence in the application of their acquired knowledge and skills in real-life situations (Srivastava, Babu, and Shetye 2019). To examine the influence of AAR features on learning efficacy, we investigated two hypotheses: first, we examined whether using the AAR perspective from a suspect view enhances learning efficacy more than other perspectives such as a review from the police officer perspective or from the bird’s eye view only. Second, we hypothesised that turning on the line of fire of the weapon of the police officers during the AAR would enhance learning efficacy compared to not using this feature during the AAR. Both hypotheses rely on the premise that they provide police officers with relevant information about their behaviours and performance during the VR training scenario that is not routinely available to them on duty or in normal practice and would thus enhance learning efficacy.

Addressing the first hypothesis, we found that the learning efficacy of police officers was significantly greater when using the combination of bird’s eye view and suspect perspective compared to bird’s eye view alone. We therefore infer that viewing one’s performance from the suspect perspective—a perspective that police officers seldomly have access to—provides officers with new information about their performance and behaviour and thus enhances learning efficacy. In addition, we found that learning efficacy of police officers did not significantly differ when they reviewed their performance using the combination of bird’s eye view and police officer perspective and the combination of bird’s eye view and suspect perspective, implying that both distinctive perspectives appear to provide officers with information that supports their learning efficacy. Note, however, that the difference in learning efficacy of officers receiving the bird’s eye view and officer perspective and bird’s eye view alone did not reach significance, presumably due to high standard deviations. Nonetheless, we argue that reviewing performance from the suspect perspective and possibly the officer perspective appears to provide more detailed information on individual behaviours than bird’s eye view alone. In the bird’s eye view, police officers can obtain movement-specific information on a group level (which may provide valuable feedback on the training objective of tactical movements). Using distinctive perspectives (i.e. suspect and possibly officer) provides more detailed information related to the other training objectives (i.e. de-escalation techniques and communication skills). The combination of bird’s eye view and distinctive perspectives seems to provide visual feedback that gives detailed information covering all three training objectives. Receiving detailed feedback from multiple perspectives appears to be more beneficial for learning efficacy (Fukkink, Trienekens, and Kramer 2011; Guadagnoli, Holcomb, and Davis 2002). In addition, receiving feedback on an individual level is more personal and confronts the trainee directly, allowing the trainee to have a more active and engaged role in their learning process (Chan and Lam 2010). In comparison, the bird’s eye view invites feedback on a team level. Conclusively, police officers who have received a combination of bird’s eye view and a distinctive perspective have received feedback on an individual and team level, providing a broader range of relevant performance feedback and addressing personal efficacy directly. This appears particularly pertinent when utilising the suspect perspective since police officers receive novel information about the impact of their behaviour. Thus, hypothesis 1 is supported by our data: reviewing the training performance from the AAR perspective bird’s eye view in combination with suspect perspective enhances learning efficacy of police officers.

Testing the second hypothesis, we found that learning efficacy did not significantly differ during the AAR with and without the line of fire of the weapon of the police officers. It is possible that the line of fire feature did not provide information relevant for the performance of the police officers within the scenario and therefore was not relevant for learning. Another reason may be that the discussion of line of fire is sufficient when done verbally; thus, not relying on visual support to provide further information on performance. Lastly, processing feedback right after high-arousal training may be cognitively demanding for trainees (Jenkins, Semple, and Bennell 2021). If too many visual features are used while trainees recover from the arousal of a scenario and already receive verbal and visual performance feedback, the additional use of visual features such as the line of fire may not enhance learning further. It is therefore possible that while cognitive abilities are limited at the time of the AAR, the information provided by additional AAR features may not be processed properly (Mugford, Corey, and Bennell 2013). Conclusively, the second hypothesis is not supported by our data: the line of fire feature of the AAR did not enhance learning efficacy in police officers.

4.1. Limitations

While the results presented in our study provide valuable insights for practice and development of VR training in police, the study also has limitations. First, in this study, we assessed learning efficacy via a self-designed questionnaire. Ideally, learning and transfer would be assessed objectively on a behavioural level in a novel performance context. This was not feasible within this study. Therefore, the question regarding learning and the transfer of VR training to operational situations remains to be explored.

Second, the study’s focus was placed on the features that the VR AAR provides. Thus, the influence of verbal feedback and expertise of instructors was not explicitly controlled for. To standardise the feedback that instructors provide during the VR AAR, instructors received reference points for providing feedback. Nonetheless, police instructors still have individuals ways of delivering feedback (Kleygrewe et al. 2022). Because the type and modality of feedback influence learning (Hattie and Timperley 2007; Zhu et al. 2020), the way that instructors provide feedback may play a role in how officers perceive their learning experience. While all police instructors who participated in this study were experienced and well-instructed to use the VR AAR, their personal feedback style may have, in some way, contributed to the learning efficacy police officers experienced.

Lastly, learning efficacy was assessed on an individual level. This study did not explore team effects or team dynamics that can influence learning efficacy. For instance, four-person teams with experienced team leaders and police officers may have had a different learning experience and thus learning efficacy than teams of different compositions (Paoline III and Terrill, 2007). In addition, gender of police officers may also influence learning efficacy. For instance, female police officers experience differences in operational task assignments (Morash and Haarr 2012; Rabe-Hemp 2009) and thus may take unfavourable roles in team compositions, reducing their learning efficacy.

5. Conclusions

In conclusion, the AAR appears to be a promising feature of VR and provides police agencies with feedback opportunities that are difficult to simulate in real-life training (Murtinger et al. 2021). Reviewing training performance from various perspectives and utilising additional features to underscore performance makes the AAR an effective training tool that supports police instructors when giving feedback. To further enhance the use of the AAR in practice, police agencies should aim to identify AAR features that align with the learning objective of the training when setting up the VR training (Bennell et al. 2021). For instance, when trainees are tasked with clearing and searching various rooms, police instructors may utilise AAR features such as the walking routes the officers took, and display of gaze areas that were covered. In the AAR, the instructors and trainees can then review and compare which paths they took to clear a room, the areas their gaze covered, and whether the strategies to conduct a clearance changed from room to room. Similarly, instructors should avoid using AAR features that do not align with the training and learning objective. For example, during a training aimed at verbal de-escalation or taking cover to observe a suspects’ behaviour (i.e. a training that has no intention to rely on the service weapon), showing the line of fire would distract from the objective of the training. Thus, utilising too many features provides trainees with information that may not be relevant to them or the purpose of the training (Bennell et al. 2021).

VR providers can further support the implementation of AAR in police practice by developing context-specific review features. For instance, next to enabling trainees to view their performance from a first-person perspective and a suspect perspective, features such as showing the hit zones after using a service weapon may be relevant performance indicators. VR providers for police training should therefore consider developing these context-specific features in collaboration with police instructors. Lastly, to make the debrief process efficient and effective, VR providers should design an accessible AAR tool that makes it easy for police instructor to select and use the relevant features during the debrief. Because ‘feedback is one of the most powerful influences on learning’ (Hattie and Timperley 2007), designing and implementing VR AAR effectively in police training can enhance the delivery of feedback and subsequently improve learning and performance of police officers.

Acknowledgements

The authors would like to thank the Stadtpolizei Zürich for their collaboration in this study. In particular, we would like to thank Christoph Altmann for the organization and the police officers and instructors for participating in this study. We would also like to thank Refense; particularly, Ronny Tobler for taking part in this project.

Disclosure statement

The authors report there are no competing interests to declare.

Additional information

Funding

This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No 833672. The content reflects only authors’ view. Research Executive Agency and European Commission are not liable for any use that may be made of the information contained herein.