Training for the job: evaluation of a self-defence training programme for correctional officers

Abstract

We investigated to what extent correctional officers were able to apply skills from their self-defence training in reality-based scenarios. Performance of nine self-defence skills were tested in different scenarios at three moments: before starting the self-defence training programme (Pre-test), halfway through (Post-test 1), and after (Post-test 2). Repeated measures analyses showed that performance on skills improved after the self-defence training. For each skill, however, there was a considerable number of correctional officers (range 4–73%) that showed insufficient performance on Post-test 2, indicating that after training they were not able to properly apply their skills in reality-based scenarios. Reality-based scenarios may be used to achieve fidelity in assessment of self-defence skills of correctional officers.

Practitioner summary: Self-defence training for correctional officers must be representative for the work field. By including reality-based scenarios in assessment, this study determined that correctional officers were not able to properly apply their learned skills in realistic contexts. Reality-based scenarios seem fit to detect discrepancies between training and the work field.

Abbreviations: DJI: Dutch National Agency for Correctional Insitutes; ICC: Intraclass Correlation Coefficient

Keywords:

• Reality-based scenarios
• self-defence training
• correctional officers
• performance
• representative learning design

1. Introduction

Correctional officers are often confronted with demanding situations, for example when prisoners behave aggressively or violate rules. An evaluation of the working pressure of Dutch correctional officers showed high levels of aggression towards correctional officers (Jurriëns, 2017). More than half of the officers experienced incidents with physical violence (e.g. beating and kicking). The report stated that correctional officers suffer from emotional strain because of high pressure and anxiety. Research has shown that correctional officers in other countries also experienced high pressure, leading to incorrect interventions (e.g. Lambert et al., 2018; Kinman, Clements, and Hart, 2016).

A correctional officer must be able to act adequately and with proportional force in all situations. Self-defence-related incidents are dynamic, stressful, and unpredictable. Correctional officers need skills to flexibly react in conflict situations (Liebling, Price, and Shefer, 2011). It is therefore essential that self-defence skills learned in training transfer to complex scenarios on duty.

Körner and Staller (2018) described three reference environments for self-defence training: 1) the learning environment, where skills are learned; 2) the testing environment, where skills are evaluated and 3) the criterion environment, where skills are used in real-life situations. The challenge is to maximise the representativeness of the learning and testing environment to the criterion environment (Körner and Staller, 2018; Staller et al., 2018).

Optimal transfer of self-defence skills to the real-life working context requires training and testing that consists of tasks, behaviours, contexts, and constraints that replicate those of the working environment (Davids, 2008; Araújo, Davids, and Passos, 2007). The extent to which learning environments match working environments distinguishes a representative learning design from a non-representative learning design (Staller and Zaiser, 2015; Staller et al., 2018).

In non-representative designs, trainees practice isolated skills in general environments (e.g. a gym hall), wearing non-representative clothing, and practice without context, that is, without a link to specific situations on duty. In such environments, trainees do not face realistic problems and solutions, and do not experience realistic stress or anxiety (Nieuwenhuys and Oudejans, 2011; Oudejans, 2008; Renden et al., 2014). Studies have found that the lack of representative contexts in training limited the ability to apply skills in both similar and different contexts (e.g. Broadbent et al., 2015; Pinder, Headrick, and Oudejans, 2015; Renden, Savelsbergh, and Oudejans, 2017).

In contrast, representative designs incorporate realistic problems and solutions in training. Trainees execute representative skills, wear representative attire, and train in reality-based contexts. They can detect relevant information and dynamically interact with the relevant constraints of the performance environment (Davids et al., 2006; Pinder et al., 2011; Pinder, Renshaw, and Davids, 2009). As such, this interaction stimulates responses of trainees, that is, decision making and movement execution, based on information comparable to that in the work-related context (Araújo et al., 2007; Pinder et al., 2011). To date, the training of work-related tasks in reality-based contexts has mainly been investigated with police officers (e.g. Andersen et al., 2016; Oudejans, 2008; Nieuwenhuys and Oudejans, 2011; Renden et al., 2017). Research into the training of work-related tasks and its transfer to reality-based contexts with correctional officers is limited. The current study aims to address this gap in the literature by investigating the self-defence training of correctional officers.

Novice correctional officers in the Netherlands receive training to master self-defence skills (e.g. punches, restraining). The training module consists of 16 half-day sessions and is part of the training to become a correctional officer. The aim is to provide trainee officers with tools for correct interventions in dangerous situations. In the current self-defence training module, it is unidentified to what extent the training is representative. Discrepancies between the circumstances during training and on duty, may make it difficult to perform well in threatening scenarios on duty (e.g. Anderson, Litzenberger, and Plecas, 2002; Renden et al., 2015). Consequently, correctional officers may not be optimally prepared to apply the self-defence skills in reality-based scenarios. The main aim of this study is to determine to what extent correctional officers can apply the learned self-defence skills in reality-based scenarios. The study provides information whether a current self-defence training programme adequately prepares trainee correctional officers for the work field.

2. Materials and methods

2.1. Participants

Twenty-eight trainee correctional officers (26 men and 2 women; M age = 28.7, SD = 5.56) participated in this study. They were new employees of the Dutch National Agency for Correctional Institutes (DJI), who participated in the self-defence training programme. Participants’ trait anxiety was assessed with the State-Trait Anxiety Inventory (validated by Van der Ploeg, Defares, and Spielberger, 1980). All scores were lower than the norm (Min = 24, Max = 36, norm is 36.7), indicating that the participants had no tendency to respond with more than average levels of anxiety to threatening situations.

Due to personal circumstances, one male participant dropped out after the first test session and was excluded from the study. 24 participants completed all three-test sessions; three participants missed a single test session. In total, 27 correctional officers were included for analysis. The protocol was approved by the ethics committee of the research institute.

2.2. Design

To determine to which degree participants were able to apply self-defence skills in reality-based scenarios before and after the self-defence training, a within-subject repeated measures design was used to compare three test sessions (Pre-test, Post-test 1, and Post-test 2). The Pre-test was carried out before the start of the training, Post-test 1 halfway through, and Post-test 2 at the end of the training. There were three weeks of training in between each test session. Within the organisational and time constraints (e.g. limited availability of participants due to work obligations), it was not possible to include a retention test after Post-test 2.

2.3. Experimental set-up

2.3.1. Training sessions

All participants followed a self-defence training module. They learned the basic skills that are needed on duty, for example, holding off or handcuffing a prisoner. The training consisted of 16 sessions of three hours, distributed over six weeks. The trainees thus received on average seven hours of training a week. The training mainly took place in a gym hall and towards the end of the training occasionally in an empty cellblock. Prior to the current study, the authors (Hutter et al., 2019) systematically observed all 16 training sessions of the programme with a view to the pedagogy used in the training sessions. The observations showed that most of the training was characterised by a traditional linear pedagogy (see Williams and Hodges, 2005), in which trainees learned isolated skills, usually practiced in pairs. Trainees practiced an isolated skill step-by-step and then moved on to practice another isolated skill in the same way, in general three or four skills per session. Instructors gave both implicit and explicit learning instructions and feedback (see Masters and Maxwell, 2004). In a relatively small part of the training programme (mainly in the second half of the training programme), attention was paid to the application of the skills in role-plays between trainees. That is, a simulated interaction between the correctional officers and aggressors. Using more reality-based contexts, implicit instructions and interaction with environmental constraints, these latter parts of the training contained aspects of non-linear pedagogy (see Chow et al., 2006, 2007).

2.3.2. Test sessions

We developed reality-based scenarios specifically for the test sessions in this study. The learning aims of the self-defence training module, as documented by the DJI, were combined into seven clusters of self-defence skills (for a description of the clusters, see below). These clusters consisted of pure self-defence skills, such as fending off a grab or hold, as well as associated de-escalating skills, such as handcuffing and uncuffing. The clusters of skills were tested in reality-based scenarios that replicated situations on duty. To design the reality-based scenarios we held two focus groups: one with current staff members and one with instructors of the self-defence training. These focus groups resulted in specific scenarios in which (clusters of) self-defence skills are needed and indicated which scenarios occurred frequently in the daily work of correctional officers. The clusters of self-defence skills are as follows:

• Fending off a grab or hold. The prisoner suddenly aggressively grabs the correctional officer by the shirt. The correctional officer has to anticipate and break free from the grab.

• Striking, punching, and kicking. The prisoner aggressively approaches the correctional officer at a fast pace. The correctional officer has to keep the prisoner at a distance through strikes, punches, or kicks with the purpose to bring the prisoner out of balance or out of action.

• Manoeuvring into safety and activating the alarm device. The prisoner threatens the correctional officer with a stabbing weapon. The correctional officer has to move to a safe distance as soon as possible, and activate their portable alarm.

• Controlling and restraining. The prisoner threatens a co-worker. The correctional officer has to help this colleague by gaining control and holding the (non-cooperative) prisoner under restraint with the help of a third colleague.

• Handcuffing and uncuffing. The correctional officer has to bring the prisoner to the ground, handcuff the prisoner on the ground, transport him to the cell and uncuff the prisoner in the cell.

• Handling the baton and keeping a safe distance. The prisoner aggressively approaches the correctional officer, holding an elongated object (e.g. spoon, pen). The correctional officer has to keep the prisoner at a safe distance with the use of a baton.

• Safety awareness. The correctional officer has to deliver bad news to the prisoner. The correctional officer has to show that he or she is aware of safety by positioning themselves adequately to the prisoner (e.g. between the prisoner and the exit of the room) and potentially dangerous materials (e.g. letter opener, scissor).

The scenarios were staged in an empty prison department. An experienced actor played the role of the prisoner (two actors were involved: both were North African men, not extraordinarily muscular, and around 1.70 m tall). The actors did not wear protective equipment. The actors were experienced to work without body protection and to take on mild physical violence such as a kick or a punch. To protect the actors against more severe physical violence, they were instructed to cooperate with the trainee directly following a physical intervention, and the researchers stopped the scenario as soon as physical intervention had taken place. In case of disproportionate or dangerous use of force, the trainee correctional officers, the actor, as well as the researchers could immediately stop the scenario. The trainee correctional officer had to anticipate and act towards the prisoner as the situation unfolded and without prior information about the type of attack or required skill(s). The scenarios were designed with the aim to provoke application of a certain (cluster of) self-defence skills. We largely succeeded in this aim as the intended skills were indeed used by the participants in each scenario. Yet, if a trainee reacted with no action at all at first, the actors were instructed to try to keep provoking the trainee towards the use of the skill the scenario aimed for, as long as this was realistic. For example by remaining verbally aggressive and try to prompt the trainee to respond by applying light physical contact. If no action by the trainee followed after sustained attempts and the scenario was verging to becoming unrealistic, the scenario was stopped by the researchers. The participants only received a simple instruction about the location. For example, they received information that the prisoner was playing table tennis in a recreational room and that their task was to tell the prisoner that the recreational time was ending. A mock scenario was included to diminish the predictability of the violent nature of the scenarios. The additional scenario contained verbal aggression, but no physical attack, and were excluded from analysis.

For each cluster of self-defence skills, three standardised scenarios were developed to provoke the application of a certain (cluster of) self-defence skills. The three scenarios per cluster differed in location and the provided instruction for the correctional officer, but the required self-defence skills were the same. In this way, the correctional officers received a different scenario for the tested cluster in each of the three test sessions. For example, for Fending off a grab or hold, the correctional officer was unexpectedly grabbed by the shirt by the prisoner, but in one scenario the encounter took place in the recreational room and the correctional officer was instructed to inform the prisoner that recreation time was ending, while in another scenario the encounter was staged at the cell door with the instruction to ask the prisoner whether he received his groceries. In this way, scenarios were comparable enough to test the effect of training, but different enough to prevent predictability of the required skills. The scenarios were performed in a recreational room, which also served as a working room, in and around cells, in a kitchen, and consultation room in an empty prison department. For an overview of locations per cluster of skills and related instructions for the participant, see Appendix.

2.4. Assessment

Three experts of the DJI judged the performance of self-defence skills. The experts were all instructors of the self-defence training and were experienced in the assessment of trainees in reality-based scenarios. The experts were regularly involved in the evaluation (in terms of passing or failing the training) of officers. An online assessment tool was developed in Qualtrics (see Figure 1). The assessors watched video recordings from two different camera perspectives and subsequently gave their ratings. The recordings were randomly presented, and the assessors were unaware which test session (Pre-test, Post-test 1, or Post-test 2) was shown in each video. A number of recordings were repeated for each assessor, to check for test-retest reliability. A seven-point Likert scale, ranging from 1 (very bad) to 7 (very good) was used to rate the overall performance holistically. The experts could next indicate which positive and negative aspects of performance (i.e. performance characteristics) steered their judgement. These performance characteristics (see Table 1) were listed under the Likert scale in the assessment tool (see Figure 1). The assessments were performed after all test sessions were completed.

Figure 1. The online assessment tool. This figure illustrates the design of the assessment tool including Likert scale and performance characteristics.

Table 1. Tested self-defence skills with associated performance characteristics.

	Proportionality /Subsidiarity	Effectiveness of action	Resolution	Safety for prisoner	Safety for participant	Body posture	Positioning	Communication	Technique handcuffing	Technique uncuffing	Scan area	Use of area	Use of baton	Use of alarm device
Fending off a hold or grab	v	v	v	v	v	v	v	v
Striking, punching and kicking	v	v	v	v	v	v	v	v
Controlling and restraining	v	v	v	v	v	v	v	v
Handcuffing	v	v	v	v	v	v	v	v	v
Uncuffing	v	v	v	v	v	v	v	v		v
Manoeuvring into safety and activating the alarm device		v	v	v	v	v	v	v			v	v		v
Handling of baton	v	v	v	v	v	v	v	v			v	v	v
Making/ keeping a safe distance			v				v	v			v	v
Safety awareness			v	v	v	v	v	v			v	v

Inter-rater reliability was determined using Quadratic Weighted Kappa, because the assessment scheme was not fully crossed and the absolute discrepancy of scores matter, particularly on a 7-point Likert scale. Inspection revealed differences between experts in distribution of scores. To be able to adequately compare their scores, we standardised the Likert scores by converting them to z-scores. The average Weighed Kappa over all skills showed a moderate inter-rater reliability of .55 (range: .44 - .60), except for the cluster Fending off a hold or grab, which showed a poor inter-rater reliability of .15. Intra-rater reliability was determined using Intraclass Correlation Coefficient (ICC), showing an excellent test-retest reliability for all skills. The average ICC was 0.88 (range: .83–.97), indicating a good agreement between the results of successive assessments.

2.5. Dependent variables

The scores on the 7-point Likert scale were used as the measurement of overall performance. For a precise assessment, we split some clusters of the self-defence skills into isolated self-defence skills for scoring purposes. In the cluster of handcuffing and transport, for example, the experts gave separated holistic scores for handcuffing and uncuffing. Similarly, in the cluster with the baton and keeping a safe distance, the handling of the baton, and the ability to make and keep distance were assessed separately. In total, performance of nine self-defence skills was thus assessed.

After the holistic score for performance as given, assessors could indicate performance characteristics that stood out in a positive or negative fashion. The performance characteristics are as follows:

• Proportionality/subsidiarity. The correctional officer used force in proportion to the behaviour of the prisoner.

• The effectiveness of action. The correctional officer repelled an attack (if necessary) and succeeded in holding of the prisoner.

• Resolution. The correctional officer showed determination in his or her actions (no hesitations).

• Safety for the participant. The correctional officer was vigilant of his own safety.

• Safety for the prisoner. The correctional officer was vigilant of the safety of the prisoner.

• Body posture. The correctional officer showed an active posture; was ready to intervene if necessary and displayed that.

• Positioning. The correctional officer moved forward when possible and kept distance when necessary.

• Communication. The correctional officer communicated what the prisoner needed to do and what the prisoner was not allowed to do.

• Technique handcuffing. The correctional officer handcuffed the prisoner adequately.

• Technique uncuffing. The correctional officer uncuffed the prisoner adequately.

• Scan area. The correctional officer scanned the area to remain aware of the surroundings.

• Use of area. The correctional officer used the area (nearest exit, use of objects, etc.) to gain control over the situation.

• Effectiveness of baton. The correctional officer used the baton in an efficient and useful way.

• Effectiveness of alarm. The correctional officer adequately activated the alarm.

Not every performance characteristic was relevant for each self-defence skills. Table 1 shows which performance characteristics were deemed relevant for which self-defence skills, and therefore presented in the online assessment tool for each cluster of skills.

2.6. Procedure

The correctional officers were tested in small groups. They received professional attire, including handcuffs and the alarm device and were placed in separate empty cells to await their turns, this prevented them from seeing or hearing their colleagues scenarios. If it was his or her turn, the correctional officer received instructions for a scenario in the waiting cell, then performed the scenario, and was brought back afterward to the cell. In between scenarios, the researchers prepared the following scenario and reminded the actor of the specific instruction for that scenario. To prevent that correctional officers could guess which cluster of skills was required next, the order of the clusters was randomised. To prevent systematic influence of the type of scenarios on the three test sessions their order was also randomised (e.g. in Pre-test, participant A performed the fending off a grab scenario in the kitchen; in Post-test 1 in the cell, and Post-test 2 in the workroom. For participant B the order was different, for example workroom first, then kitchen, then cell).

2.7. Data analysis

We performed a non-parametric Friedman ANOVA to evaluate differences in performance scores among the three test sessions. Post-hoc analysis, using Wilcoxon Signed-Rank tests were conducted to identify specific differences between test sessions. We observed frequency tables to examine to what extent the different performance characteristics were indicated, and how the assessors’ feedback changed over time.

To determine whether correctional officers’ performance was of a sufficient level after the full programme, median Likert scores of Post-test 2 were calculated. Because we used an uneven Likert scale from 1 (very bad) to 7 (very good), we deemed median Likert scores below 4 as indicative of an insufficient level of performance.

3. Results

Participants’ performance variables are presented in Figure 2. This figure shows the z-scores of each self-defence skill over the Pre-test, Post-test 1 and Post-test 2 as well as the % participants that showed a sufficient level of performance (Likert score 4 or higher) at Post-test 2.

Figure 2. Mean z-scores (on the primary y-axis) and percentage of prison guards that showed a sufficient level performance per skill (on the secondary y-axis) for the Pre-test, Post-test 1 and Post-test 2. The lines represent mean z-scores displayed on the primary y-axis and illustrates the improvement of performance between the three test sessions. Significant differences are indicated by continued lines, dashed lines indicate non-significant changes. The bars represent the percentage (secondary y-axis) of prison guards that scored 4.0 or higher (sufficient level of performance) in each test session.

3.1. Performance of self-defence skills

The Friedman ANOVA on the performance scores for fending off a hold or grab, controlling and restraining, handcuffing, uncuffing, handling the baton, making/keeping safe distance, and safety awareness revealed significant differences among the three test sessions, χ²(2) = 7.59, p < .05; χ²(2) = 9.02, p < .05; χ²(2) = 18.500, p < .001; χ²(2) = 13.561, p < .01; χ²(2) = 7.18, p < .05; χ²(2) = 8.24, p < .05; χ²(2) = 20.36, p < .001, respectively. For striking/punching and kicking and manoeuvring into safety and activating the alarm device, no significant difference in distributions among the three test sessions was found, χ²(2) = 2.57, p = .28; χ²(2) = 5.02, p = .08.

Wilcoxon Signed Ranks tests showed that performance scores for fending of a hold or grab, striking/punching and kicking, controlling and restraining, handcuffing, uncuffing, manoeuvring into safety and activating the alarm device, keeping safe distance, and safety awareness were significantly higher in Post-test 2 compared to the Pre-test, Z = −2.83; p <.01; Z = −2.00, p < .05; Z = −2.75, p < .01; Z = −3.92, p < .01; Z = −3.67, p < .01; Z = 2.93, p < .01; Z = −2.34, p < .05; Z = −3.72, p <.01, indicating better performance on these skills after the training. For handling the baton, there was no significant difference between Post-test 2 and Pre-test, Z= −1.703, p =.088.

Only for handcuffing, statistical analyses revealed a significant increase in performance between all tests (from Pre-test to Post-test 1, Z = −2.10, p < .05, and from Post-test 1 to Post-test 2, Z = −2.65, p < .001). For fending off a hold or grab, striking, punching and kicking, controlling and restraining, uncuffing, manoeuvring into safety and activating alarm device, and safety awareness, performance scores were significantly higher in Post-test 1 compared to Pre-test, Z = 2.71, p < .01, Z = −2.04, p < .05, Z = −2.47, p < .01, Z = −2.64, p < .01, Z = −2.02, p < .05, Z = −3.92, p < .001, but not for Post-test 1 to Post-test 2, indicating no significant further improvement in the second part of the training. For handling the baton and making/keeping safe distance there were no significant differences between Pre-test and Post-test 1, but significant differences were found between Post-test 1 and Post-test 2, Z = −2.03, p < .05; Z = −2.92, p < .01, indicating only significant performance improvements after the second half of the training.

3.2. Level of performance

Median Likert scores showed a sufficient performance after the training in fending off a grab or hold (Mdn = 5.0, IQR = 1.25), making/keeping safe distance (Mdn = 5.0, IQR = 2.0), and safety awareness (Mdn = 4.5, IQR = 4). Moreover, less than 30% of the correctional officers still scored below 4.0 in Post-test 2 for these clusters (see Figure 2). For manoeuvring into safety and activating the alarm device (Mdn = 4.0, IQR = 4.0), controlling and restraining (Mdn = 4.0, IQR = 3.25), and striking, punching and kicking (Mdn = 4.0, IQR = 3.0), median Likert scores showed a neutral level (not bad, not good) after training. For these skills respectively 38%, 35% and 44% of the correctional officers still scored below 4.0 in Post-test 2, indicating an insufficient level of performance. For handcuffing (Mdn = 3.5, IQR = 4.0), uncuffing (Mdn = 2.0, IQR = 3) and handling the baton (Mdn = 3.5, IQR = 2.0), the median Likert scores showed an insufficient level of performance after the training. For these skills respectively 50%, 73%, and 50% of the correctional officers still scored below 4.0 in Post-test 2, also pointing to an insufficient level of performance after training.

Figure 3. Frequencies of performance characteristics per cluster. This figure illustrates to what extent different performance characteristics were indicated in negative or positive sense in the three test sessions and whether this feedback changed over the three test sessions (first bar each time represents the Pre-test, the second bar Post-test 1 and the third bar Post-test 2). The y-axis represents frequencies.

3.3. Performance characteristics

The experts explicated their assessment of the participants' performance by tagging performance characteristics when these characteristics stood out positively or negatively during the performance. Figure 3 shows the performance characteristics indicated as positive, neutral or negative over Pre-test, Post-test 1 and Post-test 2.

Frequency inspection revealed three patterns of changes in the performance characteristics that point to improvement in the performance of the participants over the test sessions. 1) Frequency of negatively indicated performance characteristics decreases from Pre-test to the Post-tests. 2) Frequency of positively indicated performance characteristics increases. 3) Performance characteristics that transformed from frequent negatives in Pre-test to frequent positives in the Post-tests. For reasons of readability, we provide an example for each of these patterns from our results, and refer the reader to Figure 3 for all other changes. In the cluster manoeuvring into safety and activating the alarm device, the criterion body posture was less frequently indicated as a negative point in Post-tests compared to the Pre-test. In the cluster striking, punching and kicking, the proportionality/subsidiarity criterion was increasingly indicated as a positive point in Post-tests compared to the Pre-test. In the cluster fending of a hold or grab, the resolution criterion was a frequent negative point in Pre-test but then changed to a frequent positive point in Post-tests, indicating major improvement in resolution.

4. Discussion

In this study, we investigated to what extent correctional officers were able to apply self-defence skills from training in reality-based scenarios. The correctional officers followed a self-defence training programme as part of their training to become correctional officers. In a within-subject design with repeated measures, the performance of self-defence skills was tested before the programme, halfway through, and after the programme. Correctional officers were tested on seven clusters of self-defence skills in different scenarios. The results showed improved performance after the self-defence training for all self-defence skills, except for handling the baton.

The performance characteristics that assessors provided, illustrated specific elements that contributed to the overall improvement. Proportionality/subsidiarity and resolution seem to improve particularly for skills that require physical contact to hold off the prisoner, such as fending off a hold or grab and striking/punching and kicking. In accordance with this finding, Renden et al. (2017) reported improvement in resolution and proportionality of police officers’ actions after reflex-based training, indicating that training in primary responses is an important aspect in skills that require physical contact. In contrast, for skills with the aim to avoid contact (e.g. manoeuvring into safety and activating the alarm device, handling the baton, and making/keeping safe distance), situational awareness (e.g. scan of area, use of area) and positioning improved particularly. This finding is in line with that of Andersen and Gustafsberg (2016) who stated that improved situational awareness and positioning led to better performance in scenarios where de-escalating skills were needed.

Another interesting suggestion from the performance characteristics relate to resolution in collaborative tasks, as seen in the controlling and restraining cluster in which participants acted together with an experienced co-worker. Resolution was already frequently indicated as a positive characteristic in the Pre-test for this cluster, and its frequency as positive characteristic clearly diminished slightly for Post-tests. This may suggests that, pre-training, correctional officers followed the instructions of the colleague automatically, showing no hesitations. During and after training, the officers might have been more autonomous, taking more time for their own decision-making, instead of unthinkingly following the instruction of a colleague, which resulted in less frequent display of strong resolution.

Overall, correctional officers were better able to apply self-defence skills in reality-based scenarios after the self-defence training programme than before. As such, the training seems to contribute to the preparation of correctional officers for duty. Nevertheless, for all skills, a considerable number of the correctional officers (range: 4% –73%) still showed an insufficient level of performance after the training (below 4 on the 7-point Likert scale). This means that, despite all improvements, almost all correctional officers still have deficiencies in certain self-defence skills after their training. This may indicate that the current self-defence training programme does not adequately prepare trainees to a sufficient level for their work as a correctional officer.

During debriefing, most correctional officers stated that the requested skills in the reality-based scenarios corresponded with the learned skills in the self-defence training. Furthermore, they stated that the scenarios offered them sufficient opportunity to demonstrate their skills. However, the correctional officers also indicated that it was not clear to what extent they could use violence towards the actors in the scenarios. The actor did not wear protection (to keep it as realistic as possible), while in training, pads were used as protection.

To diminish discrepancies in circumstances between the training, test environment, and the work field, the learning and testing environment must be representative for the criterion (i.e. real-life) environment (Körner and Staller, 2018). In our study, the correctional officers wore full equipment including a belt with handcuffs and alarm device in the tests, while in training correctional officers were in sports gear and shoeless, thus pointing to a lower level of representativeness in the training environment than the testing environment. A frequently discussed step towards higher representativeness of self-defence training is to create a more realistic training and test environment (Staller et al., 2018). Implementation of reality-based contexts will ensure that correctional officers feel better prepared for threatening situations due to a better linkage between physiological and psychological processes (Andersen et al., 2016). This notion is supported by Renden et al. (2015). They found that police officers wished for more realistic training to improve their skills. Reality-based contexts involve realistic problems, solutions, stress, and contexts and will so improve the transfer and retention of skill to the work field (Oudejans and Pijpers, 2009, 2010; Staller and Zaiser, 2015). Including a reality-based context in the training is a fast and effective way to increase the performance in scenarios in which trainees are under pressure (e.g. Oudejans, 2008; Nieuwenhuys and Oudejans, 2011).

This study not only evaluates the self-defence training programme for correctional officers, but also provides an extensive example of methodology for evaluation, that is how training can be evaluated using a representative testing design including reality-based scenarios. By stressing representativeness in the test scenarios (test environment), we could determine that training (learning environment) is not yet fully representative for the eventual performance environment (criterion environment; Körner and Staller, 2018). In developing our test sessions and assessment protocol, we borrowed from the methodology of Renden et al. (2017) to test self-defence skills of police officers, and we used focus groups with expert instructors to determine which self-defence skills are needed in the work of correctional officers. In this way, we were able to identify the relevant personal, task, and environmental constraints on duty (Davids et al., 2006; Pinder et al., 2011) and develop a representative test protocol to test the relevant practical skills and competencies of correctional officers. In addition, it would be beneficial for follow-up studies to investigate the relationship between the identified tasks and constraints on duty, and different solutions provided by the officers. While the reality-based test scenarios point to gaps between training and performance environments, they also immediately provide a rich source of examples of how training can be made more representative.

Several limitations should be kept in mind. When interpreting the results of this study, the complexity of assessing performance in reality-based scenarios should be taken into account. Our assessment tool showed only a moderate inter-rater reliability. This may be caused by two reasons. First, despite anchors, assessors possibly use a personal interpretation of the scale when assessing reality-based performance. Inspections confirmed differences in distributions on scores assessed by the assessors. Some assessors tended to use only central values of the Likert scale, while others used the entire range. Second, inspection of performance characteristics indicated that assessors focus on different aspects of performance. For example, in the cluster fending off a hold or grab, the effectiveness of the action was mainly indicated as a positive point by one assessor but barely indicated by another assessor. Thus, it seems difficult to achieve homogeneity in assessment in reality-based scenarios. By standardising the assessment scores of the assessors and by randomising the participants to be assessed by each assessor we have prevented a systematic bias, however.

Because of organisational reasons (e.g. limited availability of participants due to work obligations), it was not possible to include a retention test and control group in this study. Due to the lack of a control group, we must be cautious in drawing conclusions about the effect of the training because the influence of other factors cannot be excluded. We therefore draw conclusions about the performance level at three moments in time, but do not strictly attribute these results to the training only.

To summarise, we have found clear indications that correctional officers’ performance in reality-based scenarios improved after the self-defence training programme. Improvement was accompanied by better resolution, proportionality/subsidiary, effect of action, technique, positioning, body posture, safety of correctional officer and prisoner and situational awareness (scanning and use of area). Nevertheless, for each skill there has been a considerable number of correctional officers that still showed insufficient performance on Post-test 2, indicating that correctional officers were not able to apply their skills to a sufficient level in reality-based scenarios. Including reality-based scenarios in the training may be one way to further increase the self-defence skills of correctional officers.

Acknowledgement

The authors would like to thank Harry Hittema, operational manager of the Dutch National Agency for Correctional Institutes (Dienst Justitiële Inrichtingen, DJI) for his help in executing this experiment and Chakib El Aomari and Armand de Wit, instructors of the DJI, for their help with the assessment. We also would like to thank Monica Tesarova for her help in conducting the experiment. Finally, we thank Sidi Bensalah and Behrouz Seiri for all their efforts in acting as prisoner in the scenarios.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability

The data that support the findings of this study are available on request from the corresponding author, [M.K.]. The data are not publicly available due to [restrictions e.g. their containing information that could compromise the privacy of research participants].

Appendix

An overview of locations per cluster of skills and related instructions for the participants

Table

	Locations	Instruction for the trainee correctional officer
Striking, punching and kicking	Recreational room	You are going to a recreational room and tell the prisoner that the recreational time is ending and to go back into cell
	Workroom	You are going to a workroom and tell de prisoner that he or she must clean up his or her working space.
	Cell	You are going to a cell and ask the prisoner if he or she wants their regular 30 min outside to get a breath of fresh air
Fending off a hold or grab	Recreational room	You are going to a recreational room and tell the prisoner that the recreational time is ending and to go back into cell
	Workroom	You are going to a workroom and ask the prisoner how the work is going
	Cell	You are going to a cell and ask if the prisoner received all his ordered groceries
Controlling and restraining + handcuffing and uncuffing	Recreational room	For all locations: A colleague is attacked by an aggressive prisoner. Your task is to help your colleague in distress
	Workroom
	Cell
Manoeuvring into safety and activating the alarm device	Kitchen	You are going to a kitchen and ask the prisoner if the cooking is going well
	Workroom	You are going to a workroom and ask the prisoner if he or she has all the materials to work properly
	Cell	You are going a cell and ask the prisoner if the meal tasted well
Handling the baton and making/keeping safe distance	Recreational room	You are going to a recreational room and tell the prisoner to finish the table tennis game because the recreational time is ending
	Kitchen	You are going to the kitchen and ask the prisoner to keep the sandwiches device clean
	Workroom	You are going to a workroom and tell the prisoner that he or she must finish their project before starting a new project
Safety awareness	Consulting room with two seats and a table with dangerous materials (e.g. scissors, pen)	You receive a prisoner for a bad news conversation. You tell the prisoner that him or her remand is extended
		You receive a prisoner for a bad news conversation. You tell the prisoner that he or she will transferred to another prison
		You receive a prisoner for a bad news conversation. You tell the prisoner that he or she will be expelled to country of origin