Pilot evaluation of a brief training video aimed at reducing mental health stigma amongst emergency first responders (the ENHANcE II study)

Abstract

Background

First responders (i.e. police and ambulance staff) have increasingly become part of the mental health care system, often being the first port of call for those experiencing a crisis. Despite their frequent involvement in supporting those with mental health problems, there is evidence that mental health stigma is high amongst first responders.

Aims

The aim of the present study was to evaluate a brief training video aimed at reducing mental health stigma amongst first responders.

Methods

First responders watched a training video based on the cognitive behavioural model of mental health stigma, and involved contributions from people with lived experience, and first responders. Measures of mental health stigma were collected before and after viewing the training.

Results

The training video produced small but significant improvements in mental health stigma, and these effects did not differ between police and ambulance staff. We were unable to determine what psychological constructs mediated this change in stigma. The feedback on the training video was generally positive, but also indicated some key areas for future development.

Conclusions

The present study provides encouraging evidence that levels of mental health stigma can be improved using a resource-light training intervention.

Keywords:

• Mental health
• stigma
• first responders
• ambulance
• police
• training

1. Introduction

The rates of both common and serious mental health problems have increased substantially in recent years (Mind, 2020), while rates of funding for UK mental health services have decreased (Roberts, 2015). The mismatch between supply and demand has meant that the ambulance and police services (hereafter referred to as first responders) have increasingly become part of the mental health care pathway.

First responders are often the initial port of call when a person is experiencing a mental health crisis (Roberts & Henderson, 2009; Shaban, 2006; Short et al., 2014). In 2018, 181,272 mental health-related 999 calls were received; ∼1.2% of all calls received (NHS Digital, 2018). This figure reflects a 44% increase in mental health-related calls over a two-year period (Rachid et al., 2018). Despite their extensive involvement in mental health care provision, levels of stigma towards people with mental health problems among police officers and paramedics can be problematic (Clayfield et al., 2011; Emond et al., 2015), with some suggestion that police officers in particular report greater stigma than their paramedic colleagues (Krakauer et al., 2020). Patient testimonies evidence a strong pattern of dissatisfaction amongst those who have received mental health support from first responders (Clarke et al., 2014). Several organisations have called for a need to improve the relationship between first responders and people experiencing mental health problems (All Party Parliamentary Group on Mental Health, 2015; Cummins & Edmondson, 2016; Department of Health Concordat, 2014; Wood & Watson, 2017).

The present study is a sub-study within the larger ENHANcE project. The overall goals of the ENHANcE project were to investigate levels of mental health stigma amongst first responders and explore ways to improve any stigma found. Within the first part of this project (ENHANcE I) (Hazell et al., 2021), we found that mental health stigma was better amongst first responders compared to figures reported in Time to Change reports, but worse than that found in our own general population sample. Rates of mental health stigma were predicted by constructs highlighted in the cognitive-behavioural model of stigma. The cognitive-behavioural model of mental health stigma stipulates that stigma is the summation of a lack of knowledge (ignorance), negative attitudes (prejudice), and disadvantaging behaviour (discrimination) (Corrigan et al., 2005). This model of mental health stigma has been well-evidenced, with studies showing robust relationships between the three constructs (e.g. Corrigan, 2000; Corrigan et al., 2003); and therein provides three potential targets for anti-stigma initiatives (i.e., increasing accurate knowledge, challenging negative attitudes, and amending disadvantageous behavioural responses).

The present study reports on the second of these overall goals i.e., improving stigma amongst first responders. The approaches that have been demonstrated to bring about such an improvement are education and contact (Corrigan et al., 2005). That is, providing accurate information about mental health and exposing individuals to the reality of living with mental health difficulties can both produce attitudinal and behavioural changes. In the ENHANcE II study, we utilised educational and contact methods within a brief training video to reduce mental health stigma, with the addition of a self-affirmation priming exercise to further promote positive change. We piloted the training video amongst first responders, using a pre-post design. The research aims for the present study are:

1. To determine whether a brief online training video can reduce mental health stigma amongst first responders.

2. To compare the efficacy of the training between police officers and ambulance services staff.

3. To identify what psychological constructs explain any changes in mental health stigma from pre- to post-training.

4. To evaluate how satisfied participants were with the online training video.

2. Method

2.1. Design

The present study uses a repeated measures pre-post design. The pre-data (T0) was collected as part of a separate cross-sectional study (ENHANcE I) (Hazell et al., 2021). Participants were identified using a self-generated unique participant ID; this ID was used to match the ENHANcE I data with the data collected in the current study (T1) to enable a pre-post assessment of our ENHANcE II training video.

2.2. Participants

To be eligible to participate in this study, persons had to be aged 18 or over and working for either police or ambulance service in the South of England. Participants were asked to complete the ENHANcE I study before the present study, but this was not compulsory. That is, participants could view the training video and complete the post-assessment without providing T0 data. An a priori power calculation determined that to conduct a 2 × 2 mixed ANOVA detecting a medium effect size with 95% power, a minimum sample size of 54 participants was required. We had a final sample of 83 participants.

2.3. Training video

The goal of the brief ENHANcE training video was to reduce mental health stigma amongst first responders. To develop the training video, we utilised two sources of information: (1) the cognitive behavioural theory of mental health stigma (Thornicroft et al., 2007); and (2) the lived experience of people with mental health problems as well as first responders. The training video contained testimonies that described what it is like to experience a mental health crisis (patient perspective) with simulations, testimonies outlining examples of good and poor practice (patient and first responder perspective), and finally a step-by-step guide for first responders on how to effectively support those with mental health difficulties. A full account of the training video development is detailed within the supplementary material.

2.4. Procedure

The procedure for the T1 data collection mirrored that of ENHANcE I. The promotional materials contained a weblink to access the training video and T1 assessment pack. Participants were first presented with the participant information sheet and then asked to complete a consent statement. Participants were then asked to provide their participant ID, followed by a self-affirmation priming question. They were then shown the brief training video and asked to complete several mental health stigma questionnaires and a training evaluation questionnaire. At the end of the survey, participants were presented with a debrief statement. All of the data (both T0 and T1) were collected online.

2.5. Measures

The measures included reflected a mixed-methods approach. As stated in the original study protocol, we collected data on the Mental Health Knowledge Schedule (MAKS) (Evans-Lacko et al., 2010). However, upon assessing the psychometric properties of the MAKS we were unable to find support for the pre-established factor structure or establish a revised valid structure. We have therefore made the decision not to report any findings related to the MAKS.

2.5.1. Social Distance Scale (SDS)

The version of the Social Distance Scale (SDS) used in the present study assessed the extent to which a person wishes to distance themselves from someone with mental health problems (Link et al., 1987). Social distancing encompasses both attitudinal (Wark & Galliher, 2007) and behavioural (Corrigan et al., 2001) responses towards a particular group. The SDS has 7 items that form a single scale, where a higher score indicates less need for social distance. In the present data, the measure has strong internal consistency (Cronbach’s α = .86).

2.5.2. Attribution Questionnaire (AQ)

The original Attribution Questionnaire (AQ) (Corrigan et al., 2002) had 20 items assessing beliefs about people with mental health problems. Within our psychometric analysis, three items did not load onto any factors and were excluded. The version of the AQ used here, therefore, has 17 items separated into two factors: (1) fear towards people with mental health problems; and (2) sympathy for people with mental health problems. A higher score indicates greater fear or sympathy towards those with mental health problems. In the present data, the measure has strong internal consistency (fear scale: Cronbach’s α = .92; sympathy scale: Cronbach’s α = .73).

2.5.3. Reported and Intended Behaviour Scale (RIBS)

The Reported and Intended Behaviour Scale (RIBS) (Evans-Lacko et al., 2011) has eight items. Our psychometric analysis found two factors: (1) current behaviour; and (2) intended behaviour. The current behaviour subscale refers to how persons are currently behaving, with a higher score indicating greater contact. The current behaviour scale will be used to describe participants familiarity with persons experiencing mental health problems. The intended behaviour subscale measures the likelihood of future discriminatory behaviour where a high score indicates greater willingness to have contact. In the present data, the measure has strong internal consistency (intended behaviour: Cronbach’s α = .73).

2.5.4. Self-affirmation prime

Self-affirmation theory posits that when people perceive their self-image has been threatened they are likely to act in a way that will support their self-image (Steele, 1988). In practice, reminding a person of a value that is important to them can improve their receptiveness to subsequent information that is concordant with that value (Crocker et al., 2008). We asked our first responder participants to rate how important the following value was to them: “It is important to ensure the safety and welfare of everyone I come into contact with as part of my role as a first responder”; using a Likert scale from 0 (not at all important) to 10 (very important). It was anticipated that where first responders perceived the statement as very important to them, they would be “affirmed” and therefore be less defensive and more open-minded to the training material.

2.5.5. Questionnaire for Professional Training Evaluation (Q4TE)

The Questionnaire for Professional Training Evaluation (Q4TE) (Grohmann & Kauffeld, 2013) has six items assessing attendees’ perceptions of the quality of the training they received—each item reflecting a separate underlying factor. A higher score indicates a more positive perception of the training.

2.5.6. Training feedback

We asked participants whether they would recommend the training to their colleagues using a yes/no response scale, as well as a free-text box where participants could provide written feedback. There was no word or character limit for this free-text box.

2.6. Ethics

All participants who took part in either the previous study or the present one did so anonymously. Participants were asked to provide consent for their data to be used in the present study, and for their data from both studies to be linked together using their participant ID. Participants provided consent for their direct quotes to be used in research publications. The previous cross-sectional survey (IRAS ID: 224998; REC reference: 17/LO/1536) and the present study (IRAS ID: 233216; REC reference: 18/HRA/0421) received approval from a Research Ethics Committee and/or Health Research Authority.

2.7. Analysis plan

All quantitative data were analysed using SPSS. The data was first assessed to ensure it met the requirements for parametric testing. Our analysis plan was devised using the SDS as a measure of overall mental health stigma (reflecting a summation of attitudinal and behavioural responses to those with mental health problems), and AQ and RIBS scales were used to measure attitudes and future discriminatory behaviour towards people with mental health difficulties, respectively. To address the first and second research aims we used a 2 (Repeated Measures: T0 vs. T1) × 2 (Independent Measures: Police vs. paramedics) mixed ANOVA with SDS as the dependent variable. The third aim was tested using mediation conducted using the SPSS PROCESS add-on (Hayes, 2013). The AQ and RIBS change scores (subtracting the T1 from T0), and self-affirmation VAS were entered as mediators. The final aim was to address using a mixed-methods approach; the Q4TE data were reported using descriptive statistics, and the qualitative data was analysed using Thematic Analysis (Braun & Clarke, 2006, 2013).

3. Results

3.1. Missing data

A total of 1,094 police and ambulance staff participated in ENHANcE I, and 238 viewed the ENHANcE II training video. From these, 97 participated in both ENHANcE I and II; however, 14 did not complete the SDS, leaving a final sample of 83 participants. We conducted an independent samples t-test to determine whether there were any differences in T0 stigma levels between those who did (n = 83) and did not (n = 1,011) complete the training and T1 assessment. We found no significant difference between the T0 only data (M = 3.01; SD = 0.52) and the matched data in terms of levels of stigma [t(1092) = −.02, p = .98; d = 0.002].

We also conducted a multiple regression to identify any patterns to the missingness of the data within the matched dataset [i.e. identifying predictors that distinguish those participants who did (n = 83) vs. did not (n = 14) complete the SDS]. We utilised a stepwise method, entering age, and dummy codes for the profession (police vs. ambulance), and gender (male vs. all other genders, and female vs. all other genders) as predictors. We found that being male significantly predicted data missingness [F(1, 95) = 6.20, p = .02; β = .25]. All other predictors were non-significant (all ps > .05).

3.2. Sample characteristics

The sample characteristics are reported in Table 1. We had a roughly equal split between police and ambulance staff. Participants tended to be female, White, from the UK, and without any management responsibilities. The results of the RIBS current behaviour scale revealed that most participants had some kind of personal contact with someone experiencing mental health problems, and the self-affirmation results demonstrate strong agreement that “It is important to ensure the safety and welfare of everyone I come into contact with as part of my role as a first responder”.

Table 1. Sample characteristics.

	M (SD) or n (%)
	Police staff only	Ambulance staff only	Total Sample
	n = 41	n = 42	N = 83
Demographics
Age	39.66 (8.48)	28.53 (7.69)	34.02 (9.80)
Gender
Male	21 (51.20)	15 (35.70)	36 (43.40)
Female	20 (48.80)	27 (64.30)	47 (56.60)
Ethnicity
White British	38 (92.70)	40 (95.20)	78 (94.00)
White Other	1 (2.40)	2 (4.80)	3 (3.60)
Other	2 (4.80)	0 (0)	2 (2.40)
Relationship status
Single	3 (7.30)	21 (50.00)	24 (28.90)
Married/civil partnership/cohabiting	37 (90.20)	19 (45.20)	56 (67.50)
Separated/divorced	1 (2.40)	2 (4.80)	3 (3.60)
Country of birth
United Kingdom	38 (92.70)	39 (92.90)	77 (92.80)
Elsewhere	3 (7.30)	3 (7.10)	6 (7.20)
Employment
Employment status
Employed	41 (100.00)	33 (78.60)	74 (89.20)
Student	0 (0)	9 (21.40)	9 (10.80)
Duration working as a first responder (years)	12.39 (8.70)	5.35 (5.29)	8.83 (7.97)
Management responsibilities
No management responsibilities	28 (68.30)	35 (83.30)	63 (75.90)
Some management responsibilities	13 (31.70)	7 (16.70)	20 (24.10)
RIBS current behaviour
Living with or ever lived with
Yes	27 (65.90)	21 (50.00)	48 (57.80)
No	11 (26.80)	19 (45.20)	30 (36.10)
Don’t know	3 (7.30)	2 (4.80)	5 (6.00)
Work with or ever worked with
Yes	35 (85.40)	29 (69.00)	64 (77.10)
No	0 (0)	7 (16.70)	7 (8.40)
Don’t know	6 (14.60)	6 (14.30)	12 (14.50)
Have or ever had a neighbour
Yes	15 (36.60)	12 (28.60)	27 (32.50)
No	4 (9.80)	10 (23.80)	14 (16.90)
Don’t know	22 (53.70)	20 (47.60)	42 (50.60)
Have or ever had a close friend
Yes	36 (87.80)	30 (71.40)	66 (79.50)
No	1 (2.40)	8 (19.00)	9 (10.80)
Don’t know	4 (9.80)	4 (9.50)	8 (9.60)
Self-affirmation
“It is important to ensure the safety and welfare of everyone I come into contact with as part of my role as a first responder”	9.68 (0.96)	9.19 (1.86)	9.43 (1.50)
Training video
Subtitles
With	28 (68.30)	24 (57.10)	52 (62.70)
Without	13 (31.70)	18 (42.90)	31 (37.30)
Sound
Sound on	20 (48.80)	32 (76.20)	52 (62.70)
Sound off	21 (51.20)	10 (23.80)	31 (37.30)
Pauses
Without pausing	29 (70.70)	40 (95.20)	69 (83.10)
With pauses	12 (29.30)	2 (4.80)	14 (16.90)
Number of pauses	2.25 (1.54)	4.00 (0)	2.50 (1.56)
Device
Phone	1 (2.40)	9 (21.40)	10 (12.00)
Tablet/iPad	0 (0)	12 (28.60)	12 (14.50)
Computer/laptop	40 (97.60)	21 (50.00)	61 (73.50)
Other
Days between T0 and T1 data collection	46.88 (90.41)	51.05 (98.86)	48.99 (94.23)
Qualitative sub-sample
Total n	9 (39.1)	14 (60.9)	23 (100.0)
Age	36.33 (7.79)	30.50 (10.19)	31.78 (9.59)
Gender
Female	4 (44.4)	10 (71.4)	14 (60.9)
Male	5 (55.6)	4 (28.6)	9 (39.1)
Ethnicity
White British	9 (100.0)	13 (92.9)	22 (95.7)
White Other	0 (0)	1 (7.1)	1 (4.3)
Employment status
Employed	9 (100.0)	12 (85.7)	21 (91.3)
Student	0 (0)	2 (14.3)	2 (8.7)
Duration working as a first responder (years)	9.67 (7.95)	6.50 (6.97)	7.74 (7.36)
Management responsibilities
No management responsibilities	7 (77.8)	11 (78.6)	18 (78.3)
Some management responsibilities	2 (22.2)	3 (21.4)	5 (21.7)

Note. All n = 83; T0: pre-training data collection; T1: post-training data collection; RIBS: Reported and Intended Behaviour Scale (Evans-Lacko et al., 2011)—items refer to current contact with people who have experienced mental health problems.

When asked about the conditions in which they viewed the training video, most participants watched the video with subtitles and were able to do so without interruptions. For ambulance staff, most-watched with sound; whereas for police there was a relatively even split between those who did or did not have the sound on. A computer or laptop was the most frequently used device to watch the video.

There was a lot of variation in the duration between the T0 and T1 data collection, with some participants completing both assessments on the same day, whereas for others there was almost a year between the two testing points. To control for this variance across our sample, we entered the time between testing as a covariate.

3.3. Parametric assumptions

Assessment of normality on the T0 and T1 SDS variables, found that the T1 SDS data significantly deviated from a normal distribution [D(83) = .12, p = .01], whereas the T0 SDS data did not [D(83) = .08, p = .20]. The T1 SDS data had a significant negative skew (z = 2.77, p < .01). After reverse-scoring the T0 and T1 SDS data and the using a log transformation (Field, 2013), the T1 skew was no longer significantly different from normal (z = 1.65, p > .05). The log transformed version of the SDS will be used for all subsequent analyses. There was no evidence of heterogeneity of variances [T0: F(1,81) = 0.45, p = .51; T1: F(1,81) = 0.31, p = .58].

3.4. Aim 1

After controlling for the duration between testing time points, there was a significant main effect of time (T0 vs. T1) on mental health stigma, as measured by the SDS [F(1,80) = 19.58, p < .001], reflecting a small effect size (d = 0.21; Cohen, 1988). Inspection of the descriptive statistics (Table 2) shows that there was an increase in the SDS scores from T0 to T1 indicating that levels of mental health stigma decreased after watching the training video.

Table 2. Descriptive statistics for the measure of stigma and mediators.

	Police				Paramedics				Total
	T0		T1		T0		T1		T0		T1
	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)	n	M (SD)
SDS	41	3.14 (0.58)	41	3.28 (0.57)	42	2.89 (0.47)	42	3.04 (0.52)	83	3.01 (0.54)	83	3.16 (0.55)
RIBS Intended	41	1.57 (0.78)	41	1.37 (0.59)	42	1.80 (0.61)	42	1.82 (0.92)	83	1.69 (0.70)	83	1.60 (0.80)
AQ Fear	41	2.02 (1.16)	41	1.82 (1.01)	42	2.52 (0.93)	42	2.46 (1.18)	83	2.27 (1.07)	83	2.15 (1.14)
AQ Sympathy	41	6.44 (1.35)	40	6.35 (1.42)	42	5.68 (1.34)	42	6.03 (1.45)	83	6.05 (1.39)	82	6.19 (1.44)

Note. M: mean; SD: standard deviation; SDS: Social Distance Scale (Link et al., 1987); RIBS: Reported and Intended Behaviour Scale (Evans-Lacko et al., 2011); AQ: Attribution Questionnaire (Corrigan et al., 2002).

3.5. Aim 2

After controlling for the time between testing points, there was no significant interaction between time (T0 vs. T1) and group (police vs. ambulance) on mental health stigma, as measured by the SDS [F(1,80) = 0.06, p = .81]. This result suggests that the efficacy of the training did not differ between police and ambulance staff.

3.6. Aim 3

To understand whether there was any shared variance across psychological mediators, changes in beliefs (AQ) and behavioural intentions (RIBS) towards those with mental health problems as well as the degree of self-affirmation were entered into a single mediation model with the pre- and post-training scores as the predictor and outcome variables, respectively.

Pre-training scores significantly predicted self-affirmation (β = −.29; p = .007), but not any other mediators: future behaviour intentions (β = −.03; p = .814), fear-related attitudes (β = −.00; p = .970), and sympathy-related attitudes (β = .08; p = .443). On the other hand, only future behavioural intentions predicted post-training scores (β = .17; p = .016), but not the others: fear-related attitudes (β = .05; p = .398), sympathy-related attitudes (β = −.07; p = .220), or self-affirmation (β = −.06; p = .525). We also checked 95% bias corrected confidence intervals with 1,000 bootstrapped resamples to see whether any indirect effect was significant. The effect sizes were small and all confidence intervals crossed zero, meaning none of the indirect effects were significant: through future behaviour intentions (β = −.00; CI = −.05, .03), fear-related attitudes (β = −.00; CI = −.02, .01), sympathy-related attitudes (β = −.01; CI = −.05, .00), and self-affirmation (β = .02; CI = −.03, .08).

3.7. Aim 4

The Q4TE was used to assess how satisfied first responders were with the training provided. Table 3 shows the results of this questionnaire. The majority of participants said they would recommend the training to their colleagues. For four of the QPTE items, average ratings leaned towards higher levels of satisfaction (score of ≥50); for the remaining two items the average score veered towards low satisfaction. The range scores and standard deviations show that perceptions of the training varied widely.

Table 3. Results of the Questionnaire for Professional Training Evaluation (Q4TE) (Grohmann & Kauffeld, 2013).

	N	M (SD) or n (%)	Range
Would you recommend the training to others working in the same profession as you?	75
Yes		52 (69.30)
No		10 (13.40)
Not sure		13 (17.30)
I will remember the training.	81	65.06 (25.70)	0–100
I enjoyed the training very much.	81	56.54 (27.76)	0–100
The training is very beneficial to my work.	80	64.50 (32.33)	0–100
Participation in this kind of training is very useful for my job.	80	70.75 (28.32)	10–100
After the training, I know substantially more about the training contents than before.	81	49.51 (32.40)	0–100
I learned a lot of new things in the training.	81	38.02 (28.21)	0–100

Note. Scale is from 0% (strongly disagree) to 100% (strongly agree); a higher score indicates greater satisfaction with the training.

The qualitative data can aid in further understanding these mixed opinions. Of the 83 participants in this study, 23 (27.71%) provided some qualitative feedback (see Table 1 for sub-sample characteristics). Using Thematic Analysis, we extracted three higher-order themes: (1) what does mental health mean?; (2) positive feedback; and (3) negative feedback. The final theme has two sub-themes: (3a) content; and (3b) delivery.

3.7.1. (1) What does mental health mean?

The training and the questionnaires used to evaluate its effectiveness did not refer to any specific mental health diagnoses and instead looked at the experience of mental health crises irrespective of diagnosis (see supplementary material for more information). First responders felt that “mental health” was too generic a concept, and that their perceptions, attitudes, and behaviours would vary depending on the person’s diagnosis:

Pp18: “I found the survey questions were difficult to answer, as ‘a person with mental health problems’ incorporates such a wide range of people, some of whom I might be happy to look after my children, some I wouldn't particularly want living next to me”.

Pp1: “A person with anxiety or depression displays symptoms very different to a person with paranoid schizophrenia who has not been taking [their] medication”.

3.7.2. (2) Positive feedback

Some participants reported finding the training helpful and that they had learned something:

Pp15: “The video was very well done and taught me the 4 step plan that can be used for when we are called to patients with a mental health illness”.

In particular, the testimonies from people who had lived experience of a mental health crisis and first responders were identified as particular impactful:

Pp7: “It was really beneficial to have people who have been through a crisis explain what they needed people to do at that time and explain that they're not necessarily present in the conversation even if it looks like they are. This makes the training definitely more memorable”.

3.7.3. (3) Negative feedback

Some participants reported negative feedback with regard to the training, including the content of the training video as well as the use of the brief, online delivery format.

3.7.3.1. (3a) Content

Several of the first responders reported that the training did not teach them anything new—they had either covered similar material as part of other training courses, or the training material did not cover anything that it wasn’t possible to learn by experience. Some of these first responders felt that the training would be more suited to new/less experienced staff members:

Pp4: “I have received this training before. I was happy to take part in the survey but don't feel I have gained any further knowledge from this”.

3.7.3.2. (3b) Delivery

Some participants felt that the length of the training video was too short to cover such a complex topic in sufficient detail: Pp3: “10-minute video not adequate training for a police officer”.

Pp17: “I don't feel that a 10 minute video will ever contain enough information to train people, to an extent where they should be giving any advice”.

Another delivery issue experienced by one first responder was technical issues caused by inadequate equipment to watch the training video:

Pp2: “I watched the subtitled version as my computer doesn't have speakers. The blue pop ups [subtitles] were very distracting to the reader, and the size and colour of the font made reading quite difficult”.

4. Discussion

The purpose of the present study was to evaluate a brief training video aimed at reducing mental health stigma amongst first responders, as well as understanding the mechanisms of change and consumer experience. We found that the training video produced a significant but small improvement in levels of mental health stigma and that these effects did not differ between professions (i.e., police and ambulance staff). The effects of the training video could not be explained by changes in mental health-related attitudes, behaviour, or degree of self-affirmation. Participants generally had a positive experience of viewing the training video, but several areas for improvement were identified.

We found a significant effect of our training video, and the equivalence of these effects between police and ambulance staff. Budget cuts have put first responders under immense pressure (Brown et al., 2019), meaning a lack of resources is a key barrier to implementing new training initiatives (Fleming & Wingrove, 2017). It is, therefore, encouraging that our findings suggest mental health stigma can be significantly improved using a brief, resource-light training video and that there is not necessarily a need for profession-specific training. Developing training that is both efficacious and not a demand on resources will aid wide-scale implementation.

However, looking beyond statistical significance, the overall effect was small, especially for a pre-post study. Our training video shows promise but requires further development. The way in which we pursue this development work however is unclear as we were unable to determine the mechanism of change. We predicted that improvements in mental health stigma would be explained by reduced prejudice and discrimination, as per the cognitive behavioural model of mental health stigma (Corrigan et al., 2005; Thornicroft et al., 2007), but this was not supported in our mediation analysis. One interpretation of this finding is that it calls into question the validity of the cognitive behavioural model (Corrigan et al., 2005; Thornicroft et al., 2007). Noteworthy though, is that we were unable to test whether the knowledge aspect of the cognitive behavioural model could explain the training effects due to issues with the psychometric validity of the MAKS (Evans-Lacko et al., 2010). To answer this question will require the development of a robust measure of mental health knowledge. Further research may be needed to assess the comprehensiveness of the cognitive behavioural model of mental health stigma. Alternatively, our results may represent a Type II error or our training may have been insufficient to bring about statistically significant changes in attitudes and behavioural intentions specifically. The small post-training improvements may instead be the product of other variables not included here.

4.1. Limitations

There are several methodological limitations that require consideration. Firstly, we used a pre-post design meaning we were unable to control for naturally occurring changes in mental health stigma, or changes attributable to events other than the intervention. Secondly, we did not include a follow-up assessment and therefore cannot make any claims regarding the longevity of the training effects. Thirdly, we measured mental health stigma using a questionnaire, meaning our data could be subject to a social desirability bias (Kopera et al., 2015); furthermore, we do not know whether the training had any real-world impact on first responders' practice when supporting people with mental health difficulties. Future studies should therefore use a between-groups design with follow-up assessments, using both implicit and explicit measures of stigma. Assessing the real-world impact of the training would present additional logistic challenges, but could be achieved using either observational methods or a proxy measure, such as the number of complaints or patient satisfaction [i.e., using the Friends and Family Test (Department of Health, 2013)].

We must also acknowledge that at T0, levels of mental health stigma were low, and attitudes were generally positive. The degree of change that could have been achieved by watching the training video was therefore limited (i.e. floor effect). In one respect, finding a significant pre-post result when stigma levels were already low provides further encouraging evidence for the training video and suggests that larger effects may be possible for those reporting higher mental health stigma. This floor effect may also indicate a sampling bias whereby we were unable to recruit those first responders with the highest stigma levels. Our qualitative findings imply this might be the case, as some participants noted that they did not learn anything new. Future studies may benefit from an inclusion/exclusion criterion that screens participants’ stigma levels with only those in need of training going on to receive it.

4.2. Implications

In addition to the methodological improvements discussed above, we must also reflect on constructive criticism given in relation to the training. Specifically, we must adapt our training and assessments to reflect the heterogeneity of mental health problems, for example, using case vignettes to assess diagnosis-specific levels of stigma (Angermeyer & Schomerus, 2017). This feedback is consistent with findings that levels of stigma vary in relation to diagnosis (Ben-Zeev et al., 2010; Caldwell & Jorm, 2001; Corrigan, 2007; Lee et al., 2016). A more immediate implication is the support for a brief video-based training format to improve mental health stigma amongst first responders. Although some participants mentioned the training was too short, in light of service constraints (Brown et al., 2019) and our promising evidence, there is little justification for increasing the training duration.

4.3. Conclusion

We found a small but significant pre-post effect of a brief training video on mental health stigma amongst police and ambulance staff, with no difference in effects between professions. Neither changes in attitudes nor intended behaviours could explain the training effects. The findings provide encouraging initial support for the efficacy of this resource-light training. However, improvements to the study design and training video are required.

Acknowledgements

Thank you to the lived experience advisory group, the police and ambulance staff that helped to develop the ENHANcE training, with special thanks to those who agreed to appear in the video. We would also like to acknowledge the work of Imrana Choudhry and Eleanna Karakousi-Patinioti who assisted with data collection. This paper is dedicated to the memory of Liv—thank you for your hard work and dedication to improving the provision of emergency mental health care, and for sharing your story in the video.

Disclosure statement

No potential competing interest was reported by the authors.

Data availability statement

The data is available from the corresponding author upon reasonable request.

Additional information

Funding

This work was supported by the Economic and Social Research Council (ESRC) Impact Acceleration Fund (reference ES/M500495/1).