Simulation training in suicide risk assessment and intervention: a systematic review and meta-analysis

ABSTRACT

Purpose. Suicide is a major cause of preventable death worldwide. Adequate training in risk assessment and intervention is key to suicide prevention. The use of simulation (role plays, simulated patients, virtual reality…) for practical training is a promising tool in mental health. The purpose of this study was to assess the effectiveness of simulation training in suicide risk assessment and intervention for healthcare professionals and gatekeepers. Methods. We conducted a systematic review in Medline and PsycINFO up to 31 July 2021 of randomized controlled trials (RCTs), non-randomized controlled trials, and pre/post-test studies. RCTs were furthermore included in a meta-analysis. We assessed the methodological quality of all studies with the Medical Education Research Study Quality Instrument, and the Cochrane Risk of Bias tool 2.0 for RCTs. Primary outcomes were changes in Kirkpatrick criteria: attitudes, skills, knowledge, behaviors, and patient outcomes. Results. We included 96 articles representing 43,656 participants. Most pre/post-test (n = 65) and non-randomized controlled (n = 14) studies showed significant improvement in attitudes, skills, knowledge, and behaviors. The meta-analysis of 11 RCTs showed positive changes in attitudes immediately after training and at 2–4 months post-training; in self-perceived skills at 6 months post-training; but not in factual knowledge. Studies assessing benefits for patients are still limited. Conclusions. The heterogeneity of methodological designs, interventions, and trained populations combined with a limited number of RCTs and studies on patients’ outcomes limit the strength of the evidence. However, preliminary findings suggest that simulation is promising for practical training in suicidal crisis intervention and should be further studied.

KEYWORDS:

• Suicide
• suicidal ideation
• teaching
• simulation training
• Mental disorders

Introduction

Background

More than 700 000 people die from suicide worldwide each year, and ten to twenty times more people attempt suicide [1]. Suicide prevention relies on various levels of collective and individual interventions [2] targeting at risk individuals (e.g., training healthcare professionals in the treatment of depression and suicidal patients), including restricting access to suicide means, and school-based programs, among others [3]. Another axis of prevention relies on gatekeepers, i.e., individuals who are not healthcare professionals but may interact with those at risk of suicide and help them seek treatment [4]. Recent studies report a significant decrease in suicide attempts in populations where gatekeepers were trained [5]. Meta-analyses have confirmed the overall effectiveness of preventative interventions on both completed and attempted suicide and a positive effect of combined interventions [6].

Suicide prevention is impeded by false and sometimes detrimental ideas even among healthcare professionals; for example, that suicide cannot be prevented or that asking about suicide may plant suicidal ideas in the patient’s mind [7]. Negative attitudes and stigma toward suicidal patients are high even among healthcare professionals [8]. Yet, it has been shown that a high number of people had met a physician prior to their death by suicide, up to 10% on the day of their death and 60% in the previous month [9]. These numbers highlight dramatically unmet needs as well as a critical opportunity for intervention in suicide prevention.

Fortunately, studies suggest that adequate training may address these needs. In a study including 196 healthcare professionals, 60% reported a lack of intervention skills and the need for specialized training in suicide [10]. Without adequate training, clinicians are fearful to ask about suicidal ideation [11] and mental healthcare providers are more avoidant, uncomfortable, and anxious facing a suicidal patient [12]. Reluctance to ask about suicide and perceived lack of skills among general practitioners were associated with lower frequency of suicide risk [7]. Conversely, previous training on suicide risk assessment was associated with more screening for suicidal ideations among depressed patients [7], improved knowledge, attitudes and confidence to deal with suicide, and reduced stigma and taboo among healthcare professionals [13,14]. Furthermore, several studies have shown a significant decrease in suicide rates when healthcare professionals received training on suicide prevention and depression [15–17].

An important question is how to best train both health professionals and gatekeepers for suicide risk assessment and intervention. Indeed, interviewing someone in a suicidal crisis is not easy [18]. The suicidal person may feel ashamed of their suicidal ideas, may fear the consequences of disclosing such ideas (hospitalization, stigma, children custody, etc.), or may feel professionals are not competent enough to help them. In these circumstances, information focusing on factual knowledge about suicidal behaviors (e.g., epidemiology and risk factors) as done in many training programs (whether initial or continuing) may be important but very insufficient. To enhance skills and confidence to carry out suicidal risk assessments and implement crisis interventions, it is crucial to provide more practical training alongside. However, only a few studies have assessed the effectiveness of suicide training according to the educational format and differences have been shown between, for example, lecture and simulation-based-education in terms of effective learning [19].

First developed in the sixties, simulation-based-education is defined as a set of ‘techniques that creates a situation or environment to allow persons to experience a representation of a real event for the purpose of practice, learning, evaluation, testing, or to gain understanding of systems or human actions’ [20]. In the case of health assessment, the experiential learning component provides the opportunity to interact with simulated patient, to reflect emotionally and practically, before embedding learning through facilitator-led peer discussions in debriefing. The debriefing session is about creating a caring and trusting environment to think about learning objectives and identify areas of improvement [21]. The benefits of medical simulation-based-education are widely recognized in terms of reducing medical errors, improving medical practices and patient safety, as well as learner satisfaction and engagement [22–25].

In mental health, several techniques have been developed to recreate clinical environments (Table 1) demonstrating the considerable potential for developing simulation practice further. The development of simulation-based-education in mental health was late compared to other specialties due to several controversies while calls have been made to further integrate this modality into mental health education [26]. Authenticity of psychiatric simulations have been questioned, alongside the ability of simulated patients to correctly portray the complexity of mental disorders, or the nature of empathy toward a ‘false’ patient compared to the singular experience of each person with a mental disorder [27–29]. Moreover, some studies have reported induction of psychological symptoms for simulated patients without previous training, while involvement of real patients can be more complex and hazardous than in other specialties. However, an increasing number of experimentations over the past decade and recent meta-analysis have supported the use of simulation-based-education in mental health for medical doctors [30] and nurses [29] in a wide range of psychiatric disorders.

Table 1. Definitions.

Technique	Definition	Applications
Human Simulation:	A ‘methodology that involves human role players interacting with learners in a wide range of experiential learning and assessment contexts’ [85]
Role Play (RP)	Role-players are ‘asked to be someone quite different from themselves and, with little or no preparation, perform in front of peers and teachers’ [28]	Different roles played (helper or patient) in various psychiatric scenarios such as depression [87]
Simulated Patient (SP)	‘A person who has been carefully coached to simulate an actual patient so accurately that the simulation cannot be detected by a skilled clinician. In performing the simulation, the SP presents the gestalt of the patient being simulated; not just the history, but the body language, the physical findings, and the emotional and personality characteristics as well’ [85]	Simulated Patients are actors with intellectual disabilities who can teach their own experience [26]
Standardized Patient (StP)	The standardized patient is trained, with a replicable scenario to simulate history of a patient and to replicate the patient’s clinical signs, personality, body language and emotions.	Often used for the assessment of medical students, clinical clerks, interns, and resident (OSCE) [88]
Virtual Reality (VR)	« A computer-generated three-dimensional environment that gives an immersion effect » [20]
Virtual Environment and Virtual Patient	Virtual reality simulation with a psychiatric environment and a patient avatar with artificial intelligence simulating a mental illness	Educational tool with a positive impact on confidence and communication [89]. It is easily accessible, replicable and reusable, and it allows to train skills in a safe environment.
Voice Simulation	‘Use of sounds and voice through an electronic medium to portray the sounds encountered by a schizophrenic patient’ [86]	The voice simulation gives students the opportunity to hear the patients’ auditory hallucinations. To make the experience more realistic, students can also be challenged with cognitive tasks while listening to the hallucinations. This allows them to better understand patients, empathize with them and reduce the stigma against them [90].
Manikins (M)	‘Full or partial body simulators that can have varying levels of physiologic function and fidelity’ [20]	Mostly used in medical specialties with technical gestures such as anaesthesia and intensive care, gynecology-obstetrics, surgery … In psychiatry, it can be used to simulate electro-convulsivo-therapy [91]
Objective Structured Clinical Examination (OSCE)	« A station or series of stations designed to assess performance competency in individual clinical or other professional skills. Learners are evaluated via direct observation, checklists, learner presentation, or written follow-up exercises. The examinations may be formative and offer feedback or summative and be used for making high stakes educational decisions [20,85]	The University of Toronto prepared an exhaustive guide to implement different psychiatric scenarios [88]. Updates have been made since [92].

To our knowledge, no systematic review or meta-analysis has been conducted on simulation-based-education relating to suicide risk assessment and crisis intervention training. Here, we reviewed studies conducted with both healthcare professionals and gatekeepers, considering their different but complementary roles and training background in suicide prevention.

Aims

This study aims to assess the effectiveness, in terms of the main Kirkpatrick criteria [32], of simulation training in suicide risk assessment and crisis intervention for healthcare professionals and gatekeepers.

Design

This systematic review and meta-analysis adheres to the principles of the PRISMA (Preferred Reporting Items for Systematic review and Meta-Analysis) statement [33]. The protocol is registered under PROSPERO: CRD42020196136 (Supplemental Digital Appendix 1).

Methods

Search strategy

We searched PubMed and PsycINFO databases for all relevant English and French language studies published from inception to July 31^st, 2021. A librarian specialized in mental health at Sainte-Anne hospital library developed the search algorithm (Supplemental Digital Appendix 2).

Eligibility criteria and selection process

We included all single group and single intervention pre-post-test (PPT) studies, non-randomized controlled trials (non-RCTs), and randomized controlled trials (RCTs) which used and assessed simulation training (Supplemental Digital Appendix 3).

The primary outcomes were based on the main Kirkpatrick criteria [32], namely changes in attitudes and knowledge, skills (levels 2a and 2b, that is, what trainees learned), behaviors (level 3, that is, what trainees do differently in their clinical practice) and patient outcomes (level 4, that is, what impact the program has had on public health e.g., reduction of attempted suicides) [34]. We removed the first level of Kirkpatrick criteria (‘reaction of learners’, that is, satisfaction of learners) because almost all studies used post-test only assessment.

Two authors (OR and FJ) independently screened abstracts and retrieved full-text articles that met inclusion criteria using Covidence (Covidence, Melbourne, VIC, Australia). We resolved disagreements by reaching consensus with a third reviewer (M-AP). The corresponding authors were contacted by email if some doubts remained about eligibility or if full texts were not available.

Quality appraisal

All studies had methodological quality assessed using the Medical Education Research Study Quality Instrument (MERSQI) [35]. For RCTs, we used the Risk of Bias tool 2.0 (RoB 2.0; Cochrane) [36]. We also reviewed key features related to effective learning in simulation training (feedback, multiple-learning strategies, controlled environment) [37,38].

Data extraction

Two authors (OR, MAP) then conducted a standardized data extraction on a Microsoft® Excel form (Microsoft Corp., Redmond, WA, USA).

Data analysis

We used RevMan Version 5.4.1 (Cochrane, London, UK) for meta-analysis.

To limit biases, we performed meta-analyses only for RCTs [39], while a narrative analysis was done for PPT and non-RCTs studies on the basis of their individual statistical significance. For each outcome, we compared separately simulation to active (another type of training such as a lecture course) or inactive (no further training, e.g., trainers who have no lectures, no problem-based learning or any other kind of mental health training) comparators at three time-points (Supplemental Digital Appendix 4).

Statistical significance was set a priori at p < 0.05. Clinical significance was based on Cohen’s effect size classification (>0.8: large effect size, 0.5–0.8: medium effect size, <0.5: small effect size) [40].

We assessed the quality of evidence with the Grading of Recommendations Assessment, Development and Evaluation system (GRADE) [41] (Supplemental Digital Appendix 12).

Results

Search results

The search strategy identified 5,094 articles (Figure 1). Of these, 429 were selected for full-text review. Among them, 96 studies fulfilled the detailed eligibility criteria: 17 RCTs (n = 15,623 participants), 14 non-RCTs (n = 2,546 participants), 65 PPT (n = 25,487) representing a total of 43,656 participants (Table 2).

Figure 1. Flow chart.

Table 2. Summary of all studies selected.

Study characteristics	Number of studies (n = 96)	Number of participants (n = 43656)
Study design Randomized controlled trial (RCT) Controlled trial not randomized (non-RCT) Pre/post-test single group (PPT)	17 14 65	15623 2546 25487
Simulation format Role Plays Standardized Patients Virtual Reality Mannikins	77 12 7 1	29207 1137 13312 12
Other learning strategies: Simulation alone Other learning strategies without blended learning Other learning strategies with blended learning (non-simulation activities included in learning outcome)	7 9 80	13271 1457 28928
Type of suicide training Gatekeeper Healthcare	56 40	36122 7534
Participants* Psychiatrists/psychiatry residents General Practitioners/GP residents Other medical specialty doctors/residents Nurses/nursing students In Psychiatry services In other medical specialty services Medical students (specialty not yet chosen) Pharmacists/pharmacy students Psychologists/psychology students Social workers/social work students Counselors/counselor trainees Teachers/school staff University students/Campus staff Hotline workers Police officers Mixed in mental health services Mixed in medical services (other than mental health services) Mixed with many different professions Others (prison staff, manufactory employees, …)	1 4 3 0 6 9 3 2 1 2 9 15 2 1 10 9 9 10
Outcome measured Attitudes Skills Knowledge Behaviors Mental health outcomes	75 34 48 22 6	18271 13551 10490 15303 14342

We included available quantitative data of 11 RCTs for meta-analysis.

Study characteristics

Characteristics of each study are summarized in Table 2 and detailed in Supplemental Digital Appendices 5, 6, 7. An overview of the scales used to assess each outcome is provided in Supplemental Digital Appendix 8.

Most studies were conducted in the USA (n = 47, 49.0%), and United Kingdom (n = 10, 10.4%). The origin by country of the different studies is reported in the Supplemental Digital Appendix 9. The main type of simulation was roleplay (n = 77 studies). Study quality evaluated by MERSQI scores reached an average of 12.13 out of 18.

Medical Education Research Study Quality (MERSQI) scores are reported on Supplemental Digital Appendix 11.

Risk of bias summary (ROB 2.0) for RCTs is reported on Supplemental Digital Appendix 10. GRADE system found that the quality of evidence is very low due to a high risk of bias in most of studies (Supplemental Digital Appendix 12).

Findings by Kirkpatrick’s criteria

Knowledge

Seven RCTs (n = 1,646) assessed knowledge. Forest-plots of meta-analyses analysis are reported in Figure 2. The number of studies included in RCTs did not allow the subgroup analysis according to the populations studied (gatekeepers or healthcare professionals). At immediate post-test to one-month follow-up, no significant difference was found when simulation was compared to an active comparator (two studies [42,43] SMD =-0.10, 95% CI −0.39-0.19) or to an inactive comparator (two studies [44,45]; SMD = 1.14, 95% CI −0.48-2.75).

Figure 2. Forest plots on knowledge.

Four non-RCTs assessed knowledge (n = 846). All showed significant improvement in knowledge compared to control condition. Three of them used declarative knowledge questionnaires [46–48] and one assessed perceived knowledge of facts and information about suicide using self-report [49].

Thirty-si PPT studies assessed knowledge (n = 1,275). Twenty-eight studies showed significant improvement of knowledge outcomes from pre- to post-test. Among these studies, one did not report a comparison test for knowledge [50]. Eleven studies evaluated knowledge with a longer follow-up, from 4 weeks to 1 year, but mostly at 3 months. All but two showed significant improvement of knowledge from pre-test to follow up.

Attitudes

Attitudes comprise ways of approaching patients with suicidal ideas, beliefs, self-efficacy, confidence, and sense of preparedness.

Twelve RCTs (n = 2,000 participants) assessed attitudes. Forest-plots of meta-analyses and subgroups analysis are reported in Figure 3. At immediate post-test to one-month follow-up, a significant large effect size was found comparing simulation training to an active comparator (five studies [42,43,51–53:] Standardized Mean Difference (SMD) = 0.49, 95% confidence interval (CI) 0.21–0.77) with moderate heterogeneity (I [2] = 46%). No significant difference was found when comparing simulation training to an inactive comparator (four studies [44,45,54,55] SMD = 0.27, 95% CI −0.46-1.01). At 2 to 4-month follow-up, no significant difference was found when comparing simulation training to an active comparator (two studies [42,53] SMD = 0.21, 95% CI −0.08-0.50). However, a significant large effect size was found when comparing simulation training to an inactive comparator (two studies [55,56] SMD = 0.65, 95% CI 0.43–0.87) with low heterogeneity (I [2] = 0%).

Figure 3. Forest plots on attitudes.

Subgroup analysis performed according to the populations trained (gatekeepers or health professionals) at immediate post-test to one-month follow-up showed the same results. Including only studies with gatekeepers, a significant large effect size was found comparing simulation training to an active comparator (three studies [42,51,52] Standardized Mean Difference (SMD) = 0.47, 95% confidence interval (CI) 0.01–0.92) with moderate heterogeneity (I [2] = 68%). Including only studies with healthcare professionals, a significantly large effect size was found comparing simulation training to an active comparator (two studies [43,53] Standardized Mean Difference (SMD) = 0.52, 95% confidence interval (CI) 0.14–0.89) with low heterogeneity (I [2] = 11%). Including only studies with gatekeepers, no significant difference was found when comparing simulation training to an inactive comparator (three studies [44,45,55] SMD = 0.41, 95% CI −0.52-1.34).

Thirteen non-RCT assessed attitudes (n = 2,192). Ten studies showed significant improvement in attitude compared to control condition. Four out of seven [13,46,47,49,57–59] non-RCT evaluating attitudes toward patients with suicidal ideas showed significant improvement, maintained at 3 months for one study [46]. Three out of four [13,48,59,60] non-RCT which evaluated self-confidence showed significant improvement, maintained at 2 months in one study [59] and 5 to 10 months in the other study [48]. Six out of six studies [49,57,61–64] which evaluated self-efficacy showed significant improvement: four studies showed a significant increase compared to an inactive comparator and two studies compared to an active comparator (brief presentations [61] and other didactic learning [62]). Most were gatekeeper trainings, and two were healthcare professional trainings [57,63].

Forty-eight PPT studies assessed attitudes (n = 14,051). Forty-two of them showed significant improvement of attitude outcomes from pre- to post-test. Among them, 19 studies proved still significant improvement after a follow-up of 1 month to 1 year.

Skills

Eight RCTs (n = 1,097) assessed skills. Forest-plots of meta-analyses and subgroups analysis are reported in Figure 4. At immediate post-test to one-month follow-up, no significant difference was found when compared to either an active comparator (four studies [51–53,65] SMD = 0.03, 95% CI −0.43-0.49) or an inactive comparator (two studies [44,45] SMD = 0.91, 95% CI −0.89-2.70). Subgroup analysis performed according to the populations trained (gatekeepers only) at immediate post-test to one-month follow-up showed the same results: no significant difference was found when compared to an active comparator (three studies [51,52,65] SMD = 0.22, 95% CI −0.17-0.61). At six-month follow-up, a significant large effect size was found when comparing simulation training to an active comparator (two studies [51,52] SMD = 0.40, 95% CI 0.04–0.76) with low heterogeneity (I [2] = 0%). The first study evaluated the intent to intervene [51] and the second one [52] assessed skills on the Suicide Intervention Response Inventory version 2 (SIRI-2) [66].

Figure 4. Forest plots on skills.

Six non-RCT studies assessed skills (n = 625). Four [59–61,64] studies showed significant improvement in skills compared to control condition while two studies [46,63] showed no differences between the two conditions. Four studies [46,59,60,64] used the SIRI-2 to evaluate skills, one used three items that assessed the likelihood that participants will use the suicide-specific skills taught in the training [61], and one evaluated five self-rated specific communication skill items [54].

Thirteen PPT studies assessed skills (n = 10,596). Eleven studies showed significant improvement in skills from pre- to post-test. Five studies [4,67–70] evaluated skills at 1-month follow-up, 3-months follow-up and 6-months follow-up, and showed significant improvement in skills from pre-test to follow-up.

Behaviors

Six RCTs (n = 2,447) assessed changes in behaviors but the variety of measures did not allow meta-analyses. Three studies found significant improvement of behaviors for simulation training compared to control condition [44,55,71]. Coleman et al. (2019) showed that college students were more likely to refer peers for help 2 months after training using virtual reality compared to no further training (Intervention: mean = 0.88 (Standard Deviation = 1.4); Inactive comparator: 0.30 (0.54), p < 0.05). Gould et al. (2013) showed that hotline counselors were more likely to increase calls duration (27.1 (18.7) vs. 24.4 (17.1), p < 0.01) and identify signs of suicide risk (8.4 (3.9) vs. 7.2 (3.5), p < 0.0001) at six to 18 months after a training using role-plays compared to no further training. However, this study found no significant increase in asking about suicide plans. Bazley & Pakenham (2019) showed significant increases in preventative behaviors at one-month for the intervention condition (17.66 (3.21)) compared to an inactive comparator (16.39 (2.28), p = 0.00)).

Six non-RCT studies assessed behaviors (n = 1,585). Four studies [17,48,58,62] showed significant improvement in behaviors compared to control condition while the other two [49,59] showed no significant differences between the two conditions. Chauliac et al. (2016) compared role-plays to no further training for 106 caregivers in 310 nursing homes for 1 year [58]. They found a significant improvement in management of suicidal crises compared to inactive comparator, with more psychological therapies and interventions (28 (77.8) vs. 15 (45.5), p < 0.006), and more contracts made with patients (4 (11.1) vs. 0 (0), p < 0.05). They also found a significant improvement for almost all measures of a recommendation list that could influence suicide prevention (suicide risk factors, assessment tools, restricted access to means). However, they found no difference for attitudes and detection of suicidal crises. Coleman and Del Quest (2015) highlighted a significantly increased frequency of asking about suicide at six-months follow-up (3.8 (0.7) vs. 2.6 (1.1), p < 0.01) but no difference on number of youths referred (4.6 (0.7) vs. 3.8 (1.2), p = 0.13) [62]. Fallucco et al. (2012) showed a significant improvement in using a depression screening tool (50% vs. 19%; p = 0.001) and having diagnosed at least one adolescent with depression in the past 3 months (96% vs. 78%; p = 0.013) [48]. Roskar et al. (2010) found a significant increase in antidepressants prescription rates (mean difference = 1.05, bias-corrected and accelerated 95% CI = 0.47, 1.80, bias = 0.01, Standard error = 0.36) [17].

Eight PPT studies assessed behaviors (n = 11,155). Three studies showed significant improvement of behaviors [70,72,73] from pre-test to follow-up, while four studies showed no significant differences from pre-test to follow-up [31,74–76] and one did not provide comparative results before and after training [77].

Among studies that found positive behavior improvements, Rallis et al. (2018) studied youth services staff’ behaviors at 3 months from a training using role-plays. They collected the number of suicidal students identified and the number of suicidal students referred for help reported by participants at baseline and three-month follow-up. They found a significant increase in the number of students referred (Pre-test: 0.14 (0.42); three-months: 0.31 (0.60) p < 0.002) but not in those identified (Pre-test: 0.23 (0.60); three-months: 0.33 (0.61) p < 0.110).

Ewell Foster et al. (2017) evaluated behavioral change in participants following a training using role-plays. At 6 to 9 months post-training, they found a significant increase in the identification of at-risk youths (Pre-test: 1.04 (1.07); follow-up: 1.43 (1.51) p = 0.000), in frequency of asking about suicide (Pre-test: 3.11 (1.40); follow-up: M 3.56 (1.32) p = 0.000), and in helping behaviors (Pre-test: 3.76 (1.35); follow-up: 3.78 (1.53) p = 0.02). However, they found no significant difference in frequency of referring youth (Pre-test: 1.00 (1.29); follow-up: 1.76 (1.97), ns).

Patient outcomes

Three RCTs (n = 13,401) assessed patient outcomes, two found significant results [71,78], while one did not show significant differences between the two conditions [1,79]°-. Wasserman et al. (2015) compared the incidences of suicide attempts and severe suicidal ideation among students after school staff were trained either with role-plays or with posters [78]. There was no difference at three-months post-training, but a significant decrease with simulation at 12-months post-training of the incidence of suicide attempts (Odd Ratio (OR) = 0.45, 95% CI 0.24–0.85, p < 0.014) and of severe suicidal ideation (OR 0.50, 95% CI 0.27–0.92, p < 0.025). Gould et al. (2013) evaluated the effects of a suicide hotline counselors’ training using role-plays, on callers’ behavioral changes at 6-, 12- and 18-months post-training [71]. They found that callers felt significantly less depressed, less overwhelmed, less suicidal, and more hopeful during the call compared to inactive comparator. Three months after a training including role-plays, De Beurs et al. (2016) did not find an improvement in the frequency of self-reported suicide attempts and satisfaction of patients about treatment and relationship with therapist [79].

One non-RCT [17] assessed patient outcomes (n = 354). It evaluated the number of suicides in regions where general practitioners followed or not a course including role-plays. No significant differences between groups were found.

Two PPT studies assessed patient outcomes (n = 587). The first one [80] only reported descriptive percentages without comparison test. However, results suggest an increase in the detection of suicidal ideation from 14% to 21% in patients presenting to the emergency department studied within 4 years of the implementation of the training. Medical staff reported feeling more comfortable questioning suicidal ideation and looking for protective and risk factors. The second one [81] found no significant difference from pre- to post-test on annual suicide rates after having trained healthcare professionals with role-plays.

Discussion

This study systematically reviewed the literature on the use of simulation-based-education for suicide risk assessment and intervention training in order to assess its effectiveness in healthcare professionals or gatekeepers involved in suicide prevention. Results from RCTs, non-RCTs and PPT studies suggest potential benefits in the use of simulation, notably in terms of short-term changes in attitudes (immediate to four-month post-training), and to some extent in improving skills, knowledge, and behaviors. However, only six studies assess changes in patients’ outcomes following simulation training to date and just two of them found a significant improvement in these criteria. It seems too few to draw any strong conclusion, whereas this criterion is precisely the one for which simulation-based education is provided, i.e., a concrete improvement in terms of public health. For example, these studies looked at the number of patients who had suicidal ideation, who had attempted suicide or the rate of completed suicides.

Change in attitudes is critical to improve the quality of relationship, prevent negative countertransference, and subsequently enhance the quality of suicide risk assessment and intervention. Consistent with previous meta-analyses on mental health simulation [29,30], a large effect on attitudes was found immediately at the end of training for simulation as compared to an active comparator, and at two- to four-months post-training as compared to an inactive comparator. The lack of significant difference at immediate post-test in the comparison with an inactive comparator may be related to the weakness of the simulation design in two out of four studies, which used role-plays with no details reported about dedicated time for roleplays among many other learning strategies [44,45]. Conversely, the strength of active comparators in studies assessing attitudes at two- to four-months follow-up may explain the absence of statistical significance: one study used a one-hour lecture with video, booklets and question-and-answer discussion period based on Question-Persuade-Refer training [42] and the other used a lecture [53]. In addition to RCTs, 10 out of 13 non-RCTs (including attitude toward suicidal patients, self-confidence, and self-efficacy) and 42 out of 48 PPT studies reporting statistical significance on attitudes provide further support on the effectiveness of simulation-based education, even if we cannot exclude natural learning effect for PPTs. One clear limitation of attitude measures in the reported studies is the reliance on subjective assessment.

Beyond possible lack of temporal distance for reporting effects on skills at immediate post-test, time devoted to role-play among the relevant studies was either less important, or not specified among several other educational approaches as seen in Suzuki et al. (2014) where role-play is included among lecture, video-modeling, and discussions, with time devoted unspecified. Improvement of skills found at six-months post-training compared to an active comparator but not at immediate post-training may be explained by the delayed effects of training on skills, notably regular personal reinforcement from practicing the skills learned during the training, engaging with Kolb’s Experiential Learning Cycle introduced through simulation. However, of the two studies included in the meta-analysis at 6-month follow-up, one study showed significant improvement of skills at immediate post-training [82]. In this study, role-plays took a large part of the training with each participant taking part into role-plays and general feedback. In addition to RCTs, four positive studies out of five non-RCTs and eleven positive PPT studies tend to support an effect of simulation-based-education on skills. Studies investigating longer follow-ups and using an external assessment of skills are needed.

No significant difference was found in meta-analyses for a change in knowledge following simulation. Beyond the few numbers of studies, another explanation is that simulation-based-education, through experience and reflection, targets more the acquisition of behaviors, skills, and attitudes than theoretical knowledge. This is consistent with a previous review on simulation in psychiatry for nurses [29]. However, 4 non-RCT and 28 out of 36 PPT studies do report significant results. As a matter of fact, many training programs include teaching basic factual knowledge.

Most studies focused on attitudes, skills, and knowledge. Assessment of changes in the learner’s objective behaviors, and of increased patient outcomes are much more limited, possibly due to methodological complexities. Three RCTs showed objective behavioral changes including an improved management of suicidal crises, an increased frequency of asking about suicidal ideas, an increased number of diagnosed depressions and increased rates of antidepressant prescription. Moreover, four out of six non-RCTs also showed positive results, which is promising.

Regarding patient outcomes, two RCTs showed significant positive effects following the training of school staff and of suicide hotline counselors. While these studies are more complex to manage, they are necessary. Simulation training can be a complex training modality to implement, requiring writing authentic scenarios, training facilitators and actors. When working with actors, additional costs, and ethical considerations present. Virtual reality necessitates programming and equipment and software to implement. Clear and robust benefits of simulation over other ‘simpler’ training modalities (such as lectures or videos) are required and while some are evident, those relating to behaviors and patient outcomes are less clear.

Strength and limitations

This is the first systematic review and meta-analysis on simulation training in suicide prevention, providing encouraging results alongside certain limitations. First, despite careful research, some studies may have been missed. Second, only RCTs were included in meta-analyses in order to raise the level of confidence, while results from non-RCTS and PPTs were less detailed. However, only 17 RCTs were found. Six RCTs could not be included in meta-analysis because of different data collection time points, comparators, outcomes measured and missing data. We kept only post-test results to homogeneous measures. Third, because of the limited number of studies, sensitivity analysis, funnel plots, and Egger tests could not be performed given the lack of statistical power [83] which undermined the strength of our conclusions. However, we were able to perform some subgroup analysis when possible, distinguishing trainings targeted for gatekeepers from that for healthcare professionals, thus reaching the same conclusions suggested by the meta-analyses. In the future, variables such as professional activities (general practitioner, psychiatrist, nurse, psychologist), the stage of education (initial versus continuing) or the type of simulation should be analyzed. Fourth, GRADE system found the quality of evidence is very low due to a high risk of bias in most of studies.

Heterogeneity was high across studies. Participants came from very different social and professional backgrounds, and simulation pedagogies and methods varied greatly across studies. In addition, simulation techniques were frequently a small part of the overall training offered, as educators’ attempts to strengthen training programs through the use of simulation. However, this made it challenging to measure the specific effect of simulation compared to other learning strategies, and their interaction. Finally, few studies evaluated the impact in terms of public health (number of suicides and suicidal ideations for example).

Implications

Regarding methodological implications, more research, with an adequate methodological design, is needed to robustly evaluate the benefits of simulation-based education for suicide prevention training, including RCTs, long-term follow-up, and measures of the highest levels of the Kirkpatrick’s criteria, namely change in participants’ behaviors and changes in patient outcomes. Given the few studies that have investigated the direct implications for patients, it would seem particularly interesting, for example, to study the number of suicide attempts and completed suicides in regions where general practitioners or psychiatrists have been trained with simulation-based education compared to a region where general practitioners or psychiatrists have been trained with more conventional training (e.g., lectures). Guidance and recommendations on study design to investigate the effectiveness of simulation training would be helpful.

Regarding educational implication, our encouraging results suggest the benefit to implement simulation-based education for all health students as is done for somatic first aids. Moreover, targeting people most likely to be close with person with suicidal idea may be an efficient public health approach.

As suicide prevention rests on the combined efforts of many types of contributors, we included here all types of participants without any limit of professional category. However, the complexity of training should be adapted to the educational level of trainees – a basic requirement in education – and the assessment of effectiveness needs to be adjusted accordingly. For example, as role-plays are more effective with unexperienced people, are well-trained standardized patients required with an advanced mental health professional? Some studies [30,84] suggested that even with experienced professionals, role-plays positively influence attitudes towards suicide. Simulation training may be complementary to other classical training techniques such as lectures, as theoretical knowledge remains necessary (including epidemiology or risk factors). Simulation allows the acquisition of ‘know-how’, experience, and practical skills, which is essential in mental health, notably for crisis situations where training with real patients might not be appropriate.

Conclusion

The present study suggests that simulation-based training may be effective to change attitudes and skills for suicide risk assessment and crisis intervention, with promising preliminary results regarding changes in behaviors and patients’ outcomes. However, numerous limitations must be acknowledged, and many challenges remain. More research of higher methodological quality must be developed.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/10872981.2023.2199469.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.