Staff Perceptions of Facilitators and Barriers to the Use of a Short- Term Risk Assessment Instrument in Forensic Psychiatry

ABSTRACT

Prospective adverse events within forensic settings should be assessed using structured risk assessment instruments. Our aim was to identify the barriers and facilitators of a structured instrument for assessment of short-term risk within inpatient forensic psychiatric care. The instrument was piloted at a forensic psychiatric clinic. Three focus group interviews were conducted with staff. Content analysis revealed three main categories of barriers and facilitators for clinical use: implementation object, context, and users. Complexity of the instrument, insufficient continuous training and support, difficulties retrieving assessments on wards, and insecurity about translating assessments into actions were perceived barriers to clinical use. Routines for documentation improved communication and the inclusion of protective and short-term dynamic clinical factors were perceived as clinically relevant. Problem-solving ability, attitude, and motivation of staff were facilitating factors. Comprehensive risk assessment instruments require substantial support for staff to find them manageable. Systematic documentation is required to measure actual daily clinical use.

KEYWORDS:

• structured risk assessment
• START
• implementation
• clinical use of research evidence

Providing good-quality care with a safe environment for forensic psychiatric patients and protecting the community by preventing recidivism are at the heart of forensic care. Assessing patient risk of future violence and other adverse events as accurately as possible is required to manage such a challenge. Research and evidence-based practice standards suggest that such assessments should be based on structured risk assessment instruments (SRAIs), rather than relying solely on clinical judgment (Allnut, O’Driscoll, Ogloff, Daffern, & Adams, 2010; NICE, 2007; Risk Management Authority, 2006). Experience-based clinical assessments tend to be subjective and may lack transparency, making them difficult to evaluate and potentially causing legal uncertainty for patients. Even using SRAIs, risk assessments might be erroneous, but an explicit and structured procedure allows such assessments to be fully analyzed and, if needed, revised (Allnut et al., 2010). However, experience-based nonstructured clinical assessment of risk remains a common practice within forensic mental health care (Nielsen et al., 2015; Singh, Serper, Reinharth, & Fazel, 2011).

Risk assessments of future violence often need to consider different time perspectives and circumstances, in the long term (months to years) and in the short term (days, weeks, and months). A service might have a well-functioning risk assessment procedure in place for long-term risk to inform planning of care and risk management, court decisions about continuing care, and granting of privileges, but still rely on clinical experience alone to make decisions about short-term risk assessment involving activities and use of privileges at the ward level (Allnut et al., 2010; Nicholls, Petersen, Brink, & Webster, 2011). Evidence-based practice standards suggest that short-term or daily clinical decisions on the wards in forensic psychiatric care need to be informed by evidence-based structured risk assessments.

A number of SRAIs are available for assessing risk and thereby reducing violence and other adverse events in settings such as forensic, correctional, psychiatric, and community care (Singh, 2013). Clinical application with poor fidelity to the SRAI manual and evidence-based guidelines can, however, result in reduced effectiveness. Previous research and clinical experience also indicate that implementing such instruments in clinical settings can be problematic, thus restricting their optimal clinical use (Vincent, Guy, Perrault, & Gershenson, 2016; Crocker et al., 2011; Levin, Nilsen, Bendtsen, & Bulow, 2016; Nonstad & Webster, 2011).

Having a sound plan for initial implementation is important when implementing a SRAI (Vincent, Paiva-Salisbury, Cook, Guy, & Perrault, 2012; Young, Moline, Farrell, & Bierie, 2006). Engaging stakeholders early in the implementation process and involving users when selecting an instrument have been found to facilitate implementation of SRAIs (Crocker et al., 2011; Kroppan et al., 2011; Mullen, Drinkwater, & Lewin, 2013; Vincent et al., 2012; Vojt, Slesser, Marshall, & Thomson, 2011; Wright & Webster, 2011; Young et al., 2006).

The complexity of the instrument and perceived clinical usefulness need to be considered when selecting the SRAI for implementation (Crocker et al., 2011; Kroppan et al., 2011; Mullen et al., 2013). Selecting a SRAI that allows professional clinical discretion can improve the perceived usefulness of an SRAI (Kroppan et al., 2011; Vincent et al., 2012). Providing sufficient education, training, and continuous support to users of the SRAI during the implementation are essential. Continuous communication of goals and provision of feedback about progress are also facilitating (Crocker et al., 2011; Kroppan et al., 2011; Vojt et al., 2011; Young et al., 2006).

Organizational complexity and staff turnover have, on the other hand, been found to hamper implementation (Bhui et al., 2000; Crocker et al., 2011). Lack of resources such as time and funding are other known barriers to implementation of SRAIs (Bhui et al., 2000; Ferguson, 2002; MacGabhann, Baker, & Dixon, 2002; Mullen et al., 2013).

Despite the growing body of research, more research is needed for improved understanding of the particular facilitators and barriers to the implementation and clinical use of different risk assessment instruments (Crocker et al., 2011; Desmarais, 2017), which is particularly true for instruments including dynamic and protective factors in different settings (Daffern, 2007; Judges, Egan, & Broad, 2016; Sturup, Forsman, Haggård, Karlberg, & Johansson, 2013; Vojt, Thomson, & Marshall, 2013; Wilson, Desmarais, Nicholls, & Brink, 2010; Wilson, Desmarais, Nicholls, Hart, & Brink, 2013).

This study aimed to identify barriers and facilitators for the clinical use of a structured instrument for assessment of short-term risk within forensic psychiatric care (START, Short Term Assessment of Risk and Treatability; Webster, Martin, Brink, Nicholls, & Middleton, 2004), as perceived by the users of the instrument, recognizing that clinical use can, according to research, be influenced by the instrument itself, the way it is implemented, as well as aspects related to the clinical setting. No previous research article has been published on the implementation of START in a Swedish forensic care setting. Knowledge gained on the use of the instrument could potentially add to the growing body of research and inform other settings providing forensic psychiatric care about a possibly useful risk assessment instrument and how to introduce it to facilitate clinical use, and thereby adhere to current demands for evidence-based care.

Method

A qualitative inductive approach was considered appropriate since the aim of the study was to investigate staff perceptions of the implementation of a new risk assessment procedure at a forensic clinic.

Study setting

The study was conducted at one of Sweden’s six major forensic psychiatric regional clinics with a particular responsibility to conduct research on SRAIs and their clinical use. The clinic provides care for 70 patients sentenced to forensic care after having committed a crime due to major mental illness. The patients are predominantly diagnosed with psychosis, personality disorders, mental retardation, and neuropsychiatric disorders. The clinic had seven wards with security levels ranging from high to medium and low. Forensic care in Sweden is governed by the Social Council and is publicly funded. HCR-20, Historical, Clinical, Risk management (Webster, Douglas, Eaves, & Hart, 1997), considered to be a state-of-the-art risk assessment instrument, was already in use at the study site for long-term risk assessments (6–12 months), whereas short-term risk assessments (days, weeks, months) on wards relied on clinical judgments. In an attempt to provide evidence-based risk assessments on the ward and at the care team level, the START was piloted.

Risk assessment instrument

The START instrument (Webster et al., 2004) provides guidance on health care and preventive measures, encompassing clinical, dynamic risk and protective factors. Additional relevant individual-specific risk and protective factors can be added. The START also includes assessment of several different types of incidents, such as violence against others, suicide, self-harm, self-neglect, being victimized, unauthorized leave, and substance abuse. START assessments are well suited to be performed by care teams including nursing staff, but can also be made by single assessors. Training concerning the instrument and its use before using START is recommended. With some experience, an assessment is estimated to take about 30 minutes (Webster, Martin, Brink, Nicholls, & Desmarais, 2010).

START is designed to be used in diverse settings (inpatient and outpatient) providing care for forensic, correctional, and psychiatric populations, and covers a wide range of adverse incidents (Nicholls et al., 2011). The predictive validity of START has been demonstrated in previous research on forensic psychiatric patients (Doyle, Lewis, & Brisbane, 2008; Nicholls, Brink, Desmaris, Webster, & Martin, 2006; Nonstad et al., 2010; O’Shea & Dickens, 2014; Webster et al., 2004). With the inclusion of short-term clinical, dynamic risk and protective factors and a variety of adverse events, START is considered well suited for the needs of forensic care at nursing level, and it addresses specific evidence-based practice recommendations for risk assessments (NICE, 2007; Risk Management Authority, 2006).

The START manual was translated into Swedish by the head of the clinic research unit and distributed to staff at the clinic with formal consent from the authors of START.

Procedure

Selection of instrument

The clinic management and research unit recognized a need to implement an evidence-based routine to the assessment of patients’ risk of adverse events at the wards. The need to evaluate which instrument could be used to achieve this was also identified. The use of a structured decision support was also requested by the nursing staff and the manager at the wards involved in the project. The research unit evaluated different short-term risk assessment instruments, from which START was selected.

Education and support

Before implementation of START, the care team and staff on the two wards were trained in its use for two days and informed of the rationale of the project, the new routines, and the clinical evidence suggesting the need for change. Training was provided by the coordinator of research at the clinic, and information about data gathering routines was provided by the first author. The example cases used in the training were drawn from real patients admitted at the ward. In order to further facilitate the transition of education to actual assessment experience, staff conducted the first START assessments on the patients at the wards under supervision by the first author. The initial supervision was part of the training and as an implementation support measure. Staff expressed a continuous need for such support during assessments. The care team psychologist, highly trained and experienced in risk SPJ assessment instruments, was therefore assigned to act as method advisor. He participated in most assessment meetings. Questions about documentation and other project routines were directed to the first author.

Start assessments

The START assessments were piloted during mid-2010 (planning and start of information meetings and training) to early 2013 (evaluation of the implementation) at two wards in a forensic psychiatric regional clinic in Sweden providing care for 24 patients with predominantly chronic psychosis, low functioning, and neuropsychiatric disorders. The START assessments conducted during the fall of 2011 to the end of 2012, approximately during 14 months, were part of the study. The assessments were planned to be made by a multiprofessional care team every 3 months to assess risk for the patients on the wards. The goal was to have at least 2 assessments for each patient during the period to be able to evaluate change, recognizing that completing the assessments on all patients would initially be made in steps. The START assessments were to be used in clinical care by being a reference to informing staff about assessments of possible risk of adverse events in the time perspective of days, weeks, and months. The care team consisted of a psychiatrist, psychologist, head nurse, occupational therapist, and councilor and ward staff. Staff members were asked to fill in background information and past risk before meetings. This was to facilitate time efficiency at assessment meetings, as lack of background information had been identified as a potential problem during the training.

Continuous feedback and follow-up during the project

Staff was given the opportunity to fill in an evaluation form after attending each assessment meeting. The purpose was to continuously gather information in a structured way about staff perception, and to identify and quickly respond to potential problems they experienced during the project. The evaluation form was used as an implementation support and for continuous follow-up, rather than as a means of evaluating the implementation itself. The evaluation form was based on a translation of the Short-Term Assessment of Risk and Treatability Pilot Questionnaire, from Doyle & Brown (2005). Staff completed the evaluation form anonymously. However, they were encouraged to construct an identification sign of their own, to enable an evaluation of how many unique staff members responded. Filled-in forms were kept in a post box at the reception at the ward. Information from the evaluation forms was collected weekly and assessed regularly by the research team during the 14 months when START assessments were carried out by the staff. Several adjustments in project routines were made based on comments on the evaluation forms. The staff comments and the adjustments are reported in the discussion.

Data collection and study participants

Data were collected by means of audio-recorded focus group interviews with ward staff from February and March 2013. Focus group interviews are commonly used to gather user-related information within less researched areas and to assess the implementation of new procedures (Hylander, 2001; Wibeck, 2010) because this approach is effective in capturing the perspectives of a group of people with a shared experience. Focus groups are considered less leading than surveys or individual interviews because the participants are allowed to direct the discussion more toward what they think is important, and they are allowed to use their own words to describe their experiences. The researcher’s influence is therefore reduced. The participants may also bring up issues that the researcher did not previously consider. A focus group also has the benefit of participants reminding each other about shared experiences, and the discussion can make participants be more open (Hylander, 2001). Staff conducted, used, and made sense of the assessment in collaboration with each other, thus making a group interview similar to the clinical reality. It was considered important to include as many of the participating staff as possible to get as many different perspectives as possible, again making focus group interviews suitable for data collection (Hylander, 2001). Focus group data can be gathered to investigate the content and group processes; addressing the aim of this study, the primary interest was the content.

The 45 staff in the care team and ward staff who were responsible for conducting the assessments were approached by mail and asked about participation in the interviews. The mail included information about the research project, terms of participation, and interview guide themes. Participation was voluntary and 23 responded positively. Five informants agreed to participate but were unable to participate at the time of the interviews because of sick leave or having to prioritize other clinical responsibilities that day as a result of staff shortages on the wards. Consenting informants received written information in connection with the interview about the research project and participation and were able to ask questions.

A semistructured interview guide was used (see Appendix 1), specifying three different themes related to the aim and previous research about possible moderating aspects: the implementation process, conducting risk assessments using the SRAI procedure, and the clinical use of these assessments. Each theme was introduced to the focus group by the moderator as open-ended questions and participant responses then determined the content of the following discussions. The moderator asked participants to elaborate on both positive and negative experiences (Hylander, 2001).

Three focus group interviews were conducted at the clinic over a period of two weeks. Gathering the five remaining consenting participants to make a fourth focus group interview was considered but not deemed appropriate because they were from a mix of wards and other professions. Further, the literature on focus groups suggests that three interviews are sufficient (Hylander, 2001). The focus group interviews lasted between 63 and 80 min. The first author, with previous experience in group interviews, was moderator for all three interviews to improve reliability (Hylander, 2001). No observer moderator was present.

Considering the differences in the formal and clinical mandate, and to avoid some individuals dictating the discussions (Hylander, 2001), ward staff and other professions were interviewed separately. One focus group consisted of an administrator, a nurse, a psychiatrist, a psychologist, a social worker, and an occupational therapist (6 in total). The other two groups consisted of 7 and 5 ward staff, respectively. All participating professions in the project were represented in the interviews. The focus group participants had worked together and knew each other. Some of the care team staff (including the psychiatrist, psychologist, and social worker) had previous experience with conducting structured risk assessments using a manual, but most participants (nurses, ward staff, and occupational therapist) had not.

Data analysis

Hsieh and Shannon (2005) state that conventional content analysis is appropriate for studies with a qualitative inductive approach to address a less well-researched phenomenon with text data gathered by means of open-ended interview questions. The data were therefore analyzed using a conventional qualitative content analysis based on the manifest content of the participants’ responses, allowing categories to emerge from the data inductively (Hseih & Shannon, 2005). The transcriptions were read repeatedly to get an overall understanding of the data. Meaning-bearing units related to the aim were marked and reflected upon, and labels and codes started to emerge from participants’ statements. Those with similar content were sorted into preliminary categories. Categories were analyzed according to their internal homogeneity and were compared to ensure their external heterogeneity. The analysis process followed the original data, constantly comparing the relationship between statements and the labeling and definitions of the category to which they were sorted. The first author sorted statements to a suggested label and category and three senior researchers (PB, PB, and PN) examined the definitions and content of the sorted statements. Any discrepancies were discussed until agreement was reached (Hsieh & Shannon, 2005; Patton, 2002).

The focus group interviews were transcribed by experienced secretaries at the clinic. Pausing was marked by (. .) and words left out by (. . .); emphasis was marked in italics; laughter, sighs, and interruptions were noted in writing. Transcriptions were made verbatim and notes were taken about how the statements were made to provide a basis for identifying meaning-bearing units. Representative quotes were selected and translated into English. Each participant was given a number, with the first number representing which focus group they were in: one, two, or three. The moderator’s quotes are marked with TL.

Ethical and methodological considerations

Ethical approval for the study was given by the Regional Ethical Review Board in Linkoping (Dnr 2011/109–31). The moderator and first author (SL) is a clinical psychologist in two other wards at the clinic and was also part of the implementation project as a research administrator. This might have affected the staff’s participation in the focus group interviews, potentially making them reluctant to speak openly (Hylander, 2001). However, the first author specifically asked for both positive and negative opinions during the focus group interviews, was mindful to avoid dominating the discussion and tried to create an open discussion climate. The moderator’s familiarity with the organization and ongoing implementation project increases the understanding of the informants’ perspectives and statements, on the one hand, but also increases the risk of being influenced by preconceived notions. The moderator was aware of this risk and made every effort during the interviews not to assume to understand unspoken or implied references, instead encouraging informants to describe what they meant and what they were referring to. Longer conversations from the interviews are reported in the Results section when possible, rather than just single comments from participants, mainly to illustrate the basis for the categories and the role of the moderator and thereby improving face validity.

Results

In total, 48 START assessments were conducted during the 14-month evaluation period, involving a total of 27 unique patients. START assessments every 3 months were planned for each patient admitted to the wards, with the goal of having at least 2 START assessments for each patient. One to 3 START assessments were completed for each patient (7 patients with 1 assessment, 20 patients with 2 assessments and one patient with 3 assessments). The mean time between first and second START assessment was 6 months. In only 5 cases of 20 patients with 2 assessments was the 3-month interval kept. The planned interval was not achieved due to new patients being admitted to the ward while others were transferred to other wards or to open forensic care. Some assessment meetings were also canceled and rescheduled due to shortage of staff, usually due to sick leave, vacation periods, or having to prioritize other work such as dealing with emergencies. The number of staff present at the START meetings was 6 on average, but the number and profession varied considerably between different assessment occasions.

Focus group interviews

Staff perceptions of barriers and facilitators that affected the clinical use of START were classified into three main categories, following the content analysis process: factors that were associated with characteristics of the implementation object (i.e., START), the context, and the users (Table 1). Barriers were negative aspects (perceived disadvantages, challenges, difficulties, etc.) concerning the clinical use of START; facilitators were positive aspects (perceived advantages, utility, benefits, etc.) regarding the use of START as part of clinical practice.

TABLE 1. Overview

Category	Facilitating	Barrier	Extracts from quotes
Implementation object
Comprehensiveness and complexity	Inclusion of protective factors	Unreasonably time consuming and extensive	21: I think it was very time-consuming . . . 23: We did use the protective factors, which is something positive, to improve care and to have it documented
Participant requirements	A diverse group of professionals; having enough staff with training and good patient knowledge	Ratings different due to composition of participants	13: It’s the composition of the group of people . . . so that it’s a mixed group of people sitting there.
Drifting from the clinical perspective		Tendency to focus too much on history and risk	21: You skip the time perspective and consider the old stuff, the risks.
Diverse reasons for assessed change		Different staff evaluating the same information differently; concern about failing to consider changes	11: I think the assessments do look different [cleared her throat] in different groups, even if that isn’t the intention of course.
Increased knowledge and understanding	Provided a learning opportunity about risk and protective factors		12: You get to know your patient more thoroughly and also how others perceive the patient.
Increased unity, clarity, and improved communication	Increased unity and consensus of defining and communicating risk and care needs; provided a standard routine for documentation, improving communication of knowledge		13: It’s good with the necessity to document, that we have considered the risks and have documented that, that’s a documentation we can use in court as well as when discussing with patients. . . . But also to have discussed it with everybody else and that’s important, to have had a mutual communication.
Clinical use when planning daily activities	Used clinically to make decisions about use of privileges and specific activities; possible implicit use	Perceived limited and unclear clinical use	11: But I think we have used START and when deciding about leave and the like [12 humming in agreement]. I think we have used them more than we think.
Context
Provision of information and training	Good and sufficient information and training provided initially	Lack of information and training to new staff; lack of booster training; uncertainty about clinical use of START	32: I think that day, when you went through each item one at a time, well, it was clear and well conducted. I think that was really good. . . . It would have been beneficial to have had a follow-up training.
Support and supervision	Expert advisor part of care team, ability to consult SL; support of mid-level manager	Lack of everyday support at ward level	24: I think it would be good to have someone on the ward with a good knowledge about it, someone you can ask . . .
Competing responsibilities and workload		Difficult to keep up with other responsibilities and increased strain; staff turnover and other organizational changes made it difficult to allocate necessary manpower	21: Even if you are entitled to, it’s a strain on the ward. You can’t leave the ward if there are only two people left on the ward.
Access to assessments		Difficult and time-consuming to find assessments in records; lack of unified documentation	24: I think it’s a major issue, the accessibility, to be able to get hold of the information you need quickly, we are reluctant to even try to find it in Cosmic [laughs].
Users
Expectations and preconceptions	Some staff perceived a need for a structured routine to make daily risk assessments	Staff perceived a need for and expected a more concise risk assessment method	12: Well, what I expected from the beginning was a simple document we could do every day to, we do daily decisions . . . but it was something much more.
Attitudes	Some staff positive to change	Perceiving START to be just another assignment adding to an already heavy workload; negative attitudes spread and were discouraging	22: Yes, that was the first impression in the staff group, we are just getting more and more things to do, another thing to do. Several at the same time: Absolutely [several laughing], yes.
Motivation and incentives	Mandatory, no choice; feeling chosen and responsible; taking the START assessments seriously		TL: What made you still put in the effort? All: Because we had to! 24: Let’s go for it anyway. If we start doing it we will get it eventually. TL: How patient. 24: Yes. 21: We were chosen . . .
Continued reliance on clinical experiences	A structured risk assessment potentially helpful in complex cases	Continued reliance on clinical judgment; finding it hard to change old habits	TL: So, what about the risk management plan, how did that work out, did it add something to the care plan already in place? 31: Not very much, no, we already had a plan.
Individual characteristics	Characteristics such as tenaciousness, ability to find meaning, supporting colleagues and solving problems		TL: Whyat made you do it at all? 31. We are stubborn as darn [laughs]. TL: Is there also a loyalty to the workplace? 31: That too. And then we found ways to solve some of the problems.

Main category 1: characteristics of the implementation object

Seven subcategories, facilitators or barriers, were found to be related to the characteristics of the SRAI, that is, the implementation object (Table 1).

Comprehensiveness and complexity

All three focus groups perceived the instrument as time-consuming, extensive, and strenuous to use, even with increased experience of the procedure. Time pressure made staff concerned about not being able to consider all relevant information and therefore making inaccurate assessments.

21: I think it was very time consuming . . .

TL: To do the assessment?

21: Yes, mm-hmm.

TL: Do you others agree?

22: Yes, I think so too, actually.

23: It felt like a lottery, like they only said next, next, you never had the time to discuss.

22: No.

. . .

24: But I think when this START assessment is being done, it’s done so quickly. It feels like we don’t have the time to consider all the information about the patient, unless you are bloody well prepared.

Participant requirements

All three focus groups stressed the need for a diverse group of professionals with different perspectives and good patient knowledge to obtain a nuanced and complete assessment. Staff perceived that when temporary psychiatrists participated and dominated the discussion, assessments differed unreasonably. According to staff, not all participants at the assessments need to have had formal training as long as most of the participants have sufficient training.

13: It’s the composition of the group of people . . . two from the ward, at best the contact persons [all humming in agreement] and the counselor, psychologist, occupational therapist, or psychiatrist or something like that, so that it’s a mixed group of people sitting there. . . . I don’t think everybody needs to have had formal training, as long as enough people have had training, but it is important that people without training have a more or less longer relationship with the patient.

Drifting from the clinical perspective

Assessments tended to drift to encompass historical and long-term future circumstances. There was also a stronger emphasis on risk factors than on protective factors. Some, especially ward staff represented in two focus groups, expressed that they tried to focus more on protective factors but found it difficult to gain the others’ support. Regardless of profession, staff perceived the skewed focus on risk to be unreasonable, and they believed increased emphasis on protective factors was needed.

21: When you mention these protective factors you feel that the others don’t listen, you don’t get any response from the rest of the care team. It can be psychiatrists, psychologists, or anyone, really, who doesn’t agree

21: I wouldn’t call it a problem, really, but one thing I found difficult with the START was to think here and now . . . but when talking about risk it is so easy to think 4 or 5 years back. And then you are not here and now you skip the time perspective and consider the old stuff, the risks. [Several participants agreeing at the same time]

Diverse reasons for assessed change

The estimated risk and protective factors were perceived to be mostly the same between assessment occasions for most patients. When changes were noted, all focus groups expressed that they were not only an expression of actual changes within the patient but also changes in the external environment, provision of care, and staff evaluating the same information differently. Some staff were concerned that previous ratings were given too much weight, which could lead to changes going unnoticed. Some suggested intervals of assessments should be adapted to the needs of the individual patients to make assessments (e.g., the effort required) more cost-effective and clinically meaningful, whereas some thought it important to also have routine intervals to ensure continuity to capture unforeseen change.

11: I think the assessments do look different (cleared her throat) in different groups, even if that isn’t the intention of course

TL: Is that something you have perceived, that the ratings are different depending on who conducted the assessment?

12: We are pretty sure about what we think, but then there might be something external. . . .

13: I think it wasn’t very often but I think in the months when there was a temporary psychiatrist attending . . . I think one or two assessments became a bit different.

13: There was one assessment where there was a rather large change to the better; actually, even though there was still a rather high risk I think it depends on different factors, the patient has calmed down, the care team have a different approach and the care team are getting along in a different manner after tutoring.

Increased knowledge and understanding

Improved knowledge about risk and protective factors were reported in all three focus groups as a consequence of using START, and some expressed that care had improved as a result of a better balance between the two (despite a continued emphasis on risk). Staff in all focus groups stated that they became more informed and updated, and achieved a more nuanced comprehension of the patient, because of the diverse information shared at the meeting by different professions, but also as a result of the necessity of being up to date with patient records to complete the assessment. Staff also perceived an improved ability to understand the content of other structured risk assessments.

12: Now it was like you get to know your patient more thoroughly and also how others perceive the patient. . . . It’s a good way for everyone to get updated about patients—to really read, write, and talk to others about it and get involved. It’s about getting involved in a different way, when you have to.

21: I think I have a better understanding, even with the larger risk assessment (HCR-20). I understand now what they are considering. I don’t think I thought about that before starting doing the START assessments. Before you thought, well the assessment (HCR-20) is there . . . but you didn’t think much about what was in it.

32: You become a bit more aware about the protective factors. I think about the social network, how big it is, is it lacking or not existing. That is something I have thought about a bit more.

31: [Humming in agreement].

Increased unity, clarity, and improved communication

In the mixed focus group, using an SRAI was associated with perceived increased consensus within the care team, because they discussed and agreed on a common view of the patient’s needs, ways of describing and communicating such needs and appropriate ways of treating the patient. The SRAI provided a standard routine for more structured documentation, which was perceived as valuable in two focus groups to explain and communicate the assessment to external care providers such as a court of law and patients. It also made such aspects clearer and more explicit to the staff themselves and improved knowledge sharing among staff. Because the assessments were made by the entire care team, no single staff member was exposed to accusations of subjectivity.

13: And it might be that there is no sensational news in the assessment, but I think it’s good with the necessity to document, that we have considered the risks and have documented that, that’s a documentation we can use in court as well as when discussing with patients, I think that is good. We might have had the same discussions before and come to the same conclusions, but the documentation is different, more standardized, and that gives it, eh, another value and importance. . . . But also to have discussed it with everybody else and that’s important—to have had a mutual communication and shared the information and come to an agreement about conclusions and understanding of what you have seen. And that’s important too, to get a unified treatment and argumentation with the patient by all . . .

32: A good thing has been that we got, I mean, we do so much all the time, but we are not always aware of all the things we are doing. We talk to the patient, we provide medical treatment, we monitor and so on. We already do all these things in our daily work, but it’s written down in a different way now.

Clinical use When planning daily activities

Despite the tendency to continue to rely on clinical judgment, the structured risk assessments were mentioned in all three focus groups as having been used clinically on occasion to make decisions about use of privileges and planning daily activities, especially when there was some disagreement among staff. Staff in the mixed focus group also stated that the START assessment was used in court and to inform staff outside the care team. Some staff were unsure about their clinical use of the risk assessments and thought it to be more implicit.

12: We have used them, too, sometimes in court, right, 13?

TL: In court, how did that work out?

13: Eh, well, I explained at the beginning what it was and they accepted it more or less to be a risk assessment, and there were no demands about doing a HCR-20 instead. . . . .

11: But I think we have used START and when deciding about leave and like [12 humming in agreement]. I think we have used them more than we think.

14: I think so, too; it’s not explicit that we do, but more implicit used . . . in everyday work somehow.

Main category 2: characteristics of the context

Four subcategories were related to contextual characteristics. The context was understood as the dynamic conditions in which something exists or occurs (Nilsen, 2015). Two of the categories constituted both facilitators and barriers; the others were solely barriers (Table 1).

Provision of information and training

The information and training provided to all staff involved at the beginning of the project were described as clear and sufficient. However, there was a perceived need for booster sessions to compensate for the considerable staff turnover at the time of the SRAI implementation, especially ward and nursing staff. Lack of training made some staff insecure about ratings, documentation, and how to use the assessments, which caused insecurity and lack of motivation.

32: It would have been beneficial to have had follow-up training after 6 months

TL: Yes, a booster training?

32: Yes, then . . . I thin,k and then we could have, well, as you said, 31, been able to well explain the purpose.

31: Yes, and it would have needed to be continuous too, because people left and new people started and that . . . that wasn’t there from the beginning.

Support and supervision

In addition to the encouragement provided by the ward manager and having a method expert advisor at meetings (care team psychologist), the two focus groups with ward staff also expressed a need for continuous supervision at ward level by someone knowledgeable about the SRAI and documentation routines.

24: It’s probably the same with all training you get, you get it at the time, eh, and then after a while you start forgetting, and then you forget more and more.

TL: Yes.

25: You need someone to remind you, who knows it well, to teach us . . .

24: I think it would be good to have someone on the ward with a good knowledge about it, someone you can ask, who knows, and then you know when they are working and whom to turn to if you have questions. . . .

32: The work manager for the day always let me leave to do the work, you only need to ask, so they always give you the time.

TL: So you felt you got the time and mandate to take your time with this?

Several [at the same time]: Yes.

Competing responsibilities and workload

All three focus groups perceived the assessments and documentation to be time-consuming, making it difficult to keep up with other responsibilities. Absence from the ward to carry out assessments created a strain on colleagues, a situation that caused stress for ward staff. In addition to staff turnover, several other organizational changes were going on simultaneously with the implementation of the SRAI, making it even more difficult to allocate the necessary time and manpower to do the assessments.

21: Yes, but it’s not easy to find, even if you are entitled to, it’s a strain on the ward. You can’t leave the ward if there are only two people left on the ward. Even if you are entitled to, you can’t. That creates stress.

31: There were all sorts of things going on.

33: There were a lot of things going on at the same time. I know many thought so. But I don’t remember what they were.

TL: Was it about the rebuilding of the clinic perhaps?

33: Yes, it might have been, because there were a lot of people leaving the ward all day and almost nobody left to do the work [laughs].

Access to assessments

The two focus groups with ward staff mentioned that it was difficult to find and understand the assessments in the patient records because of inconsistent documentation. They also perceived that there was not enough time to retrieve and read them to make quick daily decisions. Ward staff in one focus group concluded that short summaries of risk and early warning signs, kept readily accessible on the ward, would have made the assessments more accessible.

24: I think it’s a major issue, the accessibility, to be able to get hold of the information you need quickly; we are reluctant to even try to find it in Cosmic [laughs]. I think so, anyway.

21: That was what I meant before (cleared her throat), when I talked about standing there in the office, and the patient is saying he wants to go out to town. He is in no condition today to do it. You are standing there and need to make a decision quickly. If you could see the risk and protective factors or something to make that decision, yes or no . . .

25: But it would be helpful to be able to access that information, then, to be able to think about your patient, what to consider, but then as I said you need to access that information rather quickly.

Main category 3: characteristics of the users

Five subcategories related to the characteristics of the users of the SRAI, that is, the staff members themselves, were found (see Table 1).

Expectations and preconceptions

Staff in all three focus groups perceived a need for and expected a more concise risk assessment method that could be completed in a matter of minutes on the ward or at round meetings to use as decision support for daily activities. As such, START did not fit staff expectations.

12: Well, what I expected from the beginning was a simple document we could do every day to—we do daily decisions . . . but it was something much more.

TL: Yes.

13: Yes, I thought it would be something short, something you could do in less than 15 minutes, perhaps when you needed to, at round meetings or something.

31: What we really wanted and asked for was a shorter process, that you could do before allowing a patient to use their permission to go on leave, where you could find signatures for this and that. That is what we wanted.

Attitudes

Some ward staff represented in two focus groups were not convinced about the clinical usefulness of an SRAI, which made the effort of using it seem like unnecessary work and not something they were interested in doing. Such attitudes initially spread among staff and sometimes made it discouraging to attend the assessment meetings, even for staff who were committed to change.

TL: But did you from the beginning perceive a need for something new, or did you think what we are doing now is working? Was there any perception of this being a problem, of needing something else?

21: Well, I don’t know where it came from. I’m speaking for myself

22: Yes, that was the first impression in the staff group, we are just getting more and more things to do, another thing to do.

Several at the same time: Absolutely [several laughing], yes.

31: You tried, but there was so much negativity. It’s not encouraging to walk into the room if everyone is sighing and saying, well, now, we have an hour in front of us and this is really hard. Then you don’t become very motivated.

Motivation and incentives

Staff completed the assessments despite all the challenges they experienced, partly because it was mandatory but also because they felt chosen and responsible for the evaluation of the SRAI. They were also aware that risk assessments can have a substantial impact on patients’ lives and hence took them seriously.

TL: What made you still put in the effort, even when you didn’t really understand why?

All: Because we had to!

24: Let’s go for it, anyway. If we start doing it we will get it eventually.

TL: How patient.

24: Yes.

21: We were chosen.

Continued reliance on clinical experiences

Staff in all three focus groups perceived that they continued to rely mainly on their own clinical experience to make daily clinical decisions and were confident in their own ability to accurately assess risk in that way. Some admitted they found it difficult to change ingrained habits of carrying out risk assessments. The structured risk assessment was perceived mainly to confirm previously clinically assessed risk as well as care and risk management already in place.

22: But I think we are also so good at reading the patient—to know when the patient shouldn’t go out and say no.

TL: But have you had any use of the START assessment to make such decisions? Have you considered the START assessment or have you continued to make such decisions as you did before?

22: Yes, I certainly have, anyway.

25 and 23 and several others at the same time: Me, too.

TL: So absolutely, yes.

21: That’s the hardest thing when implementing something new, to break it off with the old behavior you have.

TL: So, what about the risk management plan, how did that work out, did it add something to the care plan already in place?

31: Not very much, no, we already had a plan.

32: We always do.

31: We do everything anyway.

[Several humming in agreement at the same time.]

Individual characteristics

Staff tenaciousness, ability to finding meaning in their work despite difficulties, and willingness to support each other were mentioned as facilitating adherence to the new assessment routines. After having perceived certain barriers, some staff made a few unplanned adaptations to routines (display of the manual at assessment meetings, preratings), showing a problem-solving ability that facilitated change.

TL: What made you do it at all?

31: We are stubborn as darn [laughs].

TL: Is there also a loyalty to the workplace?

31: That too. And then we found ways to solve some of the problems, by preparing the assessments by doing preliminary preratings.

32: Yes, we prepared them, to make everyone.

31: And it felt good be able to say, well, we have prepared this, let’s do it!

Discussion

This study sought to identify barriers and facilitators for the clinical use of START, a structured instrument for assessment of short-term risk within forensic psychiatric care, as perceived by the users of the instrument.

We found that the perceived complexity of using START posed a major barrier (Crocker et al., 2011; Kroppan et al., 2011; Mullen et al., 2013). Selecting an SRAI that covers the range of adverse events and relevant mediating factors fitting the clinical context, and still is convenient and quick to use, poses a challenge (Wright & Webster, 2011). START is quite comprehensive and an assessment requires considerable time, something that previous research has identified as a potential barrier for clinical use (O’Shea & Dickens, 2016), especially because daily decisions often need to be made quickly (Starzomski & Wilson, 2015). Increased assessment experience and preparation by conducting preratings have been found to limit time requirements when completing START assessments (Crocker et al., 2011; Vojt et al., 2013), but did not seem to be sufficient to overcome the strain perceived by staff in this study. Provision of information and training to new employees and offering follow-up booster sessions were suggested, as well as continuous supervision in the ward, to facilitate clinical use of the risk assessments. Research has shown that such efforts can facilitate implementation of SRAIs (Crocker et al., 2011; Damschroder et al., 2009). Other efforts to limit the workload could include replacing staff that need to leave the ward for other tasks and coordinating the different changes in the clinic.

Staff claimed to prefer to continue to rely on their own clinical experience and routine to accurately assess patients on a daily basis. This was possibly due to the perceived lack of user friendliness and difficulties in translating the assessment to management plans (Gilbert, Adams, & Buckingham, 2011) or problems accessing the assessments. These difficulties were noted early in the implementation process, as seen in comments on the evaluation forms. The problem persisted despite the efforts taken to mitigate it. However, it was mentioned that the structured risk assessments were used clinically to make decisions about granting of leave and activities. Hence, there is somewhat contradictory information about clinical use and staff perception of business as usual. Staff speculated that the clinical use of the assessments might be more common than perceived, which could be an explanation, but problematic to evaluate. Part of the purpose of the evidence-based practice and structured risk assessment approach is to make such implicit knowledge and clinical decisions clear and explicit. This finding indicates a need to formally document daily clinical decisions and the basis for them more systematically, to be able to track the clinical use of SRAIs, and to improve staff understanding of their decision making.

Despite the challenges of using START, conducting the assessments also had several benefits. The use of START improved staff knowledge of relevant risk factors and the risk assessment process, increased staff unity, and improved structuring of discussions and communication within the care team (Doyle et al., 2008). Having a standard routine for assessments, risk management, and documentation were also perceived as valuable to clarify and communicate the care and risk management provided to other stakeholders and to disseminate knowledge among the staff. This is in line with the standards of evidence-based practice and the purpose of SRAIs to create transparency and facilitate evaluations of clinical practice (Carroll, 2007; Crocker et al., 2011).

The inclusion of protective factors was perceived to be a major strength of START, even though the assessments still tended to focus on risk and previous historical incidents. This may be the result of a clinical culture of predominantly looking at psychiatric symptoms or previous experience with traditional risk assessment training focusing on risk (Wilson et al., 2010). Nursing and ward staff with less previous experience with SRAIs than other professions seemed to include protective factors more easily. Previous research suggests that different professions focus on risk factors related to their area of responsibility and expertise and hence their ability to mediate the factor (Davies, Heyman, Godin, Shaw, & Reynolds, 2006; Sturup et al., 2013), and base their assessments on somewhat different sources of information, potentially causing differences in assessments (De Vogel & De Ruiter, 2004). Staff, regardless of profession, considered the strong focus on risk and historical incidences to be unreasonable, a point that has received growing support in research (Chu, Thomas, Ogloff, & Daffern, 2011; De Vries Robbé, De Vogel, & Douglas, 2013; O’Shea & Dickens, 2016; Singh et al., 2014).

In previous studies, users of START have reported it to be useful to capture changes in risk and protective factors over time (Kroppan et al., 2011), and START has been found to be potentially useful to evaluate forensic care outcomes (Shinkfield & Ogloff, 2014). Users in this study, however, found few changes and, when changes were noticed, they perceived they could be attributed to actual changes in circumstances or staff assessing the same information differently. The amount and type of changes might differ between populations. The study was conducted on wards caring for patients with chronic psychosis and developmental disorders, with many patients already having long care periods, suggesting that changes were likely to be small. Given that START assessments were completed during a limited time period, it might have been too soon for changes to occur. Previous research indicates that START assessments should be revised every 3 months, but that individual circumstances and behaviors in between assessments need to be considered and assessments revised accordingly (Dickens & O’Shea, 2015). This approach was also suggested by staff in this study.

We also found a number of other user characteristics that affected the use of START. This is in line with much implementation research, which has established that the influence of user characteristics for implementation success is important (Damschroder et al., 2009). The differences in staff approach to change and the clinical use of SRAI could also be explained with reference to Rogers’ Diffusion of Innovation theory (Rogers, 1983), which specifies different adopter categories and how they are associated with different needs for information and support to engage in and manage change. Staff in this study who took initiatives to manage perceived barriers fit the description of early adopters. They acted with a sense of professional responsibility, being aware of the need for change, and were keen to take leader roles to support change. They needed only information on how to proceed with the implementation to engage in new routines. Staff in the study who tried to adapt to new routines but were unable to see the full clinical relevance of the SRAI can be described as an early majority. According to Rogers, they need more evidence of the clinical relevance of the SRAI before being able to comply with change. Some staff followed assessments and documentation routines because it was mandatory and made no effort to use it in daily work, more fitting the description of laggards. They are more bound by existing routines and are hence in need of more information and convincing to engage in new routines. It could be that some staff lacked the necessary information, knowledge, or skills required to assimilate evidence-based practices into everyday working routines, which can also be related to individuals’ sense of self mastery (Damschroder et al., 2009; Webster, Nicholls, Martin, Desmarais, & Brink, 2006).

Attitudes and motivation to using the SRAI differed among the staff. These differences can be understood by means of the Self-Determination Theory, which describes different types of motivation to engage in a particular behavior (Ryan & Deci, 2000). Staff described motivation at different levels, from adhering to new routines because it was mandatory (external motivation), to realizing the relevance of change (identified motivation) and adhering and solving problems out of a sense of joy and pride (intrinsic motivation). The latter kind of motivation is facilitated by a sense of self-control, competence, and a social belonging. Lack of motivation and or self-efficacy was partly overcome by staff offering support to one another, especially by staff more positive to change. But less motivated staff also hampered more motivated staff.

The introduction of a new routine for clinical assessments could be seen as a critique of the current practice, being an attempt to replace clinical expertise and experience with a tick-box exercise, which is likely to generate negative attitudes and hamper motivation for clinical use (Dickens, 2015). Such perspective has been found to provoke a perceived loss of professional discretion and control among staff (Webster et al., 2006). Established clinical experience and routines are also easier and quicker to assess and use than documented assessments, especially when faced with time pressure and fatigue among staff. It might also be the case that staff are already considering many of the relevant risk and protective factors and risk management strategies within their clinical assessments and have access to the long-term risk assessment HCR-20 in patient records. Without specific and systematic routines for documentation of how everyday decisions are reached, this is difficult to confirm. Engaging staff when selecting the SRAI could have mitigated a perceived loss of professional discretion, facilitated engagement, and increased the understanding of and motivation for using the new routine (Nonstad & Webster, 2011).

Staff perceptions were collected during the implementation by evaluation forms and several revisions were made continuously to mitigate the identified problems. Staff perceived the START assessment to be time-consuming, and that there was insufficient time to finish the assessments due to other meetings. Revisions of times of scheduled START assessment meetings were made to prevent stress and having less time for other meetings. The perception of the START assessment taking too long partially decreased over time. Increased assessment experience, revisions of previous estimates taking less time, but also by more extensive preparations explained the perceived change.

Staff reported that assessments varied unreasonably depending on the profession composition of the group at different assessment meetings. It was decided in communication with staff to set a limit of at least 3 staff members. At least a psychiatrist and at least one ward staff member needed to be present to go through with the assessment. Otherwise the assessment meeting was rescheduled. Variations in assessments were also perceived to be caused by a drift from the manual. The START manual was converted into a PowerPoint presentation on display during the assessment meetings to make it more accessible. Continuous explanations of headings by the method expert further facilitated adherence to protocol. Staff emphasized the importance of having the method specialist to guide and coordinate START meetings.

Staff further mentioned difficulties transferring the START assessment into everyday clinical practice. This was partly due to difficulties accessing the START assessments in patient records. Revision of the documentation routines and requirements were specified and communicated to staff, to avoid staff documenting inconsistently or not at all.

In the later part of the project, staff made several suggestions that were not addressed, but were considered when evaluating the project after the pilot implementation had ended. Providing continuous education and training and having individual time interval of assessments based on patient needs was suggested. Another suggestion was to have a printed short summary of the START assessment kept at the ward expedition to make it more easily accessible to use when planning activities.

Staff also reported several benefits of START assessments at the evaluation forms. Doing the assessments created a common understanding and consensus on the risks, protective factors, and health needs for patients. It was also perceived as beneficial to communicate and disseminate such knowledge within the staff group and to other actors.

Overall, the adaptations made or suggested by some staff are in good agreement with recommendations suggested by research (Levin, Nilsen, Bendtsen, & Bulow, 2015) to facilitate implementation of new routines and to manage known barriers, supporting the importance of listening to staff when implementing and evaluating new risk assessment procedures. The staff evaluation of this pilot implementation of START resulted in a decision not to go forward with full-scale implementation at the clinic. The START assessments were, despite their many benefits, perceived as too complex and time-consuming to be conducted by care teams every three months and efforts needed to compensate for the remaining identified barriers would have been substantial (for example, continuous follow-up training and support for all staff at the clinic). In addition, there were several other major organizational changes that needed to take precedence (for example, mandatory self-protection education instigated by the county council, and the rebuild of the clinic).

Conclusions

Specific interventions to make assessments quick and easily assessable and specific formulation of interventions for specific risk situations could help facilitate clinical use. Inclusion of protective and short-term dynamic clinical factors as well as several adverse events were perceived as benefits of the SRAI, but such aspects need further work before they can be used at full potential, which could also facilitate the perceived relevance and clinical use of the SRAI. Several of the principles and purposes of the SRAI were achieved, such as more structure and consequent documentation and communication of clinical decisions, improved knowledge and skill in conducting structured risk assessments, limiting subjectivity in assessments, and improving the possibility of scrutiny and evaluation of clinical practice. The staff’s different accounts of clinical use indicate a need for daily decision making in the wards to be more systematically documented to be able to fully understand the actual clinical use of risk assessments. Staff characteristics and incentives vary and influence how they engage in new routines, indicting a need to provide a range of information, education and support strategies.

Clinical implications

Using a more comprehensive SRAI might be clinically relevant but would also require more resources and support to be manageable. Specific interventions to make assessments quick and easily assessable and specific formulation of interventions for specific risk situations could help facilitate clinical use of assessments. To be able to fully understand the actual clinical use of structured risk assessments in daily decision making, there also seems to be a need to document and analyze daily clinical decisions and the basis for them more systematically, to be able to track the actual clinical use of the assessment, and to make users aware of such processes. Findings from this study can be used to inform selection of a structured risk assessment instrument on clinical wards, the implementation of such instruments, and also clinical follow-up and research of actual clinical use of these instruments.

Limitations

The moderator and first author, SL, is a clinical psychologist in two other wards at the clinic and was also part of the implementation project as a research administrator. This might have affected the staff’s participation in the focus group interviews and made the first author blinded by preconceptions and hence imagining an understanding. The moderator was aware of this risk and made every effort during the interviews to not assume to understand unspoken or implied references and encouraged informants to describe what they meant and what they were referring to. The moderator (first author) specifically asked for both positive and negative opinions and the data analysis also shows that participants could express critique. Considering the differences in the formal and clinical mandate and a tendency for some professions to dictate discussions, ward staff and other professions were interviewed separately. Triangulation was conducted in the transcription of the focus group interviews, and senior researchers participated in the data analysis process to limit bias. The Swedish care context and small numbers of participants make generalization difficult. Repeated focus group interviews could potentially have added important information. However, the findings in this study are in line with previous research using different methodology and larger number of participants.

Further research

Differences in ward or nursing staffs’ accounts of clinical use of structured risk assessment indicate that such use can vary between staff, but also that it might be implicit and hence not conscious and also not systematically documented. Further studies need to consider how such uses are documented and made explicit to fully investigate staff perception and actual clinical practice. The legitimization of structured risk assessments lies in how it influences care. Further research is needed to improve knowledge and clinical guidelines about what constitutes effective risk management interventions to an identified risk. The inclusion of protective factors seems relevant, but also seems to have its own barriers to implementation, which need to be investigated further.

Appendix 1: interview guideInterview guide

How have you perceived the implementation, the introduction, of the START instrument? What worked? What obstacle or problems have you experienced? Do you have any suggestions for improvements?

How have you perceived the use of the START instrument? What benefits have you experienced? What problems have you experienced? Do you have any suggestions for improvements?

How have you perceived the usefulness of START assessments to make every day clinical decisions at the ward? Did the START assessment influence these decisions in any way? What worked? What problems have you experienced? Do you have any suggestions for improvements?

Other comments and or suggestions?