The short- to medium-term predictive validity of the HoNOS-Secure on violence in a medium-secure forensic psychiatric ward in New Zealand

ABSTRACT

Violence risk prediction in forensic mental health care settings is important in facilitating its management. Although not specifically developed for risk assessment, previous studies have demonstrated that the Health of the Nation Outcome Scales-Secure (HoNOS-secure) and its predecessor may have some predictive validity for violence. This retrospective cohort study examines the predictive validity of HoNOS-secure scores rated on admission in a medium-secure forensic psychiatric ward in New Zealand for inpatient violence in the short- to medium-term at 30 and 90 days respectively. A sample size of 33 eligible admissions was obtained within the study period (2005–2017), and incident reports were rated by two independent clinicians for the occurrence and level of inpatient violence. Poisson, negative binomial and Cox regression analyses were used to investigate HoNOS-secure predictivity for the occurrence of inpatient violence, number of violent events, and time to the first violent event respectively. Although the HoNOS-secure total and subscale scores were not found to be significant predictors, post hoc analysis at the item level demonstrated association of various items with violence. Future studies could consider pooling data from multiple study sites, extending follow-up periods and adopting a prospective design.

KEYWORDS:

• HoNOS-secure
• predictive validity
• inpatient violence
• forensic

It is widely recognized that forensic mental health care settings present a high-risk environment for violence, with negative consequences on both staff and service users (Cornaggia et al., 2011; Zimmer & Cabelus, 2003). Previous studies in the USA and Canada found that as high as 20%–30% of forensic psychiatric inpatients were involved in at least one act of violence during their hospitalization (Broderick et al., 2015; Nicholls et al., 2009). Violence risk prediction tools have been developed to assist clinicians (Ramesh et al., 2018), and comprise a variety of actuarial and structured professional judgement assessments (Dolan & Doyle, 2000), such as the Historical Clinical Risk Management-20 (HCR-20; Douglas et al., 2013; K. J. Smith et al., 2020) and the Hare Psychopathy Checklist – Revised (PCL-R; Hare, 2003). Additionally, instruments have been developed for the prediction of imminent inpatient violence within the next 24-hour period, such as the Dynamic Appraisal of Situational Aggression (DASA; Maguire et al., 2017; Ogloff & Daffern, 2006) and the Brøset Violence Checklist (BVC; Hvidhjelm et al., 2014; Woods & Almvik, 2002).

The Health of the Nation Outcome Scales-Secure (HoNOS-secure) is a reliable outcome measurement tool (G. Dickens et al., 2007) developed specifically for users of secure and forensic mental health services to be rated with reference to their need for care and clinical risk management procedures (Royal College of Psychiatrists, 2013). Its use is mandated in the UK (Macdonald & Fugard, 2015) and New Zealand (Te Pou o te Whakaaro Nui, 2017) for the collection of routine forensic outcome data, and previous studies have explored its use for tracking risk profiles and recovery (G. Dickens et al., 2010; G. L. Dickens & O’shea, 2017; Long et al., 2010) and service evaluation (Liddiard et al., 2019). Although the HoNOS-secure was not developed for risk assessment, there is previous research on its predecessor, the Health of the Nation Outcome Scales for Mentally Disordered Offenders (HoNOS-MDO), which indicates that the HoNOS-MDO may have some predictive validity for violence.

A study in the Netherlands had used the HoNOS-MDO to assess psychiatric and social functioning of community forensic patients and found it to be a significant univariable predictor of imminent violent or criminal behavior (van den Brink et al., 2010). More recently, an Australian study examining the predictive validity of risk-of-violence measures in medium- to low-secure forensic and civil inpatients found that the predictive accuracy of the HoNOS-secure for short- to medium-term inpatient aggression in the forensic group was in the excellent to outstanding range (Finch et al., 2017). However, overall, there is paucity of data, and, to the best of our knowledge, no research of this nature has been done in New Zealand.

Objectives

The study’s primary objective was to examine the predictive validity of the HoNOS-secure scores rated on admission in a medium-secure forensic psychiatric ward in New Zealand for inpatient violence in the short- to medium-term at 30 and 90 days of admission respectively. The violence parameters investigated were (a) the occurrence of any inpatient violence, (b) the number of inpatient violent events, and (c) the time to the first inpatient violent event.

In addition, should evidence be found for such associations, the study aimed to further determine whether the HoNOS-secure scores provided useful predictive information beyond that reflected in already captured patient characteristics.

Method

Study sample

Table 1 shows the inclusion and exclusion criteria of the study. The start date of the study period corresponded to the operationalization date of the Southern District Health Board (SDHB)’s online incident reporting system, from which the study’s outcome measures for violence were derived. Within the study period, a total of 160 patients were admitted to the ward comprising 306 admissions. Based on the inclusion criterion, an initial sample size of 57 patients with a total of 95 admissions was obtained. After the exclusion criteria were considered, the final sample size consisted of 29 patients with a total of 33 admissions (see Figure 1).

Figure 1. Flow diagram for final sample size obtained.

Table 1. Inclusion and exclusion criteria.

Inclusion Criterion: All forensic psychiatric patients with at least one inpatient admission of a minimum duration of 90 days to the specified ward between 7 August 2005, and 31 December 2017 Exclusion Criteria: Admissions with an absent HoNOS-secure admission score ^a, a HoNOS-secure admission score completed more than one week from the date of admission ^b, or missing outcome measures in the incident reporting database during the study period

^aAn absent HoNOS-secure admission score was defined either as an unrated HoNOS-secure score, a HoNOS-secure score rated as unknown, or an original 12-scale HoNOS score rated instead of a HoNOS-secure score.

^bThe one-week window for the rating of HoNOS-secure admission scores from the date of admission was selected to optimize inclusion of relevant data as it was recognized that not all HoNOS-secure admission scores were entered into the HoNOS-secure database within 24 hours of admission.

Measures

Health of the Nation Outcome Scale-Secure, version 2b (HoNOS-secure)

The HoNOS-secure is a clinician-rated tool developed for use in secure and forensic mental health services and consists of 19 severity scales, comprising seven security scales and an amended version of the 12 original HoNOS scales (see Table 2; Royal College of Psychiatrists, 2013). The seven security scales of the HoNOS-secure relate to a patient’s current need for care and clinical risk management in a secure or forensic setting, while the 12 amended HoNOS scales measure different aspects of a patient’s clinical and functional status. The HoNOS-secure is administered by a trained rater based on clinical information from the preceding 2 weeks of a patient’s presentation. Each scale is rated on a 5-point scale from 0 to 4 based on increasing severity using a standardized glossary. The HoNOS-secure was reported to have acceptable internal consistency for both the seven security scales (α = 0.73) and 12 amended HoNOS scales (α = 0.79; G. Dickens et al., 2007).

Table 2. Structure of the HoNOS-secure (version 2b).

Seven Security Scales (A) Risk of harm to adults or children (B) Risk of self-harm (C) Need for building security to prevent escape (D) Need for a safely staffed living environment (E) Need for escort on leave (F) Risk to individual from others (G) Need for risk management procedures 12 Amended HoNOS Scales Overactive, aggressive, disruptive or agitated behavior Non-accidental self-injury Problem drinking or drug taking Cognitive problems Physical illness or disability problems Problems with hallucinations and delusions Problems with depressed mood Other mental and behavioral problems Problems with relationships Problems with activities of daily living Problems with living conditions Problems with occupation and activities Five Subscales of the 12 Amended HoNOS Scales Behavioral problems (Items 1 and 3) Impairment (Items 4 and 5) Delusions/Hallucinations (Item 6) Depression (Items 2, 7, 8, and 9) Social problems (Items 9, 10, 11, and 12)

Outcome measures of violence

Data were collected for outcome measures of violence during the first 30 and 90 days of each eligible admission. In this study, short-term was defined as the first 30 days of admission, while medium-term referred to the first 90 days. These definitions were comparable to other recent studies of violence risk assessment measures (Chu et al., 2013; Finch et al., 2017). The outcome measures were the occurrence of inpatient violence, the level of violence involved, the total number of violent events, and the time taken for the first violent event to occur.

Violence was defined in this study as threatened, attempted or actual harm to others and was differentiated into three categories. ‘Level 1 violence’ involved interpersonal violence, which included any form of direct or indirect contact with the physical body of the intended victim, such as pushing to get pass, spitting, or splashing with water. ‘Level 2 violence’ incorporated other aggressive acts without physical contact with the intended victim, such as threats of violence and property damage. These categories were in line with previous violence prediction studies (Doyle et al., 2002; Steadman et al., 1998). The remaining category, ‘any violence’, was defined as an admission with at least one incident report involving either Level 1 violence, Level 2 violence, or both.

Procedure

A retrospective cohort study was designed for a medium-secure forensic psychiatric ward managed by the SDHB in Dunedin, New Zealand. The ward was a 13-bed mixed-gender unit which provided psychiatric assessment, treatment, or rehabilitation for regional forensic patients in Otago who were transferred from another facility or service. Due to the relatively low annual turnover rate of patients in the ward and the availability of administrative data routinely collected over the years, a retrospective approach was chosen for the study.

Ethics approval was obtained from the University of Otago Human Ethics Committee prior to study commencement (Reference Number HD18/015), and patient consent was waived in accordance with New Zealand research guidelines (National Ethics Advisory Committee, 2012) in view of the study’s retrospective nature. Local authorization was granted to access relevant patient data from the SDHB’s electronic health care database, and a list of eligible admissions based on the inclusion criterion was obtained. The HoNOS-secure admission score for each admission was then retrieved from the SDHB’s electronic HoNOS database. All scales of the HoNOS-secure were considered in the study, and focus was placed on the seven security scales as they were demonstrated in a previous study to have significant predictive ability for the classification of forensic mental health patients based on their clinical and risk-related needs (Shinkfield & Ogloff, 2016).

To determine outcome measures for violence, inpatient incident reports lodged within the first 90 days of each admission were extracted from the SDHB’s incident reporting database. Each incident report was then rated by the key investigator for the occurrence of inpatient violence and the level of violence involved, and a second blinded clinician with forensic psychiatric expertise was enlisted to rerate the incident reports separately to determine interrater reliability. All data collected were deidentified prior to analysis.

Power calculation and statistical analyses

A power analysis was conducted prospectively to determine the minimum number of admissions required for the study. Based on guidelines from Peduzzi et al. (1995, 1996), a minimum sample size of 20 admissions comprising at least 10 violent admissions and 10 non-violent admissions involving different patients would allow the statistical models used in the study to be reliably estimated with a single independent variable, including the HoNOS-secure admission score. For the binary regression models, even with only 10 violent and 10 non-violent admissions, there would be 80% power to detect an area under the curve (AUC) of .77 or higher using a two-sided test at a significance level of .05. With more patients in each group, adjustment for patient characteristics would be possible. Based on the yearly patient turnover rate of the ward, a sample size of around 60 admissions was estimated for the study period.

All results were presented with 95% confidence intervals (CI) included, so that the actual power of the analyses is apparent. All statistical analyses were performed using Stata 15.1, and two-sided p < .05 was considered statistically significant in this study.

Cohen’s kappa was calculated to measure interrater reliability for the rating of incident reports on the occurrence and level of violent events. The degree of interrater reliability is indicated by the following κ values: 0–.20 = none, .21–.39 = minimal, .40–.59 = weak, .60–.79 = moderate, .80–.90 = strong, and above .90 = almost perfect (McHugh, 2012). Cronbach’s alpha was determined for both the 12 amended HoNOS scales and seven security scales of the HoNOS-secure to assess for internal consistency, and an α of .7 and above is considered acceptable (G. Dickens et al., 2007).

The HoNOS-secure was analyzed for predictive validity on inpatient violence at the total score and subscale levels. Post hoc analysis was subsequently performed at the item level to establish if the predictive validity of individual items had been concealed during the analysis at the total score and subscale levels. As there were patients who were admitted multiple times, robust clustered standard errors at the patient level were used for all regression analyses. Poisson regression analysis was used to investigate the significance of the HoNOS-secure as a predictor variable for both the occurrence of violence and the number of violent events at 30 and 90 days of admission. Where overdispersion was present, as indicated by a statistically significant likelihood ratio test from models without robust standard errors, negative binomial regression was used for the number of violent events. Similarly, Cox regression analysis was used for the time to the first violent event at 30 and 90 days of admission.

Additionally, to determine the predictive validity of the HoNOS-secure scales for the occurrence of inpatient violence, receiver operating characteristic analyses were performed at the first 30 and 90 days of admission, and the AUC was calculated in each case to reflect discrimination performance (Mandrekar, 2010).

Results

Table 3 describes the patient characteristics of the study cohort and total admissions to the ward during the study period. For the 33 admissions in the study, the median scores of the 12 amended HoNOS scales and seven security scales were 17 (IQR = 12) and 16 (IQR = 8) on a possible maximum score of 48 and 28 respectively. Of the 33 admissions, 76% had the HoNOS-secure rated on the same day as the admission, and more than 90% had the HoNOS-secure rated within the first five days of admission.

Table 3. Patient characteristics of the study cohort and total admissions.

Patient characteristic	Study cohort (n = 29)		Total admissions (N = 160)
	n	%	n	%
Sex
Male	26	90	120	75
Female	3	10	40	25
Ethnicity
New Zealand European	17	59	102	64
Other European	1	3	9	6
Māori	8	28	36	23
Pacific Peoples	1	3	6	4
Asian	2	7	7	4
Primary Diagnosis
Adjustment disorder	1	3	1	1
Bipolar disorder	8	28	26	16
Borderline personality disorder	–	0	6	4
Conduct disorder	–	0	2	1
Data deficient	–	0	2	1
Delusional disorder	–	0	4	3
Intellectual disability	–	0	5	3
Major depressive disorder	–	0	18	11
Schizoaffective disorder	4	14	10	6
Schizophrenia	14	48	63	39
Substance use disorder	–	0	8	5
Unspecified schizophrenia spectrum and other  psychotic disorders	2	7	15	9
Secondary Diagnosis ^a
Antisocial personality disorder	4	14	28	18
Attention deficit hyperactivity disorder	2	7	6	4
Autism spectrum disorder	1	3	3	2
Borderline personality disorder	–	0	10	6
Intellectual disability	2	7	7	4
Post-traumatic stress disorder	1	3	16	10
Substance use disorder	23	79	90	56

Note. The mean ages for the study cohort and total admissions were 38.2 years old (SD = 10.3) and 36.2 years old (SD = 11.9) respectively. These included the mean ages of patients with multiple admissions. Dash (–) = diagnosis not present.

^aSome patients had multiple secondary diagnoses or none.

An acceptable internal consistency was obtained in this study for the 12 amended HoNOS scales (α = 0.73), but not for the seven security scales (α = 0.64). For the rating of violent events, the degree of interrater reliability was found to be strong to almost perfect for the four categories rated: occurrence of any inpatient violence (κ = 0.96), Level 1 violence present only (κ = 0.92), Level 2 violence present only (κ = 0.90), and both Level 1 and 2 violence present concurrently (κ = 0.82).

Table 4 presents the number of admissions with at least one violent incident during the study period. The number of admissions involving the various violence categories increased by 6%–12% from 30 to 90 days of admission.

Table 4. Number of admissions involving violence.

Violence category	At 30 days of admission		At 90 days of admission
	n	%	n	%
Level 1	7	21	9	27
Level 2	11	33	15	45
Any violence	12	36	16	48

Note. Total number of admissions for the study cohort (n = 33).

Given the available sample size for the analysis, patients were dichotomized into two groups for each categorical variable. For primary diagnosis, patients were divided into subgroups for mood-related disorders and psychotic disorders respectively. The mood-related disorders subgroup comprised predominantly patients with bipolar disorder apart from one patient with adjustment disorder with depressive symptoms, while the psychotic disorders subgroup consisted of patients with schizophrenia, schizoaffective disorder, and unspecified schizophrenia spectrum and other psychotic disorders.

Table 5 displays the findings in terms of associations between HoNOS-secure total and subscale admission scores, demographics, and primary diagnosis, and the occurrence of any violent events at 30 and 90 days of admission. Only a mood-related primary diagnosis was statistically significant for this at both 30 (RR = 3.22, [1.27, 8.14]) and 90 days (RR = 2.30, [1.06–5.00]) of admission.

Table 5. Associations between demographics, primary diagnosis, HoNOS-secure scores, and occurrence of any violent events.

Demographic or clinical variable	First 30 days of admission			First 90 days of admission
	RR	95% CI	p	RR	95% CI	p
Age (per 5 years)	1.01	[0.79, 1.28]	.962	0.93	[0.76, 1.13]	.474
Sex
Female	Reference			Reference
Male	0.69	[0.18, 2.57]	.580	0.97	[0.26, 3.53]	.958
Ethnicity
Non-Māori	Reference			Reference
Māori	1.25	[0.52, 3.01]	.619	1.36	[0.60, 3.06]	.456
Primary diagnosis
Psychotic disorders group	Reference			Reference
Mood disorders group	3.22	[1.27, 8.14]	.014*	2.30	[1.06, 5.00]	.035*
Total scores (per point)
Seven security scales	1.04	[0.96, 1.13]	.309	1.04	[0.97, 1.12]	.227
12 amended HoNOS scales	1.02	[0.96, 1.07]	.560	1.01	[0.95, 1.06]	.779
Subscale scores (per point)
Behavioral problems	1.12	[0.95, 1.33]	.173	1.14	[0.99, 1.32]	.070
Impairment	1.03	[0.86, 1.22]	.767	1.01	[0.82, 1.25]	.912
Delusions/Hallucinations	1.18	[0.80, 1.72]	.405	1.05	[0.78, 1.41]	.769
Depression	1.05	[0.91, 1.22]	.519	1.01	[0.89, 1.14]	.887
Social problems	0.94	[0.83, 1.05]	.262	0.96	[0.84, 1.09]	.491

Note. CI = confidence interval.

*p < .05.

The HoNOS-secure total and subscale admission scores and other clinical variables were also analyzed for their predictive validity on the number of violent events at 30 and 90 days of admission. As shown in Table 6, no statistically significant predictor variables were found.

Table 6. Associations between demographics, primary diagnosis, HoNOS-secure scores, and number of violent events.

Demographic or clinical variable	First 30 days of admission			First 90 days of admission
	IRR	95% CI	p	IRR	95% CI	p
Age (per 5 years)	1.05	[0.83, 1.33]	.679	0.85	[0.63, 1.13]	.256
Sex
Female	Reference			Reference
Male	0.37	[0.07, 1.88]	.230	0.78	[0.13, 4.54]	.778
Ethnicity
Non-Māori	Reference			Reference
Māori	1.46	[0.50, 4.25]	.490	1.45	[0.43, 4.93]	.553
Primary diagnosis
Psychotic disorders group	Reference			Reference
Mood disorders group	2.30	[0.77, 6.91]	.138	3.00	[0.83, 10.89]	.095
Total scores (per point)
Seven security scales	1.05	[0.97, 1.13]	.236	1.05	[0.93, 1.18]	.439
12 amended HoNOS scales	1.03	[0.97, 1.10]	.302	1.03	[0.97, 1.10]	.301
Subscale scores (per point)
Behavioral problems	1.09	[0.88, 1.35]	.423	1.05	[0.85, 1.31]	.655
Impairment	1.13	[0.95, 1.36]	.174	1.03	[0.79, 1.35]	.806
Delusions/Hallucinations	1.27	[0.78, 2.06]	.334	1.25	[0.87, 1.80]	.229
Depression	1.13	[0.89, 1.44]	.305	1.14	[0.89, 1.46]	.288
Social problems	0.95	[0.82, 1.11]	.545	0.99	[0.82, 1.19]	.913

Note. CI = confidence interval.

Additionally, AUC values were calculated for the predictive validity of the HoNOS-secure on the occurrence of any violence at 30 and 90 days of admission. The AUC values for the seven security scales at 30 (AUC = 0.62, 95% CI [0.41, 0.83]) and 90 days (AUC = 0.62, [0.43, 0.82]) were both below the acceptable minimum of 0.7 (Mandrekar, 2010). The AUC values for the 12 amended HoNOS scales at 30 (AUC = 0.55, [0.34, 0.75]) and 90 days (AUC = 0.53, [0.32, 0.73]) were also low with point estimates close to the chance-level value of 0.5.

Using Cox regression analysis, the HoNOS-secure total and subscale admission scores and other clinical variables were examined as predictor variables for the time to the first violent event within the first 30 and 90 days of admission. As reflected in Table 7, a mood-related primary diagnosis was the only statistically significant predictive variable for this at both 30 (HR = 4.75, [1.47, 15.35]) and 90 days (HR = 4.03, [1.29, 12.60]) of admission.

Table 7. Associations between demographics, primary diagnosis, HoNOS-secure scores, and time to first event involving any violence.

Demographic or clinical variable	First 30 days of admission			First 90 days of admission
	HR	95% CI	p	HR	95% CI	p
Age (per 5 years)	1.03	[0.77, 1.39]	.820	0.95	[0.73, 1.24]	.699
Sex
Female	Reference			Reference
Male	0.68	[0.13, 3.42]	.636	0.91	[0.15, 5.53]	.919
Ethnicity
Non-Māori	Reference			Reference
Māori	1.44	[0.48, 4.32]	.515	1.64	[0.53, 5.10]	.394
Primary diagnosis
Psychotic disorders group	Reference			Reference
Mood disorders group	4.75	[1.47, 15.35]	.009*	4.03	[1.29, 12.60]	.016*
Total scores (per point)
Seven security scales	1.07	[0.95, 1.20]	.261	1.07	[0.96, 1.19]	.202
12 amended HoNOS scales	1.03	[0.96, 1.12]	.386	1.02	[0.94, 1.11]	.571
Subscale scores (per point)
Behavioral problems	1.20	[0.92, 1.58]	.177	1.23	[0.98, 1.55]	.072
Impairment	1.07	[0.85, 1.35]	.566	1.06	[0.78, 1.43]	.723
Delusions/Hallucinations	1.25	[0.75, 2.09]	.389	1.11	[0.72, 1.70]	.646
Depression	1.07	[0.92, 1.25]	.372	1.03	[0.90, 1.19]	.648
Social problems	0.94	[0.84, 1.07]	.366	0.96	[0.83, 1.10]	.546

Note. CI = confidence interval.

*p < .05.

Due to the available sample size, the planned multivariable analyses were not performed.

As a post hoc examination, the subgroups for mood-related disorders and psychotic disorders were compared for qualitative differences in terms of secondary diagnoses. The mood-related disorders subgroup had a similar proportion of patients with substance use disorder (78%) as the psychotic disorders subgroup (80%), but had a higher percentage of patients with antisocial personality disorder (22%) compared to the psychotic disorders subgroup (10%). The sample sizes of the subgroups were too small to allow for meaningful statistical comparison of differences.

Lastly, a post hoc analysis of the HoNOS-secure at the item level found Items 1, 8, and G to be positively associated with inpatient violence (see Supplementary Tables S1, S2, and S3). Item 1 was statistically significant in predicting the occurrence of inpatient violence and the time to the first violent event at both 30 days (RR = 1.41, [1.03, 1.93]; HR = 1.51, [1.02, 2.25]) and 90 days (RR = 1.31, [1.02, 1.70]; HR = 1.43, [1.02, 2.00]) of admission and the number of violent events in the first 90 days (IRR = 1.73, [1.05, 2.86]). Item 8 was positively associated with the number of violent events and the time to the first violent event at both 30 days (IRR = 1.85, [1.19, 2.87]; HR = 1.74, [1.03, 2.96]) and 90 days (IRR = 1.98, [1.23, 3.18]; HR = 1.70, [1.03, 2.81]). Item G also showed a positive association in the first 30 days to the number of violent events (IRR = 2.02, [1.26, 3.26]) and the time to the first violent event (HR = 1.50, [1.02, 2.22]). Conversely, Item 11 was found to be a negative predictor of the number of violent events at 30 days of admission (IRR = 0.54, [0.32, 0.91]). There were no statistically significant associations found for other items of the HoNOS-secure.

Discussion

This study sought to address the literature gap in New Zealand on the predictive validity of the HoNOS-secure for inpatient violence in a medium-secure forensic psychiatric setting. In the study, none of the HoNOS-secure total or subscale scores were statistically significant predictors for violence in the short- to medium-term at 30 and 90 days of admission respectively.

This was inconsistent with previous findings from an Australian study which found the predictive validity of the HoNOS-secure to be in the excellent to outstanding range for short- to medium-term forensic inpatient aggression (Finch et al., 2017). While both studies were retrospective cohort studies involving small sample sizes, inherent differences between the two may have contributed to the difference in observations. The Australian study focused solely on male patients from either low- or medium-secure units, and the predominant primary diagnosis of the 37 forensic patients in the study was schizophrenia (78%). Additionally, in the Australian study, acts of aggression were coded from daily case notes rather than being retrieved from the incident reporting system, and the investigators used six-month periods with the most completed risk-assessment measures instead of selecting admission as a starting point. These may have implications on outcomes, and the differing sample demographics and data collection methods of the two studies make it difficult for direct comparisons of their results.

Additionally, a post hoc analysis of the HoNOS-secure in our study at the item level demonstrated a positive association of the various outcome measures of violence with Items 1, 8, and G and a negative association with Item 11. These associations were not apparent when the HoNOS-secure was analyzed at the total score and subscale levels. It would appear to make clinical sense that a patient presenting with overactive or aggressive behavior at admission and requiring risk management procedures would be more likely to present with violence in the first 30 and 90 days of admission. In contrast, the study by Finch et al. (2017) did not analyze the predictive validity of the HoNOS-secure for violence at the item level. Further studies are needed to corroborate the results from our post hoc analysis due to the possibility of false positive findings.

Another interesting observation from our study was that a mood-related primary diagnosis was the only significant predictive variable for both occurrence of violence and time to the first violent event in the short- to medium-term. In our study, the subgroup with a mood-related primary diagnosis comprised mainly patients with bipolar disorder. Although some past studies indicated a correlation between bipolar disorder and inpatient aggression, previous systematic reviews had found that inpatient aggression was associated more frequently with a diagnosis of schizophrenia rather than bipolar or affective disorders (Cornaggia et al., 2011; Dack et al., 2013). The positive findings in our study for the mood-related subgroup could be attributed to potential confounding due to its higher percentage of patients with antisocial personality disorder (22%) compared to the psychotic disorders subgroup (10%).

The main strengths of the study lie in its in-depth exploration of various violence parameters and excellent rating for outcome measures of violent events with strong to almost perfect interrater agreement. In addition, its use of definitions for categories of violence and time periods (i.e. short-term and medium-term) are comparable to previous violence prediction studies, which helps facilitate future research. The study’s pragmatic nature also highlights the need for timely and appropriately rated HoNOS-secure scores as there were a substantial number of admissions with absent or delayed HoNOS-secure scores, a finding consistent with other studies (Edworthy & Khalifa, 2014; M. Smith & Baxendine, 2015).

Limitations

Due to the study’s retrospective nature, control over data quality was limited. A considerable proportion of HoNOS-secure scores were absent or delayed, and the submission of incident reports by ward staff was subject to varying definitions of violence and reporting threshold. Additionally, it was difficult to ascertain if HoNOS-secure scores had been rated in accordance with the standardized glossary by adequately-trained staff, which may explain the lower internal consistency for the HoNOS-secure scales derived in our study compared to a previous study (G. Dickens et al., 2007) as internal consistencies of scales could increase over time with user training (Taber, 2018).

The total admissions in the study also differed to some extent from the study cohort in terms of sex ratio and composition of primary diagnoses due to attrition as a result of exclusion of patients with incomplete data and the requirement for a minimum admission duration of 90 days. Lastly, although the study had achieved the minimum sample size of at least 10 violent and 10 non-violent admissions required to meet its objectives, the limited sample size did not permit further statistical comparisons between the mood-related and psychotic disorders subgroups, the total admissions and the study cohort, and the subgroups involving Level 1 and 2 violence.

Conclusion

This study examined the predictive validity of the HoNOS-secure for short- and medium-term violence in a medium-secure forensic psychiatric setting in New Zealand and found no evidence for this when the HoNOS-secure was analyzed at the total score and subscale levels. However, post hoc analysis at the item level demonstrated a positive association of various violence outcome measures with Items 1, 8, and G, and a negative association with Item 11. This suggests that the HoNOS-secure may potentially be useful in predicting violence at an item level. In a manner similar to the HCR-20, an individual item may have a disproportionate effect on the future risk of violence for a particular individual and may be more useful than total or subscale scores. Keeping in view that our findings differ from a previous Australian study, more research is warranted. Future studies could consider pooling data from multiple study sites for a larger sample size, extending the follow-up period to explore predictive validity in the longer term, and adopting a prospective study design to ensure data quality. Given the variability in the completion of the HoNOS-secure, it may also be worthwhile for surveys to be conducted with recommendations in place to support changes in practice. At present, the HoNOS-secure serves a variety of purposes which include assessing the need for secure care, and further studies on its violence predictivity may add to its clinical utility.

Acknowledgements

We have no conflicts of interest to disclose and received no funding for this study. We acknowledge that this study was conducted as a Scholarly Project for the Royal Australian and New Zealand College of Psychiatrists Fellowship Program.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/14789949.2023.2189150