Forensic Mental Health Practitioners’ Use of Structured Risk Assessment Instruments, Views about Bias in Risk Evaluations, and Strategies to Counteract It

Figures & data

Table 4. Means and Frequencies of Effectiveness Ratings for Debiasing Strategies (N = 110).

Strategy	M (SD)	Effectiveness Ratings n (%)
		Not at all	Slightly	Moderately	Very	Extremely
Using structured evaluation methods	4.00 (0.89)	1 (0.9%)	5 (4.5%)	22 (20.0%)	47 (42.7%)	35 (31.8%)
Taking personal responsibility to continue learning after completing formal training and education	3.82 (0.79)	0 (0.0%)	6 (5.5%)	28 (25.5%)	56 (50.9%)	20 (18.2%)
Investigating all relevant data before forming an opinion^a	3.75 (0.82)	1 (0.9%)	6 (5.5%)	30 (27.3%)	56 (50.9%)	17 (15.5%)
Basing conclusions and opinions on sound data^b	3.74 (0.82)	0 (0.0%)	9 (8.2%)	28 (25.5%)	56 (50.9%)	17 (15.5%)
Consulting with colleagues about issues of potential bias	3.73 (0.80)	1 (0.9%)	6 (5.5%)	30 (27.3%)	58 (52.7%)	15 (13.6%)
Avoiding advocacy	3.50 (0.97)	1 (0.9%)	18 (16.4%)	33 (30.0%)	41 (37.3%)	17 (15.5%)
Fostering a continuing commitment to objectivity	3.45 (0.88)	1 (0.9%)	15 (13.6%)	39 (35.5%)	44 (40.0%)	11 (10.0%)
Being an active consumer of scientific knowledge	3.45 (0.96)	4 (3.6%)	15 (13.6%)	29 (26.4%)	52 (47.3%)	10 (9.1%)
Critically examining conclusions (e.g., considering alternative hypotheses)	3.38 (0.82)	2 (1.8%)	11 (10.0%)	47 (42.7%)	43 (39.1%)	7 (6.4%)
Limiting the scope of the inquiry and report to the referral question	3.36 (0.98)	4 (3.6%)	16 (14.5%)	38 (34.5%)	40 (36.4%)	12 (10.9%)
Receiving explicit didactic training about objectivity	3.32 (0.97)	3 (2.7%)	20 (18.2%)	36 (32.7%)	41 (37.3%)	10 (9.1%)
Taking time to think about evaluation information rather than immediately writing the report	3.29 (0.94)	5 (4.5%)	15 (13.6%)	40 (36.4%)	43 (39.1%)	7 (6.4%)
Resisting allegiance effects	3.27 (.99)	4 (3.6%)	18 (16.4%)	44 (40.0%)	32 (29.1%)	12 (10.9%)
Continuous introspection about personal biases	3.26 (0.81)	1 (0.9%)	17 (15.5%)	49 (44.5%)	38 (34.5%)	5 (4.5%)
Attending to wording choice in reports to edit out value-laden language	3.19 (0.92)	4 (3.6%)	19 (17.3%)	46 (41.8%)	34 (30.9%)	7 (6.4%)
Exposure to the importance of objectivity through reading professional literature	3.12 (0.99)	6 (5.5%)	23 (20.9%)	40 (36.4%)	34 (30.9%)	7 (6.4%)
Taking careful notes during an evaluation^c	3.11 (1.04)	8 (7.3%)	21 (19.1%)	40 (36.4%)	33 (30.0%)	8 (7.3%)
Disengaging emotionally from cases	3.07 (0.92)	5 (4.5%)	23 (20.9%)	45 (40.9%)	33 (30.0%)	4 (3.6%)
Examining patterns of personal decision-making (e.g. agreement with referral party preferences)^d	3.05 (0.96)	8 (7.3%)	18 (16.4%)	49 (44.5%)	30 (27.3%)	5 (4.5%)
Restricting conclusions and opinions to scientific foundations	2.89 (1.02)	11 (10.0%)	24 (21.8%)	47 (42.7%)	22 (20.0%)	6 (5.5%)
Intentionally controlling existing bias	2.83 (0.92)	9 (8.2%)	26 (23.6%)	54 (49.1%)	17 (15.5%)	4 (3.6%)
Observing others who manage their personal biases successfully	2.60 (1.02)	18 (16.4%)	32 (29.1%)	38 (34.5%)	20 (18.2%)	2 (1.8%)
Developing a sense of pride in one’s professional identity	2.45 (1.20)	31 (28.2%)	26 (23.6%)	31 (28.2%)	16 (14.5%)	6 (5.5%)
Accepting referrals only for cases in which bias is unlikely	2.14 (0.97)	35 (31.8%)	35 (31.8%)	30 (27.3%)	10 (9.1%)	0 (0.0%)
Limiting empathy and rapport in forensic cases	1.87 (1.06)	56 (50.9%)	23 (20.9%)	22 (20.0%)	7 (6.4%)	2 (1.8%)

Note. Items are presented according to mean ratings arranged in descending order. Modal effectiveness ratings are shown in bold. Debiasing strategies identified by Neal and Brodsky (2016) as appearing as suggestions in scientific literature are in italics.

^{a, b, c, d} Additional strategies we identified as suggested in the literature about FREs.

Table 1. Risk assessment instruments: required and optional users.

Risk assessment instrument	Instrument type	Prescribed use	Required users n (%)	Optional users n (%)
Static-99 (Hanson & Thornton, 1999) or Static-99R (Helmus et al., 2012)	Actuarial	Sexual reoffending risk	59 (53.6%)	10 (9.1%)
Historical Clinical Risk Management-20 (HCR-20V2; Webster et al., 1997; HCR-20^V3; Douglas et al., 2014)	SPJ	Violence risk	53 (48.2%)	14 (12.7%)
Historische, Klinische, Toekomstige-30 (HKT-30; Werkgroep Pilotstudy Risicotaxie, 2002) or HKT-R (Spreen et al., 2014)	SPJ	Violence risk	52 (47.3%)	7 (6.4%)
Structured Assessment of PROtective Factors (SAPROF; de Vogel et al., 2009, 2012)	SPJ	Protective factors	52 (47.3%)	22 (20.0%)
Psychopathy Checklist Revised (PCL-R; Hare, 2003) or the Psychopathy Checklist: Screening Version (PCL:SV; Hart et al., 1995)	Actuarial	Psychopathic traits	49 (44.5%)	18 (16.4%)
OTHER (Free text entry)	N/A	N/A	39 (35.5%)	43 (39.1%)
Structured Assessment of Violence Risk in Youth (SAVRY; Borum, Bartel, & Forth, 2002)	SPJ	Youth (12-18 years) violence risk	18 (16.4%)	5 (4.5%)
Short-Term Assessment of Risk and Treatability (START; Webster et al., 2004)	SPJ	Violence risk	10 (9.1%)	2 (1.8%)
None	N/A	N/A	9 (8.2%)	5 (4.5%)
Brief Spousal Assault Form for the Evaluation of Risk (B-SAFER; Kropp et al., 2005)	Actuarial	Spousal assault risk	8 (7.3%)	2 (1.8%)
Violence Risk Appraisal Guide (VRAG; Harris, Rice, & Quinsey, 1993)	Actuarial	Violence risk	1 (0.9%)	1 (0.9%)
Recidive Inschattingsschalen [Recidivism Assessment Scales] (RISC; Adviesbureau Van Montfoort & Reclassering Nederland, 2004)	Actuarial	General risk of antisocial behavior, criminogenic needs	1 (0.9%)	1 (0.9%)
Violence Risk Screening-10 (V-RISK-10; Hartvig et al., 2007)	Actuarial	Violence risk screening	0 (0.0%)	0 (0.0%)
Violence Risk Scale (VRS; Wong & Gordon, 2006)	Actuarial	Violence risk	0 (0.0%)	1 (0.9%)
Classification of Violence Risk (COVR; Monahan et al., 2006)	Actuarial	Violence risk in community	0 (0.0%)	0 (0.0%)
Forensic Operationalized Therapy/Risk Evaluation System (FOTRES; Urbaniok, 2007)	SPJ	Recidivism risk and treatment progress	0 (0.0%)	0 (0.0%)
Sex Offender Risk Appraisal Guide (SORAG; Quinsey et al., 1998)	Actuarial	Sexual reoffending risk	0 (0.0%)	0 (0.0%)

Note. SRAIs are arranged in descending order of number of required users.

Table 3. Mean ratings and frequencies of beliefs about potential sources of cognitive bias in forensic risk evaluations.

Item	M (SD)	Strongly disagree n (%)	Somewhat disagree n (%)	Neither agree nor disagree n (%)	Somewhat agree n (%)	Strongly agree n (%)
An evaluator’s prior beliefs and expectations can affect how s/he goes about analyzing a forensic case	4.15 (0.74)	1 (0.9%)	4 (3.6%)	5 (4.5%)	67 (60.9%)	33 (30.0%)
An evaluator’s prior beliefs and expectations can affect his/her ultimate opinion about a forensic case	4.12 (0.71)	1 (0.9%)	3 (2.7%)	7 (6.4%)	70 (63.6%)	29 (26.4%)
An evaluator who makes a conscious effort to set aside his or her prior beliefs and expectations is less likely to be influenced by them	3.75 (0.85)	4 (3.6%)	3 (2.7%)	24 (21.8%)	65 (59.1%)	14 (12.7%)
When evaluators know what they are expected to find, it affects the conclusions they reach	3.53 (1.01)	6 (5.5%)	11 (10.0%)	25 (22.7%)	55 (50.0%)	13 (11.8%)
Evaluators sometimes know what conclusion they are expected to reach	3.49 (1.05)	7 (6.4%)	10 (9.1%)	30 (27.3%)	48 (43.6%)	15 (13.6%)
An experienced evaluator is less likely than a new evaluator to be influenced by prior beliefs/expectations	2.38 (1.09)	27 (24.5%)	37 (33.6%)	25 (22.7%)	19 (17.3%)	2 (1.8%)
Cognitive bias is generally less of a problem in forensic psychology than in other forensic sciences (e.g., fingerprint analysis; ballistics; hair matching, etc.)	1.69 (0.87)	57 (51.8%)	35 (31.8%)	14 (12.7%)	3 (2.7%)	1 (0.9%)

Note. Items are presented according to mean ratings in descending order. Modal responses are shown in bold.

Table 2. Group mean and differences in usefulness ratings of most frequently used SRAIs.

Risk assessment instrument	Required users M (SD)	Optional users M (SD)	t	p	Hedges’ g [95% CI]
HCR-20	3.98 (0.72)	3.93 (0.83)	t(65) = 0.24	.815	0.07 [-0.52, 0.66]
HKT-30/R	3.92 (0.88)	3.57 (0.98)	t(57) = 0.98	.332	0.39 [-0.40, 1.18]
SAPROF	3.71 (0.85)	3.91 (0.75)	t(72) = −0.95	.347	0.24 [-0.26, 0.74]
PCL-R/SV	3.88 (1.03)	3.89 (0.83)	t(65) = −0.04	.967	0.01 [-0.64, 0.66]
Static-99/99R^a	4.31 (0.75)	3.90 (0.32)	t(30.02) = 2.90	.007	0.57 [-0.10, 1.26]

Note. SRAIs were rated on a Likert scale from 1 = not at all useful to 5 = extremely useful.

^a Welch’s t-test (unequal variance between groups).

Neal, T. M. S., & Brodsky, S. L. (2016). Forensic psychologists’ perceptions of bias and potential correction strategies in forensic mental health evaluations. Psychology, Public Policy, and Law, 22(1), 58–76. https://doi.org/https://doi.org/10.1037/law0000077

 Web of Science ®Google Scholar

Hanson, R. K., & Thornton, D. (1999). Static 99: Improving actuarial risk assessments for sex offenders. (No. 1999–02). Solicitor General Canada.

 Google Scholar

Helmus, L., Thornton, D., Hanson, R. K., & Babchishin, K. M. (2012). Improving the predictive accuracy of Static-99 and Static-2002 with older sex offenders: Revised age weights. Sexual Abuse : a Journal of Research and Treatment, 24(1), 64–101. https://doi.org/https://doi.org/10.1177/1079063211409951

 PubMed Web of Science ®Google Scholar

Webster, C. D., Douglas, K. S., Eaves, D. C., & Hart, S. D. (1997). HCR-20: Assessing risk for violence (version 2). Mental Health Law and Policy Institute.

 Google Scholar

Douglas, K. S., Hart, S. D., Webster, C. D., Belfrage, H., Guy, L. S., & Wilson, C. M. (2014). Historical-Clinical-Risk Management-20, Version 3 (HCR-20v3): Development and overview. International Journal of Forensic Mental Health, 13(2), 93–108. https://doi.org/https://doi.org/10.1080/14999013.2014.906519

 Web of Science ®Google Scholar

Risicotaxie, W. P. (2002). Findings of a nationwide pilot study on the HKT-30. Ministerie van Justitie.

 Google Scholar

Spreen, M., Brand, E., Ter Horst, P., & Bogaerts, S. (2014). Handleiding en Methodologische Verantwoording HKT-R, Historische, Klinische en Toekomstige – Revisie [Guidelines and Methodological Research of the HKT-R, Historical, Clinical and Future – Revision]. Dr. van Mesdag kliniek.

 Google Scholar

de Vogel, V., de Ruiter, C., Bouman, Y. H. A., & de Vries Robbé, M. (2009). SAPROF. Guidelines for the assessment of protective factors for violence risk. Forum Educatief.

 Google Scholar

de Vogel, V., de Ruiter, C., Bouman, Y. H. A., & de Vries Robbé, M. (2012). SAPROF. Guidelines for the assessment of protective factors for violence risk (2nd ed.). Forum Educatief.

 Google Scholar

Hare, R. D. (2003). Manual for the revised psychopathy checklist (2nd ed.). Multi-Health Systems.

 Google Scholar

Hart, S. D., Cox, D. N., & Hare, R. D. (1995). The Hare psychopathy checklist: Screening version. Multi-Health Systems.

 Google Scholar

Borum, R., Bartel, P., & Forth, A. E. (2002). Manual for the structured assessment of violence risk in youth (SAVRY). University of South Florida.

 Google Scholar

Webster, C. D., Martin, M. L., Brink, J., Nicholls, T. L., & Middleton, C. (2004). Manual for the short-term assessment of risk and treatability (START). St. Joseph's Healthcare, Hamilton and British Columbia Mental Health and Addiction Services.

 Google Scholar

Kropp, P. R., Hart, S. D., & Belfrage, H. (2005). The brief spousal assault form for the evaluation of risk (BSAFER): User manual. ProActive ReSolutions, Inc.

 Google Scholar

Harris, G. T., Rice, M. E., & Quinsey, V. L. (1993). Violent recidivism of mentally disordered offenders: The development of a statistical prediction instrument. Criminal Justice and Behavior, 20(4), 315–335.

 Google Scholar

Adviesbureau Van Montfoort & Reclassering Nederland. (2004). RISc versie 1.0. Recidive Inschattings Schalen. Handleiding [RISc version 1.0. Risk Assessment Scales. Manual]. Flevodruk.

 Google Scholar

Hartvig, P., Ostberg, B., Alfarnes, S., Moger, T. A., Skjønberg, M., & Bjørkly, S. (2007). Violent risk screen 10 (V-RISK-10). www.forensicpsychiatry

 Google Scholar

Wong, S. C. P., & Gordon, A. (2006). The validity and reliability of the violence risk scale: A treatment-friendly violence risk assessment tool. Psychology, Public Policy, and Law, 12(3), 279–309. https://doi.org/https://doi.org/10.1037/1076-8971.12.3.279

 Google Scholar

Monahan, J., Steadman, H. J., Appelbaum, P. S., Grisso, T., Mulvey, E. P., Roth, L. H., Robbins, P. C., Banks, S., & Silver, E. (2006). The classification of violence risk. Behavioral Sciences and the Law, 24(6), 721–730. https://doi.org/https://doi.org/10.4135/9781412959537.n35

 Google Scholar

Urbaniok, F. (2007). FOTRES: Forensisches Operationalisiertes Therapie-Risiko-Evaluations-System [FOTRES: Forensic Operationalized Therapy/Risk Evaluation System]. Zytglogge.

 Google Scholar

Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (1998). Violent offenders: Appraising and managing risk. American Psychological Association.

 Google Scholar

Data availability statement

The data that support the findings of this study are available from the corresponding author, JK, upon reasonable request.