Adherence to Structured Risk Assessment Guidelines: Development and Preliminary Evaluation of an Adherence Scale for the START:AV

ABSTRACT

Risk assessment instruments are widely used to predict risk of adverse outcomes, such as violence or victimization, and to allocate resources for managing these risks among individuals involved in criminal justice and forensic mental health services. For risk assessment instruments to reach their full potential, they must be implemented with fidelity. A lack of information on administration fidelity hinders transparency about the implementation quality, as well as the interpretation of negative or inconclusive findings from predictive validity studies. The present study focuses on adherence, a dimension of fidelity. Adherence denotes the extent to which the risk assessment is completed according to the instrument’s guidelines. We developed an adherence measure, tailored to the Short-Term Assessment of Risk and Treatability: Adolescent Version (START:AV), an evidence-based risk assessment instrument for adolescents. With the START:AV Adherence Rating Scale, we explored the degree to which 11 key features of the instrument were adhered to in 306 START:AVs forms, completed by 17 different evaluators in a Dutch residential youth care facility over a two-year period. Good to excellent interrater reliability was found for all adherence items. We identified differences in adherence scores on the various START:AV features, as well as significant improvement in adherence for those who attended a START:AV refresher workshop. Outcomes of risk assessment instruments potentially impact decision-making, for example, whether a youth’s secure placement should be extended. Therefore, we recommend fidelity monitoring to ensure the risk assessment practice was delivered as intended.

KEYWORDS:

• Implementation
• adherence
• fidelity
• structured risk assessment
• START:AV

Acknowledgments

The authors wish to acknowledge M. J. A. Gradussen, research assistant at CONRISQ group, Zetten, for her assistance in the development of the STARS.

Disclosure statement

The O.G. Heldring Institution publishes the Dutch START:AV User Guide and provides START:AV training. All proceeds go to research, without personal revenues for the authors.