Predictive validity and measurement invariance in juvenile risk assessment: Implications for racial and ethnic disparities in juvenile justice

ABSTRACT

U.S. states have made a good deal of progress in reducing the number of youths incarcerated in residential facilities in recent decades. While this is a positive development, racial and ethnic disparities (RED) persist in which youths are being placed in these facilities. That incongruence means that youths of color are still poised to experience the downsides of those custodial experiences. Juvenile risk and needs assessment (JRNA) has been endorsed as one means of reducing RED in the juvenile justice system, but the literature on performance across race and ethnicity subgroups is limited and mixed to date. This study uses data from 680 youth in juvenile residential facilities in a Western U.S. state, including large subsamples of White and Hispanic youths, to investigate the performance of a JRNA tool, the Ohio Youth Assessment System. The analyses focus on race and ethnicity differences in prevalence of risk and needs; predictive validity across groups; and measurement invariance between White and Hispanic youths. The discussion focuses on key findings and their implications for addressing RED in juvenile justice.

KEYWORDS:

• Juvenile risk and needs assessment
• racial and ethnic disparities
• juvenile justice

Acknowledgments

We wish to acknowledge and thank participating juvenile justice agency and staff. We also wish to thank research staff and students at the University of Cincinnati School of Criminal Justice for their work on this project and Dr. Matthew Diemer for helpful comments on the study’s analytic plan.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Data are drawn from https://www.ojjdp.gov/ojstatbb/ezacjrp/

2. Available at https://www.ojjdp.gov/ojstatbb/special_topics/qa11801.asp?qaDate=2019 and https://www.ojjdp.gov/ojstatbb/corrections/qa08203.asp. The relative rate index (RRI) is computed with the rate of residential placement for minority youth divided by the rate of residential placement involving majority youth. Values above ‘1’ on the RRI reflect disproportionate placement in this instance (Office of Juvenile Justice and Delinquency Prevention [OJJDP] 2012).

3. Burgess (1928) considered this question decades earlier with respect to the usefulness of prediction based on parole records, but Meehl’s work really looked closely at the relative costs/benefits of using distinct approaches to investigation and classification of clinical cases and prediction of later outcomes.

4. The measurement analysis section of the paper focuses on the White and Hispanic subsamples of youth. Those groups have sufficient statistical power to carry out the hypothesis tests associated with the relevant estimates and ensure greater variation in risk factor responses as compared to other race and ethnicity subgroups (e.g., African American youth comprise just 15% of the sample or 96 cases). See note under Summary of Key Findings below for more detail.

5. The indicator for juvenile justice adjudication history is coded ‘0’ for no prior history, ‘1’ for one prior adjudication, and ‘2’ for 2+ prior adjudications.

6. These findings should be interpreted with caution as the data comprise small and varying sample sizes among the groups, which raises challenges in estimating moderation relationships in these models (see, e.g., Aguinis et al., 2017),

7. Based on the range of correlations for African-American youths (e.g., rpb = 0.12 to 0.19), statistical power analysis suggests that roughly twice as many cases (n = 215 at a correlation of 0.20) and often more (n = 428 at a correlation of 0.12) would be required in order to identify statistically significant relationships at conventional testing levels. Future studies should attempt to oversample cases from underrepresented groups to ensure that all can be compared in studies such as this one.

8. Data obtained from https://www.ojjdp.gov/ojstatbb/ezacjrp/asp/Offense_Race.asp

Additional information

Funding

This project was supported in part by Award No. 2014-MU-FX-0006 by the Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions or recommendations expressed in this report are those of the author and do not necessarily reflect those of the Department of Justice.

Notes on contributors

Christopher J. Sullivan

Christopher J. Sullivan is Professor and Director of the School of Criminal Justice and Criminology at Texas State University. He has been Co-Editor of the Journal of Research in Crime and Delinquency since 2017. His recent work has appeared in Criminal Justice Policy Review, Children & Youth Services Review, Justice Quarterly, and the Journal of Quantitative Criminology. He is author of Taking Juvenile Justice Seriously: Developmental Insights and System Challenges (Temple University Press, 2019) and co-author of Juvenile Risk and Needs Assessment: Theory, Research, Policy, and Practice (Routledge, 2021).

James McCafferty

James McCafferty is an Associate Professor of Criminal Justice in the Department of Sociology and Criminal Justice at Kennesaw State University. His research focuses on juvenile risk assessments, offender victimization, and campus carry policy. He has peer-review publications that have appeared in Journal of Crime and Justice, Youth Violence and Juvenile Justice, Criminal Justice Policy Review, and Journal of School Violence.

Jamie Newsome

Jamie Newsome is currently the Research Coordinator for the University of Cincinnati Corrections Institute (UCCI). She received her doctorate in 2013 from the School of Criminal Justice at UC. Her professional goals are centered around translating research into practice and collaborating with practitioners to design, implement, and evaluate the effectiveness of correctional interventions. Dr. Newsome currently serves as the Principal Investigator for a randomized controlled trial testing the effectiveness of CBI-EMP in two midwestern state prisons (Grant No. 2019-RY-BX-0003), and is collaborating with Weber County, Utah to design, implement, and evaluate the Weber Addictions and Reentry Program (WARP) with support from the Bureau of Justice Assistance (Grant No. 2019-RW-BX-0004). Her research has been featured in the Journal of Criminal Justice, Criminal Justice and Behavior, Development and Psychopathology, and the Journal of Youth and Adolescence.

Amber Mandalari

AmberPetkus is a doctoral student in criminal justice at the University of Cincinnati. Her interests include correctional practice and policy, collateral consequences, rehabilitation, juvenile justice, child welfare, and family violence. She focuses especially on translational, implementation, and evaluation research. Her recent work has appeared in Criminal Justice & Behavior, Federal Sentencing Reporter, Journal of Contemporary Criminal Justice, Social Sciences, and Youth Violence and Juvenile Justice.