Weights matter: Improving the predictive validity of risk assessments for criminal offenders

References

• Andrews, D., & Bonta, J. (1995). LSI-R user’s manual. Toronto, ON: Multi-Health Systems Inc.
   Google Scholar
• Andrews, D., & Bonta, J. (2006). The psychology of criminal conduct. Newark: Anderson Publishing Co.
   Google Scholar
• Angrist, J., & Pischke, J. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton, New Jersey, USA: Princeton University Press.
   Google Scholar
• Aos, S., & Barnoski, R. (2005). “Washington’s Offender Accountability Act: A first look at outcomes,” (Document No. 05-07-1202). Olympia: Washington State Institute for Public Policy.
   Google Scholar
• Austin, J. (2006). How much risk can we take? The misuse of risk assessment in corrections. Federal Probation, 70(2), 58–63.
   Google Scholar
• Austin, J., Coleman, D., Peyton, J., & Johnson, K. D. (2003). Reliability and validity study of the LSI-R risk assessment instrument. Washington DC: The Institute on Crime, Justice, and Corrections, George Washington University.
   Google Scholar
• Baird, C. (2009). A question of evidence: A critique of risk assessment models used in the justice system. Madison, WI: National Council on Crime and Delinquency.
   Google Scholar
• Bamber, D. (1975). The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. Journal of Mathematical Psychology, 12(4), 387–415. doi:10.1016/0022-2496(75)90001-2
   Web of Science ®Google Scholar
• Barnoski, R. (1997). Standards for improving research effectiveness in adult and juvenile justice (Document No. 97-12-1201). Olympia: Washington State Institute for Public Policy.
   Google Scholar
• Barnoski, R., & Aos, S. (2003). Washington’s Offender Accountability Act: An analysis of the Department of Corrections’ risk assessment (Document No. 03-12-1201). Olympia: Washington State Institute for Public Policy.
   Google Scholar
• Barnoski, R., & Drake, E. (2007). Washington’s Offender Accountability Act: Department of Corrections’ static risk instrument (Document No. 07-03-1201). Olympia: Washington State Institute for Public Policy.
   Google Scholar
• Beech, A., Friendship, C., Erikson, M., & Hanson, R. K. (2002). The relationship between static and dynamic risk factors and reconviction in a sample of U.K. child abusers. Sexual Abuse: A Journal of Research and Treatment, 14(2), 155–167. doi:10.1177/107906320201400206
   PubMedGoogle Scholar
• Bobko, P., Roth, P. L., & Buster, M. A. (2007). The usefulness of unit weights in creating composite scores a literature review, application to content validity, and meta-analysis. Organizational Research Methods, 10(4), 689–709. doi:10.1177/1094428106294734
   Web of Science ®Google Scholar
• Bonta, J., Bogue, B., Crowley, M., & Motiuk, L. (2001). Implementing offender classification systems: Lessons learned. In G. Bernfeld, D. Farrington, & A. Leschied (Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp. 227–245). West Sussex, UK: John Wiley and Sons, Ltd.
   Google Scholar
• Brennan, T., Dieterich, W., & Ehret, B. (2009). Evaluating the predictive validity of the COMPAS risk and needs assessment system. Criminal Justice and Behavior, 36(1), 21–40. doi:10.1177/0093854808326545
   Web of Science ®Google Scholar
• Bureau of Justice Statistics. (2017). Probation and Parole in the Unites States, 2015 (NCJ 250230). Washington, D.C.: U.S. Department of Justice.
   Google Scholar
• Caballero, J. (2012). Do surges in international capital inflows influence the likelihood of banking crises? Cross-country evidence on bonanzas in capital inflows and bonanza-boom-bust cycles. Inter-American Development Bank Working Paper Series No. IDB-WP-305.
   Google Scholar
• Caudy, M. S., Durso, J. M., & Taxman, F. S. (2013). How well do dynamic needs predict recidivism? Implications for risk assessment and risk reduction. Journal of Criminal Justice, 41(6), 458–466. doi:10.1016/j.jcrimjus.2013.08.004
   Web of Science ®Google Scholar
• Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. doi:10.1089/big.2016.0047
   PubMed Web of Science ®Google Scholar
• Claudy, J. G. (1972). A comparison of five variable weighting procedures. Educational and Psychological Measurement, 32(2), 311–322. doi:10.1177/001316447203200208
   Web of Science ®Google Scholar
• Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45(12), 1304. doi:10.1037/0003-066X.45.12.1304
   Web of Science ®Google Scholar
• Cohen, T. H., Lowenkamp, C. T., & VanBenschoten, S. W. (2016). Does change in risk matter? Examining whether changes in offender risk characteristics influence recidivism outcomes. Criminology & Public Policy, 15(2), 263–296. doi:10.1111/1745-9133.12190
   Web of Science ®Google Scholar
• Dawes, R. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571. doi:10.1037/0003-066X.34.7.571
   Web of Science ®Google Scholar
• DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3), 837–845. doi:10.2307/2531595
   PubMed Web of Science ®Google Scholar
• Dorans, N., & Drasgow, F. (1978). Alternative weighting schemes for linear prediction. Organizational Be- Havior and Human Performance, 21(3), 316–345. doi:10.1016/0030-5073(78)90057-0
   Google Scholar
• Doverspike, D., Winter, J. L., Healy, M. C., & Barrett, G. V. (1996). Simulations as a method of illustrating the impact of differential weights on personnel selection outcomes. Human Performance, 9(3), 259–273. doi:10.1207/s15327043hup0903_5
   Web of Science ®Google Scholar
• Drake, E. (2014). Predicting criminal recidivism: A systematic review of offender risk assessments in Washington State (Document No. 14-02-1901). Olympia: Washington State Institute for Public Policy.
   Google Scholar
• Drake, E., Aos, S., & Barnoski, R. (2010). Washington’s Offender Accountability Act: Final report on recidivism outcomes (Document No. 10-01-1201). Olympia: Washington State Institute for Public Policy.
   Google Scholar
• Duwe, G. (2014). The development, validity, and reliability of the Minnesota screening tool assessing recidivism risk (MnSTARR). Criminal Justice Policy Review, 25(5), 579–613. doi:10.1177/0887403413478821
   Google Scholar
• Einhorn, H., & Hogarth, R. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13(2), 171–192. doi:10.1016/0030-5073(75)90044-6
   Web of Science ®Google Scholar
• Gendreau, P., Goggin, C., & Smith, P. (2002). Is the PCL-R really the “unparalleled” measure of offender risk?. Criminal Justice and Behavior, 29(4), 397–426. doi:10.1177/0093854802029004004
   Web of Science ®Google Scholar
• Gendreau, P., Little, T., & Goggin, C. (1996). A meta-analysis of the predictors of adult offender recidivism: What works! Criminology, 34(4), 575–608. doi:10.1111/j.1745-9125.1996.tb01220.x
   Web of Science ®Google Scholar
• Georgiou, G. (2017). The innovative bureaucrat: Evidence from the correctional authorities in Washington State. International Review of Applied Economics, 32, 348–373. doi:10.1080/02692171.2017.1351527.
   Web of Science ®Google Scholar
• Goddard, T., & Myers, R. R. (2017). Against evidence-based oppression: Marginalized youth and the politics of risk-based assessment and intervention. Theoretical Criminology, 21(2), 151–167. doi:10.1177/1362480616645172
   Web of Science ®Google Scholar
• Gottfredson, D. M., & Snyder, H. N. (2005). The mathematics of risk classification: Changing data into valid instruments for juvenile courts (NCJ 209158). Washington, DC: US Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention.
   Google Scholar
• Gottfredson, S. D., & Moriarty, L. J. (2006). Statistical risk assessment: Old problems and new applications. Crime & Delinquency, 52(1), 178–200. doi:10.1177/0011128705281748
   Web of Science ®Google Scholar
• Grann, M., & Långström, N. (2007). Actuarial assessment of violence risk to weigh or not to weigh? Criminal Justice and Behavior, 34(1), 22–36. doi:10.1177/0093854806290250
   Web of Science ®Google Scholar
• Grove, W., & Meehl, P. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293. doi:10.1037/1076-8971.2.2.293
   Web of Science ®Google Scholar
• Grove, W., Zald, D., Lebow, B., Snitz, B., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19. doi:10.1037/1040-3590.12.1.19
   PubMed Web of Science ®Google Scholar
• Hamilton, Z., Kigerl, A., Campagna, M., Barnoski, R., Lee, S., Van Wormer, J., & Block, L. (2016). Designed to fit: The development and validation of the STRONG-R recidivism risk assessment. Criminal Justice and Behavior, 43(2), 230–263. doi:10.1177/0093854815615633
   Web of Science ®Google Scholar
• Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36. doi:10.1148/radiology.143.1.7063747
   PubMed Web of Science ®Google Scholar
• Hannah-Moffat, K. (2013). Actuarial sentencing: An ‘unsettled’ proposition. Justice Quarterly, 30(2), 270–296. doi:10.1080/07418825.2012.682603
   Web of Science ®Google Scholar
• Harris, G., Rice, M., & Cormier, C. (2002). Prospective replication of the Violence Risk Appraisal Guide in predicting violent recidivism among forensic patients. Law and Human Behavior, 26(4), 377–394. doi:10.1023/A:1016347320889
   PubMed Web of Science ®Google Scholar
• Holsinger, A., Lowenkamp, C., & Latessa, E. (2003). Ethnicity, gender, and the Level of Service Inventory-Revised. Journal of Criminal Justice, 31(4), 309–320. doi:10.1016/S0047-2352(03)00025-4
   Web of Science ®Google Scholar
• Hosmer, D. W., Jr, Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression. Hoboken, NJ, USA: John Wiley & Sons.
   Google Scholar
• Jones, P. (1996). Risk prediction in criminal justice. In A. Harland (Ed.), Choosing correctional options that work: Defining the demand and evaluating the supply (pp. 33–68). Thousands Oaks, USA: Sage Publications, Inc.
   Google Scholar
• Kroner, D., Mills, J., & Reddon, J. (2005). A coffee can, factor analysis, and prediction of antisocial behavior: The structure of criminal risk. International Journal of Law and Psychiatry, 28(4), 360–374. doi:10.1016/j.ijlp.2004.01.011
   PubMed Web of Science ®Google Scholar
• Labrecque, R. M., Smith, P., Lovins, B. K., & Latessa, E. J. (2014). The importance of reassessment: How changes in the LSI-R risk score can improve the prediction of recidivism. Journal of Offender Rehabilitation, 53(2), 116–128. doi:10.1080/10509674.2013.868389
   Google Scholar
• Latessa, E. J., Lemke, R., Makarios, M., & Smith, P. (2010). The creation and validation of the Ohio Risk Assessment System (ORAS). Federal Probation, 74(1), 16–22.
   Google Scholar
• Latessa, E. J., & Lovins, B. (2010). The role of offender risk assessment: A policy maker guide. Victims and Offenders, 5(3), 203–219. doi:10.1080/15564886.2010.485900
   Google Scholar
• Laughlin, J. (1978). Comment on “Estimating coefficients in linear models: It don’t make no nevermind.” Psychological Bulletin, 85(2), 247–253. doi:10.1037/0033-2909.83.2.213
   Web of Science ®Google Scholar
• Meehl, P. (1954). Clinical versus statistical prediction. Minneapolis: University of Minnesota Press.
   Google Scholar
• Morgan, R. D., Kroner, D. G., Mills, J. F., Serna, C., & McDonald, B. (2013). Dynamic risk assessment: A validation study. Journal of Criminal Justice, 41(2), 115–124. doi:10.1016/j.jcrimjus.2012.11.004
   Web of Science ®Google Scholar
• Mossman, D. (1994). Assessing predictions of violence: Being accurate about accuracy. Journal of Consulting and Clinical Psychology, 62(4), 783. doi:10.1037//0022-006X.62.4.783
   PubMed Web of Science ®Google Scholar
• Parsons, C., & Hulin, C. (1982). “Differentially weighting linear models in organizational research: A cross-validation comparison of four methods.” Organizational Behavior and Human Performance, 30(3), 289–311. doi:10.1016/0030-5073(82)90222-7
   Google Scholar
• Phenix, A., Fernandez, Y., Harris, A. J., Helmus, M., Hanson, R. K., & Thornton, D. (2016). Static-99R coding rules: Revised-2016. Ottawa: Public Safety Canada.
   Google Scholar
• Quinsey, V. (1995). The prediction and explanation of criminal violence. International Journal of Law and Psychiatry, 18(2), 117–128.
   PubMed Web of Science ®Google Scholar
• Quinsey, V., Harris, G., Rice, M., & Cormier, C. (2006). Violent offenders: Appraising and managing risk. Washington, DC: American Psychological Association.
   Google Scholar
• Raju, N. S., Bilgic, R., Edwards, J. E., & Fleer, P. F. (1999). ccuracy of population validity and cross-validity estimation: An empirical comparison of formula-based, traditional empirical, and equal weights procedures. Applied Psychological Measurement, 23(2), 99–115. doi:10.1177/01466219922031220
   Web of Science ®Google Scholar
• Raynor, P. (2007). Risk and need assessment in British probation: The contribution of LSI-R.” Psychology, Crime & Law, 13(2), 125–138. doi:10.1080/10683160500337592
   Web of Science ®Google Scholar
• Schularick, M., & Taylor, A. (2012). Credit booms gone bust: Monetary policy, leverage cycles, and financial crises, 1870–2008. American Economic Review, 102(2), 1029–1061. doi:10.1257/aer.102.2.1029
   Web of Science ®Google Scholar
• Silver, E., Smith, W. R., & Banks, S. (2000). Constructing actuarial devices for predicting recidivism: A comparison of methods. Criminal Justice and Behavior, 27(6), 733–764. doi:10.1177/0093854800027006004
   Web of Science ®Google Scholar
• Singh, J. P., & Fazel, S. (2010). Forensic risk assessment: A metareview. Criminal Justice and Behavior, 37(9), 965–988. doi:10.1177/0093854810374274
   Web of Science ®Google Scholar
• Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31(3), 499–513. doi:10.1016/j.cpr.2010.11.009
   PubMed Web of Science ®Google Scholar
• Singh, J. P., Kroner, D. G., Desmarais, S. L., Wormith, J. S., & Hamilton, Z. (2018). Handbook of recidivism risk/needs assessment tools. Hoboken, NJ, USA: John Wiley & Sons.
   Google Scholar
• Smith, W. R. (1996). The effects of base rate and cutoff point choice on commonly used measures of association and accuracy in recidivism research. Journal of Quantitative Criminology, 12(1), 83–111. doi:10.1007/BF02354472
   Web of Science ®Google Scholar
• Smith, W. R., & Smith, D. R. (1998). The consequences of error: Recidivism prediction and civil-libertarian ratios. Journal of Criminal Justice, 26(6), 481–502. doi:10.1016/S0047-2352(98)00026-9
   Web of Science ®Google Scholar
• Vose, B., Smith, P., & Cullen, F. T. (2013). Predictive validity and the impact of change in total LSI-R score on recidivism. Criminal Justice and Behavior, 40(12), 1383–1396. doi:10.1177/0093854813508916
   Web of Science ®Google Scholar
• Wainer, H. (1976). Estimating coefficients in linear models: It don’t make no nevermind. Psychological Bulletin, 83(2), 213. doi:10.1037/0033-2909.83.2.213
   Web of Science ®Google Scholar
• Wainer, H. (1978). On the sensitivity of regression and regressors. Psychological Bulletin, 85(2), 267–273. doi:10.1037/0033-2909.85.2.267
   Web of Science ®Google Scholar
• Walters, G. D. (2011). Taking the next step: Combining incrementally valid indicators to improve recidivism prediction. Assessment, 18(2), 227–233. doi:10.1177/1073191110397484
   PubMed Web of Science ®Google Scholar
• Webster, C. D., Hucker, S. J., & Haque, Q. (2014). Violence risk-assessment and management: advances through structured professional judgement and sequential redirections. West Sussex, UK: John Wiley and Sons, Ltd.
   Google Scholar
• Wooldridge, J. (2002). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press.
   Google Scholar
• Wormith, J. S. (2017). Automated offender risk assessment. Criminology & Public Policy, 16(1), 281–303. doi:10.1111/1745-9133.12277
   Web of Science ®Google Scholar