Racial fairness in violence risk instruments: a review of the literature

References

• Allan, A., & Dawson, D. (2002). Developing a unique risk of violence tool for Australian Indigenous offenders. http://crg.aic.gov.au/reports/200001-06.pdf
   Google Scholar
• Allan, A., Dawson, D., & Allan, M. M. (2006). Prediction of the risk of male sexual reoffending in Australia. Australian Psychologist, 41(1), 60–68. https://doi.org/10.1080/00050060500391886
   Web of Science ®Google Scholar
• Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017). The earth is flat (p>0.05): Significance thresholds and the crisis of unreplicable research. PeerJ, 5, e3544. https://doi.org/10.7717/peerj.3544
   PubMed Web of Science ®Google Scholar
• Andrews, D. A., & Bonta, J. (1994). The psychology of criminal conduct. Anderson Publishing Co.
   Google Scholar
• Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
   Google Scholar
• Australian Bureau of Statistics. (2018). 4517.0 - Prisoners in Australia, 2018. http://www.abs.gov.au/ausstats/[email protected]/Lookup/by%20Subject/4517.0~2018~Main%20Features~Aboriginal%20and%20Torres%20Strait%20Islander%20prisoner%20characteristics%20~13
   Google Scholar
• Berk, R. (2009). The role of race in forecasts of violent crime. Race and Social Problems, 1(4), 231–242. https://doi.org/10.1007/s12552-009-9017-z
   Google Scholar
• Berk, R. (2019). Accuracy and fairness for juvenile justice risk assessments. Journal of Empirical Legal Studies, 16(1), 175–194. https://doi.org/10.1111/jels.12206
   Web of Science ®Google Scholar
• Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1), 3–44. https://doi.org/10.1177/0049124118782533
   Web of Science ®Google Scholar
• Berry, K. R., Kennedy, S. C., Lloyd, M., Veeh, C. A., & Tripodi, S. J. (2020). The intersectional effects of race and gender on time to reincarceration. Justice Quarterly, 37(1), 132–160. https://doi.org/10.1080/07418825.2018.1524508
   Web of Science ®Google Scholar
• Biau, J. D., Jolles, M. B., & Porcher, M. R. (2010). P value and the theory of hypothesis testing: An explanation for new researchers. Clinical Orthopaedics and Related Research, 468(3), 885–892. https://doi.org/10.1007/s11999-009-1164-4
   PubMed Web of Science ®Google Scholar
• Boer, D. P., Couture, J., Geddes, C., & Ritchie, A. (2004). Yókw'tól: Risk management guide for aboriginal offenders. Correctional Services of Canada.
   Google Scholar
• Bonta, J. (1996). Risk-needs assessment and treatment. In A. T. Harland (Ed.), Choosing correctional options that work: Defining the demand and evaluating the supply (pp. 18–32). Sage.
   Google Scholar
• Bonta, J., Laprairie, C., & Wallace-Capretta, S. (1997). Risk prediction and re-offending: Aboriginal and non-aboriginal offenders. Canadian Journal of Criminology, 39(2), 127–144.
   Google Scholar
• Calmon, F. P., Wei, D., Vinzamuri, B., Ramamurthy, K. N., & Varshney, K. R. (2017, December 4–9). Optimized pre-processing for discrimination prevention [Paper presentation]. 31st International Conference on Neural Information Processing Systems, Long Beach, California, United States.
   Google Scholar
• Celis, L., Huang, L., Keswani, V., & Vishnoi, N. (2019, January 29–31). Classification with fairness constraints: A meta-algorithm with provable guarantees [Paper presentation]. Conference on Fairness, Accountability, and Transparency, Atlanta, GA, United States.
   Google Scholar
• Chenane, J. L., Brennan, P. K., Steiner, B., & Ellison, J. M. (2015). Racial and ethnic differences in the predictive validity of the Level of Service Inventory–Revised among prison inmates. Criminal Justice and Behavior, 42(3), 286–303. https://doi.org/10.1177/0093854814548195
   Web of Science ®Google Scholar
• Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047
   PubMed Web of Science ®Google Scholar
• Cohen, H. W. (2011). P values: Use and misuse in medical literature. American Journal of Hypertension, 24(1), 18–23. https://doi.org/10.1038/ajh.2010.205
   PubMed Web of Science ®Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
   Google Scholar
• Cohen, R. J., Swerdlik, M. E., & Sturman, E. D. (2013). Psychological testing and assessment: An introduction to tests and measurement (8th ed.). McGraw Hill.
   Google Scholar
• Cook, N. R. (2007). Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation, 115(7), 928–935. https://doi.org/10.1161/circulationha.106.672402
   PubMed Web of Science ®Google Scholar
• Corbett-Davies, S., & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv:1808.00023 [cs:CY].
   Google Scholar
• Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017, August 13–17). Algorithmic decision making and the cost of fairness [Paper presentation]. 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada. https://doi.org/10.1145/3097983.3098095
   Google Scholar
• Dawes, R., Faust, D., & Meehl, P. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668–1674. https://doi.org/10.1126/science.2648573
   PubMed Web of Science ®Google Scholar
• Dawson, D. (1999, October 13–15). Risk of violence assessment: Aboriginal offenders and the assumption of homogeneity [Paper presentation]. Best Practice Interventions in Corrections for Indigenous People Conference, Adelaide, Australia.
   Google Scholar
• Day, A. (2003). Reducing the risk of re-offending in Australian Indigenous offenders: What works for whom? Journal of Offender Rehabilitation, 37(2), 1–15. https://doi.org/10.1300/J076v37n02_01
   Google Scholar
• Day, A., Tamatea, A. J., Casey, S., & Geia, L. (2018). Assessing violence risk with Aboriginal and Torres Strait Islander offenders: Considerations for forensic practice. Psychiatry, Psychology and Law, 25(3), 452–464. https://doi.org/10.1080/13218719.2018.1467804
   PubMed Web of Science ®Google Scholar
• Dieterich, W., Mendoza, C., & Brennan, T. (2016). COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Technical report, Northpointe. https://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf
   Google Scholar
• Douglas, K. S., Cox, D. N., & Webster, C. D. (1999). Violence risk assessment: Science and practice. Legal and Criminological Psychology, 4(2), 149–184. https://doi.org/10.1348/135532599167824
   Google Scholar
• Douglas, T., Pugh, J., Singh, I., Savulescu, J., & Fazel, S. (2017). Risk assessment tools in criminal justice and forensic psychiatry: The need for better data. European Psychiatry, 42, 134–137. https://doi.org/10.1016/j.eurpsy.2016.12.009
   PubMed Web of Science ®Google Scholar
• Doyle, M., & Dolan, M. (2002). Violence risk assessment: Combining actuarial and clinical information to structure clinical judgements for the formulation and management of risk. Journal of Psychiatric and Mental Health Nursing, 9(6), 649–657. https://doi.org/10.1046/j.1365-2850.2002.00535.x
   PubMedGoogle Scholar
• Dragomir, R. R., & Tadros, E. (2020). Exploring the impacts of racial disparity within the American juvenile justice system. Juvenile and Family Court Journal, 71(2), 61–73. https://doi.org/10.1111/jfcj.12165
   Web of Science ®Google Scholar
• Eckhouse, L., Lum, K., Conti-Cook, C., & Ciccolini, J. (2019). Layers of bias: A unified approach for understanding problems with risk assessment. Criminal Justice and Behavior, 46(2), 185–209. https://doi.org/10.1177/0093854818811379
   Web of Science ®Google Scholar
• Eisenberg, M. J., van Horn, J. E., Dekker, J. M., Assink, M., van der Put, C. E., Hendriks, J., & Stams, G. J. J. M. (2019). Static and dynamic predictors of general and violent criminal offense recidivism in the forensic outpatient population: A meta-analysis. Criminal Justice and Behavior, 46(5), 732–750. https://doi.org/10.1177/0093854819826109
   Web of Science ®Google Scholar
• Ellerby, L., & MacPherson, P. (2002). Exploring the profiles of Aboriginal sexual offenders: Contrasting Aboriginal and non-Aboriginal sexual offenders to determine unique client characteristics and potential implications for sex offender assessment and treatment strategies (research report no. R-122). Correctional Service of Canada. https://www.publicsafety.gc.ca/lbrr/archives/e%2078.c2%20e45e%202002-eng.pdf
   Google Scholar
• Ewert v. Canada, SCC 30 (2018).
   Google Scholar
• Fazel, S. (2019). The scientific validity of current approaches to violence and criminal risk assessment. In J. W. de Keijser, J. V. Roberts, & J. Ryberg (Eds.), Predictive sentencing: Normative and empirical perspectives (1st ed.). Hart Publishing. https://doi.org/10.5040/9781509921447.ch-011
   Google Scholar
• Fazel, S., Singh, J. P., Doll, H., & Grann, M. (2012). Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24,827 people: Systematic review and meta-analysis. BMJ, 345(2), e4692–e4692. https://doi.org/10.1136/bmj.e4692
   PubMedGoogle Scholar
• Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015, August 10–13). Certifying and removing disparate impact [Paper presentation]. 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia.
   Google Scholar
• Fisher, R. A. (1925). Statistical methods for research workers. Oliver and Boyd.
   Google Scholar
• Flores, A., Bechtel, K., & Lowenkamp, C. (2016). False positives, false negatives, and false analyses: A rejoinder to “Machine bias: There's software used across the country to predict future criminals. And it's biased against blacks”. Federal Probation, 80(2), 38–46.
   Google Scholar
• Frize, M., Kenny, D., & Lennings, C. (2008). The relationship between intellectual disability, Indigenous status and risk of reoffending in juvenile offenders on community orders. Journal of Intellectual Disability Research, 52(6), 510–519. https://doi.org/10.1111/j.1365-2788.2008.01058.x
   PubMed Web of Science ®Google Scholar
• Gallistel, C. R. (2009). The importance of proving the null. Psychological Review, 116(2), 439–453. https://doi.org/10.1037/a0015251
   PubMed Web of Science ®Google Scholar
• Gordon, H., Kelty, S. F., & Julian, R. (2015). Psychometric evaluation of the Level of Service/Case Management Inventory among Australian offenders completing community-based sentences. Criminal Justice and Behavior, 42(11), 1089–1109. https://doi.org/10.1177/0093854815596419
   Web of Science ®Google Scholar
• Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. https://doi.org/10.1037/1076-8971.2.2.293
   Web of Science ®Google Scholar
• Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037/1040-3590.12.1.19
   PubMed Web of Science ®Google Scholar
• Gutierrez, L., Helmus, L. M., & Hanson, R. K. (2016). What we know and don’t know about risk assessment with offenders of Indigenous heritage. Journal of Threat Assessment and Management, 3(2), 97–106. https://doi.org/10.1037/tam0000064
   Google Scholar
• Gutierrez, L., Wilson, H. A., Rugge, T., & Bonta, J. (2013). The prediction of recidivism with Aboriginal offenders: A theoretically informed meta-analysis. Canadian Journal of Criminology and Criminal Justice, 55(1), 55–99. https://doi.org/10.3138/cjccj.2011.E.51
   Web of Science ®Google Scholar
• Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.
   Google Scholar
• Hajian, S., & Domingo-Ferrer, J. (2013). A methodology for direct and indirect discrimination prevention in data mining. IEEE Transactions on Knowledge and Data Engineering, 25(7), 1445–1459. https://doi.org/10.1109/TKDE.2012.72
   Web of Science ®Google Scholar
• Hannah-Moffat, K. (2013). Actuarial sentencing: An “unsettled” proposition. Justice Quarterly, 30(2), 270–296. https://doi.org/10.1080/07418825.2012.682603
   Web of Science ®Google Scholar
• Hannah-Moffat, K., & Maurutto, P. (2010). Re-contextualizing pre-sentence reports: Risk and race. Punishment & Society, 12(3), 262–286. https://doi.org/10.1177/1462474510369442
   Web of Science ®Google Scholar
• Hanson, R. K. (2017). Assessing the calibration of actuarial risk scales: A primer on the E/O index. Criminal Justice and Behavior, 44(1), 26–39. https://doi.org/10.1177/0093854816683956
   Web of Science ®Google Scholar
• Hanson, R. K., Lunetta, A., Phenix, A., Neeley, J., & Epperson, D. (2014). The field validity of Static-99/R sex offender risk assessment tool in California. Journal of Threat Assessment and Management, 1(2), 102–117. https://doi.org/10.1037/tam0000014
   Google Scholar
• Harcourt, B. E. (2007). Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press.
   Google Scholar
• Hardt, M., Price, E., & Srebro, N. (2016, December 5–10). Equality of opportunity in supervised learning [Paper presentation]. 30th International Conference on Neural Information Processing Systems, Barcelona, Spain.
   Google Scholar
• Hart, S. D. (1998). The role of psychopathy in assessing risk for violence: Conceptual and methodological issues. Legal and Criminological Psychology, 3(1), 121–137. https://doi.org/10.1111/j.2044-8333.1998.tb00354.x
   Google Scholar
• Hart, S. D. (2016). Culture and violence risk assessment: The case of Ewert v. Canada. Journal of Threat Assessment and Management, 3(2), 76–96. https://doi.org/10.1037/tam0000068
   Google Scholar
• Hart, S. D., Douglas, K. S., & Guy, L. (2017). The structured professional judgment approach to violence risk assessment: Origins, nature, and advances. In D. P. Boer, A. R. Beech, T. Ward, L. A. Craig, M. Rettenberger, L. E. Marshall, & W. L. Marshall (Eds.), The Wiley handbook on the theories, assessment, and treatment of sexual offending (pp. 643–666). Wiley-Blackwell. https://doi.org/10.1002/9781118574003.wattso030
   Google Scholar
• He, J., & van de Vijver, F. (2012). Bias and equivalence in cross-cultural research. Online Readings in Psychology and Culture, 2(2). https://doi.org/10.9707/2307-0919.1111
   Google Scholar
• Hébert-Johnson, U., Kim, M. P., Reingold, O., & Rothblum, G. N. (2018, July 10–15). Calibration for the (computationally-identifiable) masses [Paper presentation]. 35th International Conference on Machine Learning, Stockholm, Sweden.
   Google Scholar
• Heckbert, D., & Turkington, D. (2001). Turning points: A study of the factors related to the successful reintegration of Aboriginal offenders. Correctional Service of Canada.
   Google Scholar
• Helmus, L., Babchishin, K., & Blais, J. (2012). Predictive accuracy of dynamic risk factors for Aboriginal and non-Aboriginal sex offenders: An exploratory comparison using STABLE-2007. International Journal of Offender Therapy and Comparative Criminology, 56(6), 856–876. https://doi.org/10.1177/0306624X11414693
   PubMed Web of Science ®Google Scholar
• Helmus, L. M., & Babchishin, K. M. (2017). Primer on risk assessment and the statistics used to evaluate its accuracy. Criminal Justice and Behavior, 44(1), 8–25. https://doi.org/10.1177/0093854816678898
   Web of Science ®Google Scholar
• Hollin, C. R., Palmer, E. J., & Clark, D. (2003). The Level of Service Inventory-Revised profile of English prisoners: A needs analysis. Criminal Justice and Behavior, 30(4), 422–440. https://doi.org/10.1177/0093854803253134
   Web of Science ®Google Scholar
• Holsinger, A. M., Lowenkamp, C. T., & Latessa, E. J. (2003). Ethnicity, gender, and the Level of Service Inventory-Revised. Journal of Criminal Justice, 31(4), 309–320. https://doi.org/10.1016/S0047-2352(03)00025-4
   Web of Science ®Google Scholar
• Holsinger, A. M., Lowenkamp, C. T., & Latessa, E. J. (2006). Exploring the validity of the Level of Service Inventory-Revised with Native American offenders. Journal of Criminal Justice, 34(3), 331–337. https://doi.org/10.1016/j.jcrimjus.2006.03.009
   Web of Science ®Google Scholar
• Homel, R., Lincoln, R., & Herd, B. (1999). Risk and resilience: Crime and violence prevention in Aboriginal communities. Australian and New Zealand Journal of Criminology, 32(2), 182–196. https://doi.org/10.1177/000486589903200207
   Web of Science ®Google Scholar
• Hsu, C., Caputi, P., & Byrne, M. K. (2011). The Level of Service Inventory-Revised (LSI-R) and Australian offenders: Factor structure, sensitivity, and specificity. Criminal Justice and Behavior, 38(6), 600–618. https://doi.org/10.1177/0093854811402583
   Web of Science ®Google Scholar
• Hsu, C.-I., Caputi, P., & Byrne, M. K. (2010). Level of Service inventory—Revised: Assessing the risk and need characteristics of Australian Indigenous offenders. Psychiatry, Psychology and Law, 17(3), 355–367. https://doi.org/10.1080/13218710903089261
   Web of Science ®Google Scholar
• Huq, A. Z. (2019). Racial equity in algorithmic criminal justice. Duke Law Journal, 68(6), 1043.
   Web of Science ®Google Scholar
• Hurducas, C. C., Singh, J. P., De Ruiter, C., & Petrila, J. (2014). Violence risk assessment tools: A systematic review of surveys. International Journal of Forensic Mental Health, 13(3), 181–192. https://doi.org/10.1080/14999013.2014.942923
   Web of Science ®Google Scholar
• Jacobs, L. A., & Skeem, J. L. (2021). Neighborhood risk factors for recidivism: For whom do they matter? American Journal of Community Psychology, 67(1-2), 103–115. https://doi.org/10.1002/ajcp.12463
   PubMed Web of Science ®Google Scholar
• Jimenez, A. C., Delgado, R. H., Vardsveen, T. C., & Wiener, R. L. (2018). Validation and application of the LS/CMI in Nebraska probation. Criminal Justice and Behavior, 45(6), 863–884. https://doi.org/10.1177/0093854818763231
   Web of Science ®Google Scholar
• Jones, N. J., Brown, S. L., Robinson, D., & Frey, D. (2016). Validity of the youth assessment and screening instrument: A juvenile justice tool incorporating risks, needs, and strengths. Law and Human Behavior, 40(2), 182–194. https://doi.org/10.1037/lhb0000170
   PubMed Web of Science ®Google Scholar
• Jones, R., & Day, A. (2011). Mental health, criminal justice and culture: Some ways forward? Australasian Psychiatry, 19(4), 325–330. https://doi.org/10.3109/10398562.2011.579613
   PubMed Web of Science ®Google Scholar
• Kamiran, F., & Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1), 1–33. https://doi.org/10.1007/s10115-011-0463-8
   Web of Science ®Google Scholar
• Kamiran, F., Karim, A., & Zhang, X. (2012, December 10–13). Decision theory for discrimination-aware classification [Paper presentation]. 2012 IEEE 12th International Conference on Data Mining, Brussels, Belgium.
   Google Scholar
• Kamishima, T., Akaho, S., Asoh, H., & Sakuma, J. (2012, September 24–28). Fairness-aware classifier with prejudice remover regularizer [Paper presentation]. The European Conference on Machine Learning and Knowledge Discovery in Databases, Bristol, United Kingdom.
   Google Scholar
• Kenny, D. T., & Nelson, P. K. (2008). Young offenders on community orders: Health, welfare and criminogenic needs. Sydney University Press. https://www.justicehealth.nsw.gov.au/publications/ch1-3.pdfhttps://books.google.com.au/books?id=g_alyfiXyjQC
   Google Scholar
• Kleinberg, J., Ludwig, J., Mullainathan, S., & Sunstein, C. R. (2018). Discrimination in the age of algorithms. Journal of Legal Analysis, 10, 113–174. https://doi.org/10.1093/jla/laz001
   Web of Science ®Google Scholar
• Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. arXiv:1609.05807 [cs.LG].
   Google Scholar
• Lakens, D. (2017). Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355–362. https://doi.org/10.1177/1948550617697177
   PubMed Web of Science ®Google Scholar
• Lakens, D., McLatchie, N., Isager, P. M., Scheel, A. M., & Dienes, Z. (2020). Improving inferences about null effects with Bayes factors and equivalence tests. The Journals of Gerontology: Series B, 75(1), 45–57. https://doi.org/10.1093/geronb/gby065
   Google Scholar
• Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269. https://doi.org/10.1177/2515245918770963
   Google Scholar
• Långström, N. (2004). Accuracy of actuarial procedures for assessment of sexual offender recidivism risk may vary across ethnicity. Sexual Abuse: A Journal of Research and Treatment, 16(2), 107–120. https://doi.org/10.1023/B:SEBU.0000023060.61402.07
   PubMed Web of Science ®Google Scholar
• LaPrairie, C. (1995). Seen but not heard: Native people in four Canadian inner cities. The Journal of Human Justice, 6(2), 30–45. https://doi.org/10.1007/BF02585441
   Google Scholar
• Larson, J., Mattu, S., Kirchner, L., & Angwin, J. (2016). How we analyzed the COMPAS recidivism algorithm. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
   Google Scholar
• Lee, S. C., Hanson, R. K., & Blais, J. (2020a). Predictive accuracy of the Static-99R and Static-2002R risk tools for identifying Indigenous and white individuals at high risk for sexual recidivism in Canada. Canadian Psychology/Psychologie Canadienne, 61(1), 42–57. https://doi.org/10.1037/cap0000182
   Web of Science ®Google Scholar
• Lee, S. C., Hanson, R. K., Calkins, C., & Jeglic, E. (2020b). Paraphilia and antisociality: Motivations for sexual offending may differ for American whites and blacks. Sexual Abuse, 32(3), 335–365. https://doi.org/10.1177/1079063219828779
   PubMed Web of Science ®Google Scholar
• Lindhiem, O., Petersen, I. T., Mentch, L. K., & Youngstrom, E. A. (2020). The importance of calibration in clinical psychology. Assessment, 27(4), 840–854. https://doi.org/10.1177/1073191117752055
   PubMed Web of Science ®Google Scholar
• Loza, W., & Simourd, D. J. (1994). Psychometric evaluation of the Level of Supervision Inventory (LSI) among male Canadian federal offenders. Criminal Justice and Behavior, 21(4), 468–480. https://doi.org/10.1177/0093854894021004006
   Web of Science ®Google Scholar
• Lundberg, S., & Lee, S.-I. (2017, December 4–9). A unified approach to interpreting model predictions [Paper presentation]. 31st International Conference on Neural Information Processing Systems, Long Beach, California, United States.
   Google Scholar
• Mann, M. (2009). Good intentions, disappointing results: A progress report on federal Aboriginal corrections. Office of the Correctional Investigator. https://www.oci-bec.gc.ca/cnt/rpt/pdf/oth-aut/oth-aut20091113-eng.pdf
   Google Scholar
• Martel, J., Brassard, R., & Jaccoud, M. (2011). When two worlds collide: Aboriginal risk management in Canadian corrections. British Journal of Criminology, 51(2), 235–255. https://doi.org/10.1093/bjc/azr003
   Web of Science ®Google Scholar
• McCuish, E. C., Mathesius, J. R., Lussier, P., & Corrado, R. R. (2018). The cross-cultural generalizability of the Psychopathy Checklist: Youth Version for adjudicated Indigenous youth. Psychological Assessment, 30(2), 192–203. https://doi.org/10.1037/pas0000468
   PubMed Web of Science ®Google Scholar
• McGrath, A. J., Thompson, A. P., & Goodman-Delahunty, J. (2018). Differentiating predictive validity and practical utility for the Australian adaptation of the Youth Level of Service/Case Management Inventory. Criminal Justice and Behavior, 45(6), 820–839. https://doi.org/10.1177/0093854818762468
   Web of Science ®Google Scholar
• Molnar, T., Allard, T., McKillop, N., & Rynne, J. (2020). Reliability and predictive validity of the Juvenile Sex Offender Assessment Protocol-II in an Australian context. International Journal of Offender Therapy and Comparative Criminology. https://doi.org/10.1177/0306624x19900978
   Google Scholar
• Monahan, J. (1981). Predicting violent behavior: An assessment of clinical techniques. Sage Publications.
   Google Scholar
• Monahan, J., & Skeem, J. L. (2016). Risk assessment in criminal sentencing. Annual Review of Clinical Psychology, 12(1), 489–513. https://doi.org/10.1146/annurev-clinpsy-021815-092945
   PubMedGoogle Scholar
• Muir, N. M., Viljoen, J. L., Jonnson, M. R., Cochrane, D. M., & Rogers, B. J. (2020). Predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) with Indigenous and Caucasian female and male adolescents on probation. Psychological Assessment, 32(6), 594–607. https://doi.org/10.1037/pas0000816
   PubMed Web of Science ®Google Scholar
• Olver, M. E. (2016). Some considerations on the use of actuarial and related forensic measures with diverse correctional populations. Journal of Threat Assessment and Management, 3(2), 107–121. https://doi.org/10.1037/tam0000065
   Google Scholar
• Olver, M. E., Kingston, D. A., & Sowden, J. N. (2020). An examination of latent constructs of dynamic sexual violence risk and need as a function of Indigenous and Nonindigenous ancestry. Psychological Services. https://doi.org/10.1037/ser0000414
   Google Scholar
• Olver, M. E., Neumann, C. S., Wong, S., & Hare, R. D. (2013). The structural and predictive properties of the Psychopathy Checklist-Revised in Canadian Aboriginal and non-Aboriginal offenders. Psychological Assessment, 25(1), 167–179. https://doi.org/10.1037/a0029840
   PubMed Web of Science ®Google Scholar
• Olver, M. E., Sowden, J. N., Kingston, D. A., Nicholaichuk, T. P., Gordon, A., Beggs Christofferson, S. M., & Wong, S. C. P. (2018). Predictive accuracy of Violence Risk Scale–Sexual Offender Version risk and change scores in treated Canadian Aboriginal and non-Aboriginal sexual offenders. Sexual Abuse: A Journal of Research and Treatment, 30(3), 254–275. https://doi.org/10.1177/1079063216649594
   PubMed Web of Science ®Google Scholar
• Olver, M. E., Stockdale, K. C., & Wong, S. C. P. (2012). Short and long-term prediction of recidivism using the Youth Level of Service/Case Management Inventory in a sample of serious young offenders. Law and Human Behavior, 36(4), 331–344. https://doi.org/10.1037/h0093927
   PubMed Web of Science ®Google Scholar
• Olver, M. E., Stockdale, K. C., & Wormith, J. S. (2014). Thirty years of research on the Level of Service Scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26(1), 156–176. https://doi.org/10.1037/a0035080
   PubMed Web of Science ®Google Scholar
• Palmer, E. J., & Hollin, C. R. (2007). The Level of Service Inventory-Revised with English women prisoners: A needs and reconviction analysis. Criminal Justice and Behavior, 34(8), 971–984. https://doi.org/10.1177/0093854807300819
   Web of Science ®Google Scholar
• Papalia, N., Shepherd, S. M., Spivak, B., Luebbers, S., Shea, D. E., & Fullam, R. (2019). Disparities in criminal justice system responses to first-time juvenile offenders according to Indigenous status. Criminal Justice and Behavior, 46(8), 1067–1087. https://doi.org/10.1177/0093854819851830
   Web of Science ®Google Scholar
• Perley-Robertson, B., Helmus, L. M., & Forth, A. (2019). Predictive accuracy of static risk factors for Canadian Indigenous offenders compared to non-Indigenous offenders: Implications for risk assessment scales. Psychology, Crime & Law, 25(3), 248–278. https://doi.org/10.1080/1068316X.2018.1519827
   Web of Science ®Google Scholar
• Perrault, R. T., Vincent, G. M., & Guy, L. S. (2017). Are risk assessments racially biased?: Field study of the SAVRY and YLS/CMI in probation. Psychological Assessment, 29(6), 664–678. https://doi.org/10.1037/pas0000445
   PubMed Web of Science ®Google Scholar
• Pierson, E., Simoiu, C., Overgoor, J., Corbett-Davies, S., Ramachandran, V., Phillips, C., & Goel, S. (2020). A large-scale analysis of racial disparities in police stops across the United States. Nature: Human Behaviour, 4(7), 736–745. https://doi.org/10.1038/s41562-020-0858-1
   PubMed Web of Science ®Google Scholar
• Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., & Weinberger, K. (2017, December 4–9). On fairness and calibration [Paper presentation]. 31st International Conference on Neural Information Processing Systems, Long Beach, California, United States.
   Google Scholar
• Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
   PubMed Web of Science ®Google Scholar
• Quinsey, V. L., & Ambtman, R. (1979). Variables affecting psychiatrists’ and teachers’ assessments of the dangerousness of mentally ill offenders. Journal of Consulting and Clinical Psychology, 47(2), 353–362. https://doi.org/10.1037/0022-006X.47.2.353
   PubMed Web of Science ®Google Scholar
• Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006). Violent offenders: Appraising and managing risk (2nd ed.). American Psychological Association. https://doi.org/10.1037/11367-000
   Google Scholar
• Reisig, M. D., Bales, W. D., Hay, C., & Wang, X. (2007). The effect of racial inequality on Black male recidivism. Justice Quarterly, 24(3), 408–434. https://doi.org/10.1080/07418820701485387
   Web of Science ®Google Scholar
• Rettenberger, M., Boer, D. P., & Eher, R. (2011). The predictive accuracy of risk factors in the Sexual Violence Risk–20 (SVR-20). Criminal Justice and Behavior, 38(10), 1009–1027. https://doi.org/10.1177/0093854811416908
   Web of Science ®Google Scholar
• Ribeiro, M. T., Singh, S., & Guestrin, C. (2016, August 13–17). “Why should I trust you?”: Explaining the predictions of any classifier [Paper presentation]. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, California, United States. https://doi.org/10.1145/2939672.2939778
   Google Scholar
• Rice, M. E., & Harris, G. T. (2005). Comparing effect sizes in follow-up studies: ROC area, Cohen's d, and r. Law and Human Behavior, 29(5), 615–620. https://doi.org/10.1007/s10979-005-6832-7
   PubMed Web of Science ®Google Scholar
• Royal Statistical Society. (2018). Algorithms in the justice system: Some statistical issues. https://www.rss.org.uk/Images/PDF/influencing-change/2018/RSS_submission_Algorithms_in_the_justice_system_Nov_2018.pdf
   Google Scholar
• Rugge, T. (2006). Risk assessment of male Aboriginal offenders: A 2006 perspective (Vol. User Report 2006-01). Public Safety Canada. https://www.securitepublique.gc.ca/cnt/rsrcs/pblctns/rsk-ssssmnt-ml/rsk-ssssmnt-ml-eng.pdf
   Google Scholar
• Schaefer, B. P., & Hughes, T. (2019). Examining judicial petrial release decisions: The influence of risk assessments and race. Criminology, Criminal Justice, Law & Society, 20(2), 47–58.
   Google Scholar
• Schmidt, F., McKinnon, L., Chattha, H., & Brownlee, K. (2006). Concurrent and predictive validity of the Psychopathy Checklist: Youth Version across gender and ethnicity. Psychological Assessment, 18(4), 393–401. https://doi.org/10.1037/1040-3590.18.4.393
   PubMed Web of Science ®Google Scholar
• Schmidt, S., Heffernan, R., & Ward, T. (2020). Why we cannot explain cross-cultural differences in risk assessment. Aggression and Violent Behavior, 50, 101346. https://doi.org/10.1016/j.avb.2019.101346
   Web of Science ®Google Scholar
• Shapley, L. S. (1953). A value for n-person games. In H. W. Kuhn & A. W. Tucker (Eds.), Contributions to the theory of games (pp. 307–317). Princeton University Press.
   Google Scholar
• Shepherd, S. (2016a). Criminal engagement and Australian culturally and linguistically diverse populations: Challenges and implications for forensic risk assessment. Psychiatry, Psychology and Law, 23(2), 256–274. https://doi.org/10.1080/13218719.2015.1053164
   Web of Science ®Google Scholar
• Shepherd, S. (2016b). Violence risk instruments may be culturally unsafe for use with Indigenous patients. Australasian Psychiatry, 24(6), 565–567. https://doi.org/10.1177/1039856216665287
   PubMed Web of Science ®Google Scholar
• Shepherd, S. (2018). Violence risk assessment and Indigenous Australians: A primer. Alternative Law Journal, 43(1), 45–47. https://doi.org/10.1177/1037969X17748210
   Web of Science ®Google Scholar
• Shepherd, S. M. (2015). Finding color in conformity: A commentary on culturally specific risk factors for violence in Australia. International Journal of Offender Therapy and Comparative Criminology, 59(12), 1297–1307. https://doi.org/10.1177/0306624x14540492
   PubMed Web of Science ®Google Scholar
• Shepherd, S. M., Adams, Y., McEntyre, E., & Walker, R. (2014). Violence risk assessment in Australian Aboriginal offender populations: A review of the literature. Psychology, Public Policy, and Law, 20(3), 281–293. https://doi.org/10.1037/law0000017
   Web of Science ®Google Scholar
• Shepherd, S. M., & Anthony, T. (2018). Popping the cultural bubble of violence risk assessment tools. The Journal of Forensic Psychiatry & Psychology, 29(2), 211–220. https://doi.org/10.1080/14789949.2017.1354055
   Web of Science ®Google Scholar
• Shepherd, S. M., & Lewis-Fernandez, R. (2016). Forensic risk assessment and cultural diversity: Contemporary challenges and future directions. Psychology, Public Policy, and Law, 22(4), 427–438. https://doi.org/10.1037/law0000102
   Web of Science ®Google Scholar
• Shepherd, S. M., Luebbers, S., Ferguson, M., Ogloff, J., & Dolan, M. (2014). The utility of the SAVRY across ethnicity in Australian young offenders. Psychology, Public Policy, and Law, 20(1), 31–45. https://doi.org/10.1037/a0033972
   Web of Science ®Google Scholar
• Shepherd, S. M., Singh, J. P., & Fullam, R. (2015). Does the Youth Level of Service/Case Management Inventory generalize across ethnicity? International Journal of Forensic Mental Health, 14(3), 193–204. https://doi.org/10.1080/14999013.2015.1086450
   Web of Science ®Google Scholar
• Shepherd, S. M., & Spivak, B. L. (2021). Finding colour in conformity part II—Reflections on structured professional judgement and cross-cultural risk assessment. International Journal of Offender Therapy and Comparative Criminology, 65(1), 92–99. https://doi.org/10.1177/0306624X20928025
   PubMed Web of Science ®Google Scholar
• Shepherd, S. M., & Strand, S. (2016). The PCL: YV and re-offending across ethnic groups. Journal Of Criminal Psychology, 6(2), 51–62. https://doi.org/10.1108/JCP-02-2016-0006
   Web of Science ®Google Scholar
• Shepherd, S. M., & Willis-Esqueda, C. (2018). Indigenous perspectives on violence risk assessment: A thematic analysis. Punishment and Society, 20(5), 599–627. https://doi.org/10.1177/1462474517721485
   Web of Science ®Google Scholar
• Singh, J. P. (2012). The history, development, and testing of forensic risk assessment tools. In E. Grigorenko (Ed.), Handbook of juvenile forensic psychology and psychiatry (pp. 215–225). Springer.
   Google Scholar
• Singh, J. P. (2013). Predictive validity performance indicators in violence risk assessment: A methodological primer. Behavioral Sciences & the Law, 31(1), 8–22. https://doi.org/10.1002/bsl.2052
   PubMed Web of Science ®Google Scholar
• Singh, J. P., Desmarais, S. L., & Van Dorn, R. A. (2013). Measurement of predictive validity in violence risk assessment studies: A second-order systematic review. Behavioral Sciences & the Law, 31(1), 55–73. https://doi.org/10.1002/bsl.2053
   PubMed Web of Science ®Google Scholar
• Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31(3), 499–513. https://doi.org/10.1016/j.cpr.2010.11.009
   PubMed Web of Science ®Google Scholar
• Skeem, J., & Lowenkamp, C. (2020). Using algorithms to address trade-offs inherent in predicting recidivism. Behavioral Sciences and the Law, 38(3), 259–278. https://doi.org/10.1002/bsl.2465
   PubMed Web of Science ®Google Scholar
• Skeem, J. L., & Lowenkamp, C. T. (2016). Risk, race, and recidivism: Predictive bias and disparate impact. Criminology; An interdisciplinary Journal, 54(4), 680–712. https://doi.org/10.1111/1745-9125.12123
   Web of Science ®Google Scholar
• Smallbone, S., & Rallings, M. (2013). Short-term predictive validity of the static-99 and static-99-R for Indigenous and nonindigenous Australian sexual offenders. Sexual Abuse A Journal of Research and Treatment, 25(3), 302–316. https://doi.org/10.1177/1079063212472937
   PubMed Web of Science ®Google Scholar
• Swets, J. A. (1988). Measuring the accuracy of diagnostic systems. Science, 240(4857), 1285–1293. https://doi.org/10.1126/science.3287615
   PubMed Web of Science ®Google Scholar
• Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest, 1(1), 1–26. https://doi.org/10.1111/1529-1006.001
   PubMedGoogle Scholar
• The United States Department of Justice. (2014). Attorney General Eric Holder speaks at the National Association of Criminal Defense Lawyers 57th annual meeting and 13th State Criminal Justice network conference. https://www.justice.gov/opa/speech/attorney-general-eric-holder-speaks-national-association-criminal-defense-lawyers-57th
   Google Scholar
• Thompson, A. P., & McGrath, A. (2012). Subgroup differences and implications for contemporary risk-need assessment with juvenile offenders. Law and Human Behavior, 36(4), 345–355. https://doi.org/10.1037/h0093930
   PubMed Web of Science ®Google Scholar
• Tsang, S., Piquero, A. R., & Cauffman, E. (2014). An examination of the Psychopathy Checklist: Youth Version (PCL:YV) among male adolescent offenders: An item response theory analysis. Psychological Assessment, 26(4), 1333–1346. https://doi.org/10.1037/a0037500
   PubMed Web of Science ®Google Scholar
• van de Vijver, F., & Tanzer, N. K. (2004). Bias and equivalence in cross-cultural assessment: An overview. European Review of Applied Psychology, 54(2), 119–135. https://doi.org/10.1016/j.erap.2003.12.004
   Google Scholar
• van Eijk, G. (2017). Socioeconomic marginality in sentencing: The built-in bias in risk assessment tools and the reproduction of social inequality. Punishment & Society, 19(4), 463–481. https://doi.org/10.1177/1462474516666282
   Web of Science ®Google Scholar
• Venner, S., Sivasubramaniam, D., Luebbers, S., & Shepherd, S. M. (2021). Cross-cultural reliability and rater bias in forensic risk assessment: A review of the literature. Psychology, Crime & Law, 27(2), 105–121. https://doi.org/10.1080/1068316X.2020.1775829
   Web of Science ®Google Scholar
• Verma, S., & Rubin, J. (2018, May 29). Fairness definitions explained [Paper presentation]. International Workshop on Software Fairness, Gothenburg, Sweden. https://doi.org/10.1145/3194770.3194776
   Google Scholar
• Viallon, V., Ragusa, S., Clavel-Chapelon, F., & Bénichou, J. (2009). How to evaluate the calibration of a disease risk prediction tool. Statistics in Medicine, 28(6), 901–916. https://doi.org/10.1002/sim.3517
   PubMed Web of Science ®Google Scholar
• Viljoen, J. L., Jonnson, M. R., Cochrane, D. M., Vargen, L. M., & Vincent, G. M. (2019). Impact of risk assessment instruments on rates of pretrial detention, postconviction placements, and release: A systematic review and meta-analysis. Law and Human Behavior, 43(5), 397–420. https://doi.org/10.1037/lhb0000344
   PubMed Web of Science ®Google Scholar
• Wadsworth, C., Vera, F., & Piech, C. (2018). Achieving fairness through adversarial learning: An application to recidivism prediction. arXiv:1807.00199 [cs.LG].
   Google Scholar
• Watkins, I. (2011). The utility of Level of Service Inventory-Revised (LSI-R) assessments within NSW correctional environments. Research bulletin. Corrective Services NSW. https://correctiveservices.dcj.nsw.gov.au/content/dam/dcj/corrective-services-nsw/documents/research-and-statistics/rb29-utility-of-level-of-service-inventory-.pdf
   Google Scholar
• Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). Assessing risk of violence to others. In C. D. Webster & M. A. Jackson (Eds.), Impulsivity: Theory, assessment, and treatment (pp. 251–277). Guilford Press.
   Google Scholar
• Westen, D., & Weinberger, J. (2004). When clinical description becomes statistical prediction. American Psychologist, 59(7), 595–613. https://doi.org/10.1037/0003-066X.59.7.595
   PubMed Web of Science ®Google Scholar
• Whiteacre, K. W. (2006). Testing the Level of Service Inventory–Revised (LSI-R) for racial/ethnic bias. Criminal Justice Policy Review, 17(3), 330–342. https://doi.org/10.1177/0887403405284766
   Google Scholar
• Wilson, H. A., & Gutierrez, L. (2014). Does one size fit all? A meta-analysis examining the predictive ability of the Level of Service Inventory (LSI) with Aboriginal offenders. Criminal Justice and Behavior, 41(2), 196–219. https://doi.org/10.1177/0093854813500958
   Web of Science ®Google Scholar
• Wormith, J., & Hogg, S. (2012). The predictive validity of Aboriginal offender recidivism with a general risk/needs assessment inventory. Program Effectiveness, Statistics, and Applied Research Unit, Ministry of Community Safety and Correctional. https://cfbsjs.usask.ca/documents/research/research_papers/LSI-OR%20Aboriginal%20Paper%20w%20Abstract.pdf
   Google Scholar
• Wormith, J., Hogg, S., & Guzzo, L. (2015). The predictive validity of the LS/CMI with Aboriginal offenders in Canada. Criminal Justice and Behavior, 42(5), 481–508. https://doi.org/10.1177/0093854814552843
   Web of Science ®Google Scholar
• Yang, M., Wong, S., & Coid, J. (2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136(5), 740–767. https://doi.org/10.1037/a0020473
   PubMed Web of Science ®Google Scholar
• Zemel, R., Wu, Y., Swersky, K., Pitassi, T., & Dwork, C. (2013, June 16–21). Learning fair representations [Paper presentation]. 30th International Conference on Machine Learning, Atlanta, GA, USA.
   Google Scholar
• Zhang, B. H., Lemoine, B., & Mitchell, M. (2018, February 2–3). Mitigating unwanted biases with adversarial learning [Paper presentation]. 2018 AAAI/ACM Conference on AI, Ethics, and Society, New Orleans, LA, United States.
   Google Scholar