Methodological Quality and Validity Issues in the Crime Prevention Literature

Abstract

This paper presents an assessment of the existing literature on validity and methodology relevant to crime prevention studies. Reports eligible for inclusion in the review focused on assessing the methodological quality of crime prevention evaluations. A narrative synthesis approach was used to review the included reports to examine how validity considerations are assessed and addressed in criminological impact evaluations. The reports reviewed included substantive discussions of the five types of validity, as well as discussions of interrelated issues of evaluation design, methodological quality scales, and evidence-based registries. We recommend that all crime prevention evaluations address the methodological issues discussed in this article. In addition, policymakers should consume research with a critical eye toward potential validity issues. Where valid evaluations show interventions to be promising, practitioners should make efforts to ensure fidelity in program implementation. Registries should support policymakers and practitioners in identifying and implementing evidence-based policy and programming through providing guidance on choosing interventions aligned to their priorities and settings, understanding the strengths and weaknesses of various strategies and programs and the conditions under which they are successful, and implementing programs and replicating evaluations with fidelity.

Keywords:

• Crime prevention
• methodological quality
• methodological scale
• quality rating
• registry
• validity

Acknowledgement

This project was funded by a contract with Public Safety Canada. We appreciate the guidance of Cameron McIntosh and Lucie Leonard on the project.

Disclosure Statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was funded in part by a contract from Public Safety Canada, Meta-Evaluation in Crime Prevention (No. 7216638).

Notes on contributors

Claire Morgan

Claire Morgan is a Senior Research Associate at WestEd where she conducts research and evaluation projects and provides technical assistance and capacity building to link research and practice in areas such as improving equity in educational outcomes for marginalized students, criminal justice, and international education development.

Anthony Petrosino

Anthony Petrosino is Director, Justice & Prevention Research Center, at WestEd. He is also Senior Fellow and Affiliated Faculty at George Mason University’s Center for Evidence-based Crime Policy. He won the 2018 Robert F. Boruch Award for Contributions to Public Policy from the Campbell Collaboration, the 2011 Paul Hood Award for Contributions to the Field from WestEd, and the 2003 Pro Humanitate Literary Award from the North American Child Welfare Resource Center.

David P. Farrington

David P. Farrington, O.B.E., is Emeritus Professor of Psychological Criminology at Cambridge University. He has received the Stockholm Prize in Criminology and he has been President of the American Society of Criminology. His major research interest is in developmental criminology, and he is Director of the Cambridge Study in Delinquent Development, which is a prospective longitudinal survey of over 400 London males from age 8 to age 61. In addition to 876 published journal articles and book chapters on criminological and psychological topics, he has published 117 books, monographs and government publications, and 164 shorter publications (total = 1,157).