Validating Peer Review in Criminal Justice Evaluation Research: Evidence from CrimeSolutions.gov

Abstract

Peer review is the bedrock of the scientific enterprise, yet it enjoys scant validation. The federal government’s CrimeSolutions.gov initiative provides a unique opportunity to address this limitation. As part of the initiative, trained experts evaluate criminal justice evaluation research on several of the same criteria editors use to make publication decisions. Data from a sample of articles published in Social Sciences Citation Index journals were obtained from the CrimeSolutions.gov database, then used to model publication quality, operationalized as the product of the journal’s five-year impact factor and article citations per year (an article-level measure). The model explained only five percent of the variation in publication quality, raising several questions about the validity of peer review in criminal justice evaluation research.

Acknowledgments

The author thanks Thomas Feucht, Alex Piquero, Travis Pratt, and the CrimeSolutions.gov staff and partners for their assistance and feedback.

Disclosure statement

No potential conflict of interest was reported by the author.