Validating Peer Review in Criminal Justice Evaluation Research: Evidence from CrimeSolutions.gov

References

• Abramo, G., D’Angelo, C. A., & Di Costa, F. (2010). Citations versus journal impact factor as proxy of quality: Could the latter ever be preferable? Scientometrics, 84, 821–833.10.1007/s11192-010-0200-1
   Web of Science ®Google Scholar
• Arms, W. Y. (2002). What are the alternatives to peer review? Quality control in scholarly publishing on the web. Journal of Electronic Publishing, 8(1). Retrieved from http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;view=text;rgn=main;idno=3336451.0008.103
   Google Scholar
• Avakame, E. (1998). How different is violence in the home? An examination of some correlates of stranger and intimate homicide. Criminology, 36, 601–632.10.1111/crim.1998.36.issue-3
   Web of Science ®Google Scholar
• Becker, J. E., Krumholz, H. M., Ben-Josef, G., & Ross, J. S. (2014). Reporting of results in clinicaltrials.gov and high-impact journals. Journal of the American Medical Association, 311, 1063–1065.
   PubMed Web of Science ®Google Scholar
• Buela-Casal, G., & Zych, I. (2012). What do the scientists think about the impact factor? Scientometrics, 92, 281–292.10.1007/s11192-012-0676-y
   Web of Science ®Google Scholar
• Callaham, M. L., Wears, R. L., & Waeckerle, J. F. (1998). Effect of attendance at a training session on peer reviewer quality and performance. Annals of Emergency Medicine, 32, 318–322.10.1016/S0196-0644(98)70007-1
   PubMed Web of Science ®Google Scholar
• Cicchetti, D. V. (1980). Reliability of reviews for the American Psychologist: A biostatistical assessment of the data. American Psychologist, 35, 300–303.10.1037/0003-066X.35.3.300
   Web of Science ®Google Scholar
• Cicchetti, D. V., & Conn, H. O. (1976). A statistical analysis of reviewer agreement and bias in evaluating medical abstracts. Yale Journal of Biology and Medicine, 49, 373–383.
   PubMed Web of Science ®Google Scholar
• Cole, S., Cole, J. R., & Simon, G. A. (1981). Chance and consensus in peer review. Science, 214, 881–886.10.1126/science.7302566
   PubMed Web of Science ®Google Scholar
• CrimeSolutions.gov. (2014a). Program review and rating from start to finish. Retrieved October 31, 2014, from https://www.crimesolutions.gov/about_starttofinish.aspx
   Google Scholar
• CrimeSolutions.gov. (2014b). Scoring instruments. Retrieved October 31, 2014, from https://www.crimesolutions.gov/about_instrument.aspx
   Google Scholar
• CrimeSolutions.gov. (2014c). FAQs. Retrieved October 31, 2014, from https://www.crimesolutions.gov/faqs.aspx
   Google Scholar
• Das Sinha, S., Sahni, P., & Nundi, S. (1999). Does exchanging comments of Indian and non-Indian reviewers improve the quality of manuscript reviews? National Medical Journal of India, 5, 210–213.
   Google Scholar
• Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).
   Google Scholar
• Elkins, M. R., Maher, C. G., Herbert, R. D., Moseley, A. M., & Sherrington, C. (2010). Correlation between the Journal Impact Factor and three other journal citation indices. Scientometrics, 85, 81–93.10.1007/s11192-010-0262-0
   Web of Science ®Google Scholar
• Fisher, M., Friedman, S. B., & Strauss, B. (1994). The effects of blinding on acceptance of research papers by peer review. Journal of the American Medical Association, 272, 143–146.10.1001/jama.1994.03520020069019
   Web of Science ®Google Scholar
• Frye v. United States, 293 F. 1013 (D.C. Cir. 1923).
   Google Scholar
• Gardner, M. J., & Bond, J. (1990). An exploratory study of statistical assessment of papers published in the British Medical Journal. Journal of the American Medical Association, 263, 1355–1357.10.1001/jama.1990.03440100061010
   PubMed Web of Science ®Google Scholar
• Garfield, E. (1998). Long-term vs. short-term impact: Does it matter? Scientist, 12, 10–12.
   Web of Science ®Google Scholar
• Garfield, E. (2005). Long-term vs. short-term journal impact, II: Cumulative impact factors. Scientist, 12, 12–13.
   Google Scholar
• Garfield, E. (2006). The history and meaning of the journal impact factor. Journal of the American Medical Association, 295, 90–93.10.1001/jama.295.1.90
   PubMed Web of Science ®Google Scholar
• Gilbert, N. G. (1977). Referencing as persuasion. Social Studies of Science, 7, 112–122.
   Google Scholar
• Godlee, F., Gale, C. R., & Martyn, C. N. (1998). Effect on the quality of peer review of blinding peer reviewers and asking them to sign their reports. Journal of the American Medical Association, 280, 237–240.10.1001/jama.280.3.237
   PubMed Web of Science ®Google Scholar
• Goodman, S. N., Berlin, J., Fletcher, S. W., & Fletcher, R. H. (1994). Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Annals of Internal Medicine, 121, 11–21.10.7326/0003-4819-121-1-199407010-00003
   PubMed Web of Science ®Google Scholar
• Goodstein, D. (2000). How science works. In Reference manual on scientific evidence (2nd ed. pp. 67–82). Washington, DC: National Academies Press.
   Google Scholar
• Herron, D. M. (2012). Is expert peer review obsolete? A model suggests that post-publication reader review may exceed the accuracy of traditional peer review. Surgical Endoscopy, 26, 2275–2280.10.1007/s00464-012-2171-1
   PubMed Web of Science ®Google Scholar
• Hoeffel, C. (1998). Journal impact factors. Allergy, 53, 1225.10.1111/all.1998.53.issue-12
   PubMed Web of Science ®Google Scholar
• Horrobin, D. F. (1990). The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association, 263, 1438–1441.10.1001/jama.1990.03440100162024
   PubMed Web of Science ®Google Scholar
• Horrobin, D. F. (1996). Peer review of grant applications: A harbinger for mediocrity in clinical research? The Lancet, 348, 1293–1295.10.1016/S0140-6736(96)08029-4
   PubMed Web of Science ®Google Scholar
• Horrobin, D. F. (2001). Something rotten at the core of science? Trends in Pharmacological Sciences, 22, 51–52.10.1016/S0165-6147(00)01618-7
   PubMed Web of Science ®Google Scholar
• Ingelfinger, F. J. (1974). Peer review in biomedical publication. The American Journal of Medicine, 56, 686–692.10.1016/0002-9343(74)90635-4
   PubMed Web of Science ®Google Scholar
• Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996). Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Controlled Clinical Trials, 17(1), 1–12.10.1016/0197-2456(95)00134-4
   PubMedGoogle Scholar
• Jefferson, T., Alderson, P., Wager, E., & Davidoff, F. (2002). Effects of editorial peer review: A systematic review. Journal of the American Medical Association, 287, 2784–2786.10.1001/jama.287.21.2784
   PubMed Web of Science ®Google Scholar
• Jefferson, T., Smith, R., Yee, Y., Drummond, M., Pratt, M., & Gale, R. (1998). Evaluating the BMJ guidelines for economic submissions. Journal of the American Medical Association, 280, 275–277.10.1001/jama.280.3.275
   PubMed Web of Science ®Google Scholar
• Justice, A. C., Cho, M. K., Winker, M. A., Berlin, J. A., & Rennie, D. (1998). Does masking author identity improve peer review quality? Journal of the American Medical Association, 280, 240–242.10.1001/jama.280.3.240
   PubMed Web of Science ®Google Scholar
• Knight, J. (2003). Negative results: Null and void. Nature, 422, 554–555.10.1038/422554a
   PubMed Web of Science ®Google Scholar
• Kronick, D. A. (1990). Peer review in 18th-century scientific journalism. Journal of the American Medical Association, 263, 1321–1322.10.1001/jama.1990.03440100021002
   PubMed Web of Science ®Google Scholar
• Laband, D., & Piette, M. (1994). The relative impact of economics journals, 1970–1990. Journal of Economic Literature, 32, 640–666.
   Web of Science ®Google Scholar
• Lee, S. Y., Lee, S., & Jun, S. H. (2010). Author and article characteristics, journal quality and citation in economic research. Applied Economics Letters, 17, 1697–1701.10.1080/13504850903120725
   Web of Science ®Google Scholar
• Lehrer, D., Leschke, J., Lhachimi, S., Vasiliu, A., & Weiffen, B. (2007). Negative results in social science. European Political Science, 6, 51–68.10.1057/palgrave.eps.2210114
   Google Scholar
• Locke, S. (1985). A difficult balance: Editorial peer review in medicine. London: Nuffield Provincial Hospitals Trust.
   Google Scholar
• Lock, S. (1988). Fraud in medicine. British Medical Journal, 296, 376–377.10.1136/bmj.296.6619.376
   PubMed Web of Science ®Google Scholar
• Maddox, J. (1992). Conflicts of interest declared. Nature, 360, 205.
   PubMed Web of Science ®Google Scholar
• Mandavilli, A. (2011). Peer review: Trial by Twitter. Nature, 469, 286–287.10.1038/469286a
   PubMed Web of Science ®Google Scholar
• McCartney, J. L. (1973). Manuscript reviewing. Sociological Quarterly, 14, 440–446.
   Google Scholar
• McNutt, R. A., Evans, A. T., Fletcher, R. H., & Fletcher, S. W. (1990). The effects of blinding on the quality of peer review. Journal of the American Medical Association, 263, 1371–1376.10.1001/jama.1990.03440100079012
   PubMed Web of Science ®Google Scholar
• Medoff, M. (2003). Article placement and market signalling. Applied Economics Letters, 10, 479–482.10.1080/1350485032000095348
   Web of Science ®Google Scholar
• Nature.com. (2006). Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment. Retrieved October 29, 2014, from http://www.nature.com/nature/peerreview/debate/nature05535.html
   Google Scholar
• Osgood, D. W., Gartner, R., & Baumer, E. P. (2012). Editors’ note. Criminology, 50(1), 1–4.10.1111/crim.2012.50.issue-1
   Web of Science ®Google Scholar
• Pasterkamp, G., Rotmans, J. I., de Kleijn, D. V. P., & Borst, C. (2007). Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles. Scientometrics, 70, 153–165.10.1007/s11192-007-0109-5
   Web of Science ®Google Scholar
• Peters, D. P., & Ceci, S. J. (1982). Peer review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5, 187–255.10.1017/S0140525X00011183
   Web of Science ®Google Scholar
• Reuters, T. (2014). Journal citation reports: Social sciences edition. New York, NY: Thomson Reuters Corporation.
   Google Scholar
• Rogers, W. H. (1993). Regression standard errors in clustered samples. Stata Technical Bulletin, 13, 19–23.
   Google Scholar
• Rosenthal, R. (1979). The ‘file-drawer problem’ and tolerance for null results. Psychological Bulletin, 86, 638–641.10.1037/0033-2909.86.3.638
   Web of Science ®Google Scholar
• Rothwell, P. M., & Martyn, C. N. (2000). Reproducibility of peer review in clinical neuroscience: Is agreement between reviewers any greater than would be expected by chance alone? Brain, 123, 1964–1969.10.1093/brain/123.9.1964
   PubMed Web of Science ®Google Scholar
• Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008). What errors do peer reviewers detect, and does training improve their ability to detect them? Journal of the Royal Society of Medicine, 101, 507–514.10.1258/jrsm.2008.080062
   PubMed Web of Science ®Google Scholar
• Schulman, K., Sulmasy, D. P., & Roney, D. (1994). Ethics, economics, and the publication policies of major medical journals. Journal of the American Medical Association, 272, 154–156.10.1001/jama.1994.03520020080023
   Web of Science ®Google Scholar
• Schwartz, J., & Ackerman, J. (2001). In search of a dependent variable: Comment on avakame, 1998. Criminology, 39, 969–980.10.1111/crim.2001.39.issue-4
   Web of Science ®Google Scholar
• Scott, W. A. (1974). Interreferee agreement on some characteristics of manuscripts submitted to the Journal of Personality and Social Psychology. American Psychologist, 29, 698–702.10.1037/h0037631
   Web of Science ®Google Scholar
• Steinmann, W. C., Lebeau, D. L., & Michaels, R. K. (1997). A survey of journal editors regarding the review process for original clinical research. Paper presented at the International Congress on Peer Review and Biomedical Publication, September 8–10, Chicago, IL.
   Google Scholar
• Strayhorn, J., McDermott, J. F., Jr., & Tanguay, P. (1993). An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry. American Journal of Psychiatry, 150, 947–952.
   PubMed Web of Science ®Google Scholar
• Stremersch, S., Verniers, I., & Verhoef, P. C. (2007). The quest for citations: Drivers of article impact. Journal of Marketing, 71, 171–193.10.1509/jmkg.71.3.171
   Web of Science ®Google Scholar
• Surowiecki, J. (2004). The wisdom of the crowds. New York, NY: Random House.
   Google Scholar
• van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers’recommendations: A randomised trial. British Medical Journal, 318, 23–27.10.1136/bmj.318.7175.23
   PubMed Web of Science ®Google Scholar
• van Rooyen, S., Godlee, F., Evans, S., Smith, R., & Black, N. (1998). Effect of blinding and unmasking on the quality of peer review. Journal of the American Medical Association, 280, 234–237.10.1001/jama.280.3.234
   PubMed Web of Science ®Google Scholar
• Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review: A randomised controlled trial. The British Journal of Psychiatry, 176, 47–51.10.1192/bjp.176.1.47
   PubMed Web of Science ®Google Scholar