Meta-Analysis in Criminology and Criminal Justice: Challenging the Paradigm and Charting a New Path Forward

References

• Aloe, A. M. (2014). An empirical investigation of partial effect sizes in meta-analysis of correlational data. The Journal of General Psychology, 141(1), 47–64. doi:10.1080/00221309.2013.853021
   PubMed Web of Science ®Google Scholar
• Aloe, A. M. (2015). Inaccuracy of regression results in replacing bivariate correlations. Research Synthesis Methods, 6(1), 21–27. doi:10.1002/jrsm.1126
   PubMed Web of Science ®Google Scholar
• Aloe, A. M., Tanner-Smith, E. E., Becker, B. J., & Wilson, D. B. (2016). Synthesizing bivariate and partial effect sizes. Campbell Systematic Reviews, 12(1), 1–9. doi:10.4073/cmpn.2016.2
   Google Scholar
• Aloe, A. M., & Thompson, C. G. (2013). The synthesis of partial effect sizes. Journal of the Society for Social Work and Research, 4(4), 390–405. doi:10.5243/jsswr.2013.24
   Google Scholar
• Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., & Cullen, F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta‐analysis. Criminology, 28(3), 369–404. doi:10.1111/j.1745-9125.1990.tb01330.x
   Web of Science ®Google Scholar
• Armelius, B. A., & Andreassen, T. H. (2007). Cognitive-behavioral treatment for antisocial behavior in youth residential treatment. Cochrane Database of Systematic Reviews, 4, CD005650.
   Google Scholar
• Assink, M., Spruit, A., Schuts, M., Lindauer, R., van der Put, C. E., & Stams, G. J. J. (2018). The intergenerational transmission of child maltreatment: A three-level meta-analysis. Child Abuse & Neglect, 84, 131–145. doi:10.1016/j.chiabu.2018.07.037
   PubMed Web of Science ®Google Scholar
• Assink, M., & Wibbelink, C. J. (2016). Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology, 12(3), 154–174. doi:10.20982/tqmp.12.3.p154
   Web of Science ®Google Scholar
• Augustyn, M. B., McGloin, J. M., & Pyrooz, D. C. (2019). Does gang membership pay? Illegal and legal earnings through emerging adulthood. Criminology, 57(3), 452–480. doi:10.1111/1745-9125.12208
   Web of Science ®Google Scholar
• Badenes-Ribera, L., Sánchez-Meca, J., & Longobardi, C. (2019). The relationship between internalized homophobia and intimate partner violence in same-sex relationships: A meta-analysis. Trauma, Violence, & Abuse, 20(3), 331–343. doi:10.1177/1524838017708781
   PubMed Web of Science ®Google Scholar
• Baker, M. (2016). 1,500 scientists lift the lid on reproducibility: Survey sheds light on the “crisis” rocking research. Nature, 533(7604), 452–454. doi:10.1038/533452a
   PubMed Web of Science ®Google Scholar
• Barnes, J. C., TenEyck, M. F., Pratt, T. C., & Cullen, F. T. (2020). How powerful is the evidence in criminology? On whether we should fear a coming crisis of confidence. Justice Quarterly, 37(3), 383–409. doi:10.1080/07418825.2018.1495252
   Web of Science ®Google Scholar
• Batastini, A. B., King, C. M., Morgan, R. D., & McDaniel, B. (2016). Telepsychological services with criminal justice and substance abuse clients: A systematic review and meta-analysis. Psychological Services, 13(1), 20–30. doi:10.1037/ser0000042
   PubMed Web of Science ®Google Scholar
• Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 499–525). San Diego, CA: Academic Press.
   Google Scholar
• Beckstead, J. W. (2012). Isolating and examining sources of suppression and multicollinearity in multiple linear regression. Multivariate Behavioral Research, 47(2), 224–246. doi:10.1080/00273171.2012.658331
   PubMed Web of Science ®Google Scholar
• Bennett, T., Holloway, K., & Farrington, D. P. (2007). Does neighborhood watch reduce crime? A systematic review and meta-analysis. Journal of Experimental Criminology, 2(4), 437–458. doi:10.1007/s11292-006-9018-5
   Google Scholar
• Berghuis, M. (2018). Reentry programs for adult male offender recidivism and reintegration: A systematic review and meta-analysis. International Journal of Offender Therapy and Comparative Criminology, 62(14), 4655–4676. doi:10.1177/0306624X18778448
   PubMed Web of Science ®Google Scholar
• Berk, R. (2007). Statistical inference and meta-analysis. Journal of Experimental Criminology, 3(3), 247–270. doi:10.1007/s11292-007-9036-y
   Google Scholar
• Bhandari, M., Morrow, F., Kulkarni, A. V., & Tornetta, I. I. I., P. (2001). Meta-analyses in orthopaedic surgery: A systematic review of their methodologies. The Journal of Bone and Joint Surgery-American Volume, 83(1), 15–24. doi:10.2106/00004623-200101000-00003
   Google Scholar
• Bøg, M., Filges, T., Brännström, L., Jørgensen, A. M. K., & Fredrikksson, M. K. (2017). 12‐step programs for reducing illicit drug use. Campbell Systematic Reviews, 13(1), 1–149. doi:10.4073/csr.2017.2
   Google Scholar
• Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Bollen, K., Cacioppo, J. T., Kaplan, R., Krosnick, J., and Olds, J. L. (2015). Social, behavioral, and economic sciences perspectives on robust and reliable science. Arlington, VA: National Science Foundation.
   Google Scholar
• Braga, A. A., Papachristos, A. V., & Hureau, D. M. (2014). The effects of hot spots policing on crime: An updated systematic review and meta-analysis. Justice Quarterly, 31(4), 633–663. doi:10.1080/07418825.2012.673632
   Web of Science ®Google Scholar
• Braga, A. A., & Weisburd, D. L. (2012). The effects of focused deterrence strategies on crime: A systematic review and meta-analysis of the empirical evidence. Journal of Research in Crime and Delinquency, 49(3), 323–358. doi:10.1177/0022427811419368
   Web of Science ®Google Scholar
• Bryk, A. S., Raudenbush, S. W., & Congdon, R. T. (2013). HLM 7.01 for Windows. Skokie, IL: Scientific Software International.
   Google Scholar
• Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews. Neuroscience, 14(5), 365–376. doi:10.1038/nrn3475
   PubMed Web of Science ®Google Scholar
• Chen, X. J., Leith, H., Aarø, L. E., Manger, T., & Gold, C. (2016). Music therapy for improving mental health problems of offenders in correctional settings: Systematic review and meta-analysis. Journal of Experimental Criminology, 12(2), 209–228. doi:10.1007/s11292-015-9250-y
   Web of Science ®Google Scholar
• Cheung, S. F., & Chan, D. K. S. (2004). Dependent effect sizes in meta-analysis: Incorporating the degree of interdependence. The Journal of Applied Psychology, 89(5), 780–791. doi:10.1037/0021-9010.89.5.780
   PubMed Web of Science ®Google Scholar
• Cheung, S. F., & Chan, D. K. S. (2008). Dependent correlations in meta-analysis: The case of heterogeneous dependence. Educational and Psychological Measurement, 68, 760–777. doi:10.1177/0013164408315263
   Web of Science ®Google Scholar
• Clear, T. R. (2001). Has academic criminal justice come of age? ACJS Presidential Address Washington, DC, April 2001. Justice Quarterly, 18(4), 709–726. doi:10.1080/07418820100095071
   Web of Science ®Google Scholar
• Collins, R. E. (2017). Applying meta-analysis to criminological research: A step-by-step approach. SAGE research methods cases. Thousand Oaks, CA: Sage.
   Google Scholar
• Cooper, H. (2017). Research synthesis and meta-analysis: A step-by-step approach (5th ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Cullen, F. T. (2005). The twelve people who saved rehabilitation: How the science of criminology made a difference: The American Society of Criminology 2004 Presidential Address. Criminology, 43(1), 1–42. doi:10.1111/j.0011-1348.2005.00001.x
   Web of Science ®Google Scholar
• Cullen, F. T. (2013). Five reasons why I agree with Gil Geis: Publish with your students. Western Criminology Review, 14, 74–79.
   Google Scholar
• Delaney, A., Bagshaw, S. M., Ferland, A., Manns, B., Laupland, K. B., & Doig, C. J. (2005). A systematic evaluation of the quality of meta-analyses in the critical care literature. Critical Care (London, England), 9(5), R575–R582.
   PubMed Web of Science ®Google Scholar
• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review. Journal of Criminal Justice Education, 24(2), 218–234. doi:10.1080/10511253.2012.730617
   Google Scholar
• Diekmann, A. (2007). Not the first digit! Using Benford’s Law to detect fraudulent scientific data. Journal of Applied Statistics, 34(3), 321–329. doi:10.1080/02664760601004940
   Web of Science ®Google Scholar
• Dijkman, B. G., Abouali, J. A., Kooistra, B. W., Conter, H. J., Poolman, R. W., Kulkarni, A. V., … Bhandari, M. (2010). Twenty years of meta-analyses in orthopaedic surgery: Has quality kept up with quantity? The Journal of Bone and Joint Surgery. American Volume, 92(1), 48–57. doi:10.2106/JBJS.I.00251
   Google Scholar
• Druyts, E., Thorlund, K., Humphreys, S., Lion, M., Cooper, C. L., & Mills, E. J. (2013). Interpreting discordant indirect and multiple treatment comparison meta-analyses: An evaluation of direct acting antivirals for chronic hepatitis C infection. Clinical Epidemiology, 5, 173–183. doi:10.2147/CLEP.S44273
   Google Scholar
• Earp, B. D., & Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6, 621–611. doi:10.3389/fpsyg.2015.00621
   PubMed Web of Science ®Google Scholar
• Evers, T. J., Ogloff, J. R., Trounson, J. S., & Pfeifer, J. E. (2020). Well-being interventions for correctional officers in a prison setting: A review and meta-analysis. Criminal Justice and Behavior, 47(1), 3–21. doi:10.1177/0093854819869975
   Web of Science ®Google Scholar
• Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist, 33(5), 517–517. doi:10.1037/0003-066X.33.5.517.a
   Web of Science ®Google Scholar
• Eysenck, H. J. (1994). Systematic reviews: Meta-analysis and its problems. BMJ (Clinical Research ed.), 309(6957), 789–792. doi:10.1136/bmj.309.6957.789
   PubMedGoogle Scholar
• Farrington, D. P., & Welsh, B. C. (2003). Family-based prevention of offending: A meta-analysis. Australian & New Zealand Journal of Criminology, 36(2), 127–151. doi:10.1375/acri.36.2.127
   Web of Science ®Google Scholar
• Farrington, D. P., & Welsh, B. C. (2008). Saving children from a life of crime: Early risk factors and effective interventions. New York: Oxford University Press.
   Google Scholar
• Ferguson, C. J. (2015). Pay no attention to that data behind the curtain: On angry birds, happy children, scholarly squabbles, publication bias, and why betas rule metas. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 10(5), 683–691. doi:10.1177/1745691615593353
   PubMedGoogle Scholar
• Fleming, P. S., Koletsi, D., & Pandis, N. (2014). Blinded by PRISMA: Are systematic reviewers focusing on PRISMA and ignoring other guidelines? PLoS One, 9(5), e96407. doi:10.1371/journal.pone.0096407
   PubMed Web of Science ®Google Scholar
• Forrest, W., Hay, C., Widdowson, A. O., & Rocque, M. (2019). Development of impulsivity and risk-seeking: Implications for the dimensionality and stability of self-control. Criminology, 57(3), 512–543. doi:10.1111/1745-9125.12214
   Web of Science ®Google Scholar
• Francis, G. (2013). Replication, statistical consistency, and publication bias. Journal of Mathematical Psychology, 57(5), 153–169. doi:10.1016/j.jmp.2013.02.003
   Web of Science ®Google Scholar
• Gelman, A. (2013). Too good to be true. Slate, July 24.
   Google Scholar
• Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time (Unpublished manuscript). Retrieved from http://www.stat.columbia.edu/∼gelman/research/unpublished/forking.pdf
   Google Scholar
• Gelman, A., & Loken, E. (2014). Data-dependent analysis—A “garden of forking paths”—explains why many statistically significant comparisons don’t hold up. American Scientist, 102(6), 460–465. doi:10.1511/2014.111.460
   Web of Science ®Google Scholar
• Gies, S. V., Healy, E., & Stephenson, R. (2020). The evidence of effectiveness: Beyond the methodological standards. Justice Evaluation Journal. doi:10.1080/24751979.2020.1727296.
   Google Scholar
• Gill, J. (2014). Bayesian methods: A social and behavioral sciences approach. New York: Chapman and Hall/CRC.
   Google Scholar
• Glass, G. V. (1978). In defense of generalization. Behavioral and Brain Sciences, 1(3), 394–395. doi:10.1017/S0140525X00075610
   Web of Science ®Google Scholar
• Glass, G. V. (2015). Meta-analysis at middle age: A personal history. Research Synthesis Methods, 6(3), 221–231. doi:10.1002/jrsm.1133
   PubMed Web of Science ®Google Scholar
• Gleser, L. J., & Olkin, I. (2009). Stochastically dependent effect sizes. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (pp. 357–376). New York: Russell Sage Foundation.
   Google Scholar
• Greenland, S. (1994). Quality scores are useless and potentially misleading. American Journal of Epidemiology, 140(3), 300–302. doi:10.1093/oxfordjournals.aje.a117250
   Web of Science ®Google Scholar
• Hanson, R. K., & Bussiere, M. T. (1998). Predicting relapse: A meta-analysis of sexual offender recidivism studies. Journal of Consulting and Clinical Psychology, 66(2), 348–362. doi:10.1037/0022-006X.66.2.348
   PubMed Web of Science ®Google Scholar
• Hanushek, E. A., & Jackson, J. E. (1977). Statistical methods for social scientists. New York: Academic Press.
   Google Scholar
• Harms, C., Genau, H. A., Meschede, C., & Beauducel, A. (2018). Does it actually feel right? A replication attempt of the rounded price effect. Royal Society Open Science, 5(4), 171127. doi:10.1098/rsos.171127
   Web of Science ®Google Scholar
• Hartog, C. S., Skupin, H., Natanson, C., Sun, J., & Reinhart, K. (2012). Systematic analysis of hydroxyethyl starch (HES) reviews: Proliferation of low-quality reviews overwhelms the results of well-performed meta-analyses. Intensive Care Medicine, 38(8), 1258–1271. doi:10.1007/s00134-012-2614-0
   Web of Science ®Google Scholar
• Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta‐regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. doi:10.1002/jrsm.5
   PubMed Web of Science ®Google Scholar
• Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ (Clinical Research ed.), 327(7414), 557–560. doi:10.1136/bmj.327.7414.557
   PubMed Web of Science ®Google Scholar
• Hoogsteder, L. M., Stams, G. J. J., Figge, M. A., Changoe, K., van Horn, J. E., Hendriks, J., & Wissink, I. B. (2015). A meta-analysis of the effectiveness of individually oriented Cognitive Behavioral Treatment (CBT) for severe aggressive behavior in adolescents. The Journal of Forensic Psychiatry & Psychology, 26(1), 22–37. doi:10.1080/14789949.2014.971851
   Web of Science ®Google Scholar
• Hox, J. J. (1995). Applied multilevel analysis. Amsterdam: T-T-Publikaties.
   Google Scholar
• Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York: Routledge.
   Google Scholar
• Hunt, M. (1997). How science takes stock: The story of meta-analysis. New York: Russell Sage Foundation.
   Google Scholar
• Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. doi:10.1371/journal.pmed.0020124
   PubMed Web of Science ®Google Scholar
• Ioannidis, J. P. A. (2016). The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Quarterly, 94(3), 485–514. doi:10.1111/1468-0009.12210
   PubMed Web of Science ®Google Scholar
• Jadad, A. R., & McQuay, H. J. (1996). Meta-analyses to evaluate analgesic interventions: A systematic qualitative review of their methodology. Journal of Clinical Epidemiology, 49(2), 235–243. doi:10.1016/0895-4356(95)00062-3
   Web of Science ®Google Scholar
• January, A. M., Casey, R. J., & Paulson, D. (2011). A meta-analysis of classroom-wide interventions to build social skills: Do they work? School Psychology Review, 40, 242–256.
   Web of Science ®Google Scholar
• Jonnson, M. R., Bird, B. M., Li, S. M., & Viljoen, J. L. (2019). The prevalence of sexual and gender minority youth in the justice system: A systematic review and meta-analysis. Criminal Justice and Behavior, 46(7), 999–1019. doi:10.1177/0093854819848803
   Web of Science ®Google Scholar
• Jørgensen, A. W., Hilden, J., & Gøtzsche, P. C. (2006). Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: Systematic review. BMJ, 333(7572), 782–787. doi:10.1136/bmj.38973.444699.0B
   PubMedGoogle Scholar
• Jussim, L., Krosnick, J. A., Stevens, S. T., & Anglin, S. M. (2019). A Social Psychological Model of Scientific Practices: Explaining Research Practices and Outlining the Potential for Successful Reforms. Psychologica Belgica, 59(1), 353–372. doi:10.5334/pb.496.
   PubMed Web of Science ®Google Scholar
• Kempf, K. L. (1993). The empirical status of Hirschi’s control theory. In F. Adler & W. S. Laufer (Eds.), New directions in criminological theory—Advances in criminological theory (Vol. 4, pp. 143–185). New Brunswick, NJ: Transaction.
   Google Scholar
• Kempf, K. L. (2019). The status of Hirschi’s social control theory after 50 years. In J. C. Oleson & B. J. Costello (Eds.), Fifty years of causes of delinquency: The criminology of Travis Hirschi—Advances in criminological theory (Vol. 25, pp. 161–208). New York: Routledge.
   Google Scholar
• Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review: An Official Journal of the Society for Personality and Social Psychology, Inc, 2(3), 196–217. doi:10.1207/s15327957pspr0203_4
   PubMedGoogle Scholar
• Kettrey, H. H., & Lipsey, M. W. (2018). The effects of specialized treatment on the recidivism of juvenile sex offenders: A systematic review and meta-analysis. Journal of Experimental Criminology, 14(3), 361–387. doi:10.1007/s11292-018-9329-3
   Web of Science ®Google Scholar
• Kleck, G., Tark, J., & Bellows, J. J. (2006). What methods are most frequently used in criminology and criminal justice? Journal of Criminal Justice, 34(2), 147–152. doi:10.1016/j.jcrimjus.2006.01.007
   Web of Science ®Google Scholar
• Koricheva, J., Gurevitch, J., & Mengersen, K. (2013). Handbook of meta-analysis in ecology and evolution. Princeton, NJ: Princeton University Press.
   Google Scholar
• Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.
   Google Scholar
• Lakens, D., Hilgard, J., & Staaks, J. (2016). On the reproducibility of meta-analyses: Six practical recommendations. BMC Psychology, 4(1), 24. doi:10.1186/s40359-016-0126-3
   PubMedGoogle Scholar
• Lam, R. W., & Kennedy, S. H. (2005). Using metaanalysis to evaluate evidence: Practical tips and traps. The Canadian Journal of Psychiatry, 50(3), 167–174. doi:10.1177/070674370505000306
   PubMed Web of Science ®Google Scholar
• Lee, Y. J., Eck, J. E., & Corsaro, N. (2016). Conclusions from the history of research into the effects of police force size on crime—1968 through 2013: A historical systematic review. Journal of Experimental Criminology, 12(3), 431–451. doi:10.1007/s11292-016-9269-8
   Web of Science ®Google Scholar
• Leo, D., Izadikhah, Z., Fein, E. C., & Forooshani, S. A. (2019). The effect of trauma on religious beliefs: A structured literature review and meta-analysis. Trauma, Violence, & Abuse. doi:10.1177/1524838019834076.
   Web of Science ®Google Scholar
• Li, J. B., Willems, Y. E., Stok, F. M., Deković, M., Bartels, M., & Finkenauer, C. (2019). Parenting and self-control across early to late adolescence: A three-level meta-analysis. Perspectives on Psychological Science, 14(6), 967–1005. doi:10.1177/1745691619863046
   Web of Science ®Google Scholar
• Lipsey, M. W. (1992). Juvenile delinquency treatment: A meta-analytic inquiry into the variability of effects. In T. D. Cook. (Eds.), Meta-analysis for explanation: A casebook (pp. 83–127). New York: Russell Sage.
   Google Scholar
• Lipsey, M. W. (2003). Those confounded moderators in meta-analysis: Good, bad, and ugly. The Annals of the American Academy of Political and Social Science, 587(1), 69–81. doi:10.1177/0002716202250791
   Web of Science ®Google Scholar
• Lipsey, M. W., & Wilson, D. B. (1998). Effective intervention for serious juvenile offenders: A synthesis of research. Thousand Oaks, CA: Sage.
   Google Scholar
• Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
   Google Scholar
• Loy, A., & Hofmann, H. (2014). HLMdiag: A suite of diagnostics for hierarchical linear models in R. Journal of Statistical Software, 56(5), 1–28. doi:10.18637/jss.v056.i05
   Web of Science ®Google Scholar
• Maalouf, F. I., Cooper, W. O., Stratton, S. M., Dudley, J. A. M., Ko, J., Banerji, A., & Patrick, S. W. (2019). Positive predictive value of administrative data for neonatal abstinence syndrome. Pediatrics, 143(1), e20174183. doi:10.1542/peds.2017-4183
   PubMed Web of Science ®Google Scholar
• Marin-Martinez, F., & Sanchez-Meca, J. (2010). Weighting by inverse variance or by sample size in random-effects meta-analysis. Educational and Psychological Measurement, 70, 56–73.
   Web of Science ®Google Scholar
• Mazerolle, L., Bennett, S., Davis, J., Sargeant, E., & Manning, M. (2013). Procedural justice and police legitimacy: A systematic review of the research evidence. Journal of Experimental Criminology, 9(3), 245–274. doi:10.1007/s11292-013-9175-2
   Web of Science ®Google Scholar
• McHugh, C. M., Corderoy, A., Ryan, C. J., Hickie, I. B., & Large, M. M. (2019). Association between suicidal ideation and suicide: Meta-analyses of odds ratios, sensitivity, specificity and positive predictive value. British Journal of Psychology Open, 5(e18), 1–12.
   Google Scholar
• McIntosh, J. E., Tan, E. S., Levendosky, A. A., & Holtzworth-Munroe, A. (2019). Mothers’ experience of intimate partner violence and subsequent offspring attachment security ages 1–5 years: A meta-analysis. Trauma, Violence, & Abuse. doi:10.1177/1524838019888560.
   Web of Science ®Google Scholar
• Meline, T. (2006). Selecting studies for systematic review: Inclusion and exclusion criteria. Contemporary Issues in Communication Science and Disorders, 33, 21–27. doi:10.1044/cicsd_33_S_21
   Google Scholar
• Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. Chicago, IL: University of Chicago Press.
   Google Scholar
• Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269. doi:10.7326/0003-4819-151-4-200908180-00135
   PubMed Web of Science ®Google Scholar
• Murray, J., Farrington, D. P., & Eisner, M. P. (2009). Drawing conclusions about causes from systematic reviews of risk factors: The Cambridge Quality Checklists. Journal of Experimental Criminology, 5(1), 1–23. doi:10.1007/s11292-008-9066-0
   Web of Science ®Google Scholar
• Myers, W., Turanovic, J. J., Lloyd, K. M., & Pratt, T. C. (2020). The victimization of LGBTQ students at school: A meta-analysis. Journal of School Violence. doi:10.1080/15388220.2020.1725530
   Web of Science ®Google Scholar
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716.
   PubMed Web of Science ®Google Scholar
• Ousey, G. C., & Kubrin, C. E. (2018). Immigration and crime: Assessing a contentious issue. Annual Review of Criminology, 1(1), 63–84. doi:10.1146/annurev-criminol-032317-092026
   Web of Science ®Google Scholar
• Özabacı, N. (2011). Cognitive behavioural therapy for violent behaviour in children and adolescents: A meta-analysis. Children and Youth Services Review, 33(10), 1989–1993. doi:10.1016/j.childyouth.2011.05.027
   Web of Science ®Google Scholar
• Page, M. J., Altman, D. G., Shamseer, L., McKenzie, J. E., Ahmadzai, N., Wolfe, D., … Moher, D. (2018). Reproducible research practices are underused in systematic reviews of biomedical interventions. Journal of Clinical Epidemiology, 94, 8–18. doi:10.1016/j.jclinepi.2017.10.017
   Web of Science ®Google Scholar
• Paternoster, R. (1987). The deterrent effect of the perceived certainty and severity of punishment: A review of the evidence and issues. Justice Quarterly, 4(2), 173–217. doi:10.1080/07418828700089271
   Google Scholar
• Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. The Journal of Applied Psychology, 90(1), 175–181. doi:10.1037/0021-9010.90.1.175
   PubMed Web of Science ®Google Scholar
• Petrosino, A. J. (1995). Specifying inclusion criteria for a meta-analysis: Lessons and illustrations from a quantitative synthesis of crime reduction experiments. Evaluation Review, 19(3), 274–293. doi:10.1177/0193841X9501900303
   Web of Science ®Google Scholar
• Piper, A., & Berle, D. (2019). The association between trauma experienced during incarceration and PTSD outcomes: A systematic review and meta-analysis. The Journal of Forensic Psychiatry & Psychology, 30(5), 854–875. doi:10.1080/14789949.2019.1639788
   Web of Science ®Google Scholar
• Piquero, A. R., Farrington, D. P., & Blumstein, A. (2003). The criminal career paradigm. Crime and Justice, 30, 359–506. doi:10.1086/652234
   Web of Science ®Google Scholar
• Piquero, A. R., Jennings, W. G., Farrington, D. P., Diamond, B., & Gonzalez, J. M. R. (2016). A meta-analysis update on the effectiveness of early self-control improvement programs to improve self-control and reduce delinquency. Journal of Experimental Criminology, 12(2), 249–264. doi:10.1007/s11292-016-9257-z
   Web of Science ®Google Scholar
• Piquero, A. R., & Weisburd, D. (2010). Handbook of quantitative criminology. New York: Springer.
   Google Scholar
• Polanin, J. R., Hennessy, E. A., & Tanner-Smith, E. E. (2017). A review of meta-analysis packages in R. Journal of Educational and Behavioral Statistics, 42(2), 206–242. doi:10.3102/1076998616674315
   Web of Science ®Google Scholar
• Pratt, T. C. (2010). Meta-analysis in criminal justice and criminology: What it is, when it’s useful, and what to watch out for. Journal of Criminal Justice Education, 21(2), 152–168. doi:10.1080/10511251003693678
   Google Scholar
• Pratt, T. C. (2012). Advancing quantitative methods in criminology and criminal justice. London, UK: Taylor and Francis.
   Google Scholar
• Pratt, T. C., & Cullen, F. T. (2000). The empirical status of Gottfredson and Hirschi’s general theory of crime: A meta‐analysis. Criminology, 38(3), 931–964. doi:10.1111/j.1745-9125.2000.tb00911.x
   Web of Science ®Google Scholar
• Pratt, T. C., & Cullen, F. T. (2005). Assessing macro-level predictors and theories of crime: A meta-analysis. Crime and Justice, 32, 373–450. doi:10.1086/655357
   Web of Science ®Google Scholar
• Pratt, T. C., Cullen, F. T., Blevins, K. R., Daigle, L. E., & Madensen, T. D. (2006). The empirical status of deterrence theory: A meta-analysis. In F. T. Cullen, J. P. Wright, & K. R. Blevins (Eds.), Taking stock: The status of criminological theory—Advances in criminological theory (Vol. 15). New Brunswick, NJ: Transaction.
   Google Scholar
• Pratt, T. C., Cullen, F. T., Sellers, C. S., Winfree, L. T., Madensen, T. D., Daigle, L. E., … Gau, J. M. (2010). The empirical status of social learning theory: A meta-analysis. Justice Quarterly, 27(6), 765–802. doi:10.1080/07418820903379610
   Web of Science ®Google Scholar
• Pratt, T. C., Turanovic, J. J., & Cullen, F. T. (2016). Revisiting the criminological consequences of exposure to fetal testosterone: A meta-analysis of the 2D:4D digit ratio. Criminology, 54(4), 587–620. doi:10.1111/1745-9125.12115
   Web of Science ®Google Scholar
• Pratt, T. C., Turanovic, J. J., Fox, K. A., & Wright, K. A. (2014). Self-control and victimization: A meta-analysis. Criminology, 52(1), 87–116. doi:10.1111/1745-9125.12030
   Web of Science ®Google Scholar
• Pyrooz, D. C., Turanovic, J. J., Decker, S. H., & Wu, J. (2016). Taking stock of the relationship between gang membership and offending: A meta-analysis. Criminal Justice and Behavior, 43(3), 365–397. doi:10.1177/0093854815605528
   Web of Science ®Google Scholar
• Quintana, D. S. (2015). From pre-registration to publication: A non-technical primer for conducting a meta-analysis to synthesize correlational data. Frontiers, 6, 1549.
   Google Scholar
• Raudenbush, S. W., & Bryk, A. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
   Google Scholar
• Reising, K., Ttofi, M. M., Farrington, D. P., & Piquero, A. R. (2019). Depression and anxiety outcomes of offending trajectories: A systematic review of prospective longitudinal studies. Journal of Criminal Justice, 62, 3–15. doi:10.1016/j.jcrimjus.2018.05.002
   Web of Science ®Google Scholar
• Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. doi:10.1037/0033-2909.86.3.638
   Web of Science ®Google Scholar
• Rothstein, H. R., & Bushman, B. J. (2015). Methodological and reporting errors in meta-analytic reviews make other meta-analysts angry: A commentary on Ferguson (2015). Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 10(5), 677–679. doi:10.1177/1745691615592235
   Google Scholar
• Saini, M. (2009). A meta-analysis of the psychological treatment of anger: Developing guidelines for evidence-based practice. Journal of the American Academy of Psychiatry and the Law, 37, 473–488.
   PubMed Web of Science ®Google Scholar
• Sakaluk, J. K., Williams, A. J., Kilshaw, R. E., & Rhyner, K. T. (2019). Evaluating the evidential value of empirically supported psychological treatments (ESTs): A meta-scientific review. Journal of Abnormal Psychology, 128(6), 500–509. doi:10.1037/abn0000421
   PubMed Web of Science ®Google Scholar
• Savage, J., & Yancey, C. (2008). The effects of media violence exposure on criminal aggression: A meta-analysis. Criminal Justice and Behavior, 35(6), 772–1136. doi:10.1177/0093854808316487
   Web of Science ®Google Scholar
• Schimmack, U. (2014). Quantifying statistical research integrity: The replicability index (Unpublished manuscript). Retrieved from https://pdfs.semanticscholar.org/5c9f/a44d1e2ffe0c48b8062fe83233d44daf7059.pdf
   Google Scholar
• Schimmack, U. (2020). The replicability index is the most powerful tool to detect publication bias in meta-analysis. Retrieved from https://replicationindex.com/01/01/the-replicability-index-is-the-most-powerful-tool-to-detect-publication-bias-in-meta-analyses/.
   Google Scholar
• Schmidt, F. L. (2015). History and development of the Schmidt-Hunter meta-analysis methods. Research Synthesis Methods, 6(3), 232–239. doi:10.1002/jrsm.1134
   PubMed Web of Science ®Google Scholar
• Schmucker, M., & Losel, F. (2015). The effects of sexual offender treatment on recidivism: An international meta-analysis of sound quality evaluations. Journal of Experimental Criminology, 11(4), 597–630. doi:10.1007/s11292-015-9241-z
   Web of Science ®Google Scholar
• Schwalbe, C. S., Gearing, R. E., MacKenzie, M. J., Brewer, K. B., & Ibrahim, R. (2012). A meta-analysis of experimental studies of diversion programs for juvenile offenders. Clinical Psychology Review, 32(1), 26–33. doi:10.1016/j.cpr.2011.10.002
   PubMed Web of Science ®Google Scholar
• Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70(1), 747–770. doi:10.1146/annurev-psych-010418-102803
   PubMed Web of Science ®Google Scholar
• Sidebottom, A., Tompson, L., Thornton, A., Bullock, K., Tilley, N., Bowers, K., & Johnson, S. D. (2018). Gating alleys to reduce crime: A meta-analysis and realist synthesis. Justice Quarterly, 35(1), 55–86. doi:10.1080/07418825.2017.1293135
   Web of Science ®Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. doi:10.1177/0956797611417632
   PubMed Web of Science ®Google Scholar
• Siontis, K. C., Hernandez-Boussard, T., & Ioannidis, J. P. (2013). Overlapping meta-analyses on the same topic: Survey of published studies. BMJ (Clinical Research ed.), 347, f4501. doi:10.1136/bmj.f4501
   Google Scholar
• Slavin, R. E. (1987). Best-evidence synthesis: Why less is more. Educational Researcher, 16(4), 15–16. doi:10.3102/0013189X016004015
   Google Scholar
• Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London: Sage.
   Google Scholar
• Spellman, B. A., Gilbert, E. A., & Corker, K. S. (2018). Open science: What, why, and how. In J. Wixted & E.‐J. Wagenmakers (Eds.), Stevens’ handbook of experimental psychology and cognitive neuroscience (4th ed., Vol. 5, 729–776). New York, NY: Wiley.
   Google Scholar
• StataCorp. (2019). Multilevel mixed-effects reference manual: Release 16. College Station, TX: Stata Press.
   Google Scholar
• Steering Group of the Campbell Collaboration. (2019). Campbell systematic reviews: Policies and guidelines. Retrieved from https://wol-prod-cdn.literatumonline.com/pb-assets/assets/18911803/Campbell%20Policies%20and%20Guidelines%20v4.pdf
   Google Scholar
• Stephens-Lewis, D., Johnson, A., Huntley, A., Gilchrist, E., McMurran, M., Henderson, J., … Gilchrist, G. (2019). Interventions to reduce intimate partner violence perpetration by men who use substances: A systematic review and meta-analysis of efficacy. Trauma, Violence, & Abuse. doi:10.1177/1524838019882357.
   Web of Science ®Google Scholar
• Sterne, J. A., Jüni, P., Schulz, K. F., Altman, D. G., Bartlett, C., & Egger, M. (2002). Statistical methods for assessing the influence of study characteristics on treatment effects in “meta‐epidemiological” research. Statistics in Medicine, 21(11), 1513–1524. doi:10.1002/sim.1184
   PubMed Web of Science ®Google Scholar
• Sullivan, C. J., McGloin, J. M., & Piquero, A. R. (2008). Modeling the deviant Y in criminology: An examination of the assumptions of censored normal regression and potential alternatives. Journal of Quantitative Criminology, 24(4), 399–421. doi:10.1007/s10940-008-9051-9
   Web of Science ®Google Scholar
• Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2(1), 85–112. doi:10.1007/s40865-016-0026-5
   Web of Science ®Google Scholar
• Tincani, M., & Travers, J. (2019). Replication research, publication bias, and applied behavior analysis. Perspectives on Behavior Science, 42(1), 59–75. doi:10.1007/s40614-019-00191-5
   PubMed Web of Science ®Google Scholar
• Tittle, C. R., Villemez, W. J., & Smith, D. A. (1978). The myth of social class and criminality: An empirical assessment of the empirical evidence. American Sociological Review, 43(5), 643–656. doi:10.2307/2094541
   Web of Science ®Google Scholar
• Turanovic, J. J., Pratt, T. C., & Piquero, A. R. (2017). Exposure to fetal testosterone, aggression, and violent behavior: A meta-analysis of the 2D: 4D digit ratio. Aggression and Violent Behavior, 33, 51–61. doi:10.1016/j.avb.2017.01.008
   Web of Science ®Google Scholar
• Turanovic, J. J., & Tasca, M. (2019). Inmates’ experiences with prison visitation. Justice Quarterly, 36(2), 287–322. doi:10.1080/07418825.2017.1385826
   Web of Science ®Google Scholar
• Turanovic, J. J., & Young, J. T. N. (2016). Violent offending and victimization in adolescence: Social network mechanisms and homophily. Criminology, 54(3), 487–519. doi:10.1111/1745-9125.12112
   Web of Science ®Google Scholar
• Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45(2), 576–594. doi:10.3758/s13428-012-0261-6
   PubMed Web of Science ®Google Scholar
• van der Put, C. E., Boekhout van Solinge, N. F., Stams, G. J., Hoeve, M., & Assink, M. (2020). Effects of awareness programs on juvenile delinquency: A three-level meta-analysis. International Journal of Offender Therapy and Comparative Criminology. doi:10.1177/0306624X20909239
   Web of Science ®Google Scholar
• Walters, G. D. (2016). Predicting recidivism with the Criminal Sentiments Scale: A meta-analysis of a putative measure of criminal thought content. Criminal Justice and Behavior, 43(9), 1159–1172. doi:10.1177/0093854816649004
   Web of Science ®Google Scholar
• Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R., … Fordwood, S. R. (2017). What five decades of research tells us about the effects of youth psychological therapy: A multilevel meta-analysis and implications for science and practice. The American Psychologist, 72(2), 79–117. doi:10.1037/a0040360
   PubMed Web of Science ®Google Scholar
• Wells, E. (2009). Uses of meta‐analysis in criminal justice research: A quantitative review. Justice Quarterly, 26(2), 268–294. doi:10.1080/07418820802119984
   Web of Science ®Google Scholar
• Welsh, B. C., & Farrington, D. P. (2009). Public area CCTV and crime prevention: An updated systematic review and meta‐analysis. Justice Quarterly, 26(4), 716–745. doi:10.1080/07418820802506206
   Web of Science ®Google Scholar
• Wickramasekera, N., Wright, J., Elsey, H., Murray, J., & Tubeuf, S. (2015). Cost of crime: A systematic review. Journal of Criminal Justice, 43(3), 218–228. doi:10.1016/j.jcrimjus.2015.04.009
   Web of Science ®Google Scholar
• Wilson, C. M., & Gerard, P. (2020). Kernel density estimation for hierarchical data. Communications in Statistics - Theory and Methods, 49(6), 1495–1512. doi:10.1080/03610926.2018.1563179
   Web of Science ®Google Scholar
• Wilson, D. B., Gottfredson, D. C., & Najaka, S. S. (2001). School-based prevention of problem behaviors: A meta-analysis. Journal of Quantitative Criminology, 17(3), 247–272. doi:10.1023/A:1011050217296
   Web of Science ®Google Scholar
• Wilson, D. B., Olaghere, A., & Gill, C. (2016). Juvenile curfew effects on criminal behavior and victimization: A Campbell Collaboration systematic review. Journal of Experimental Criminology, 12(2), 167–186. doi:10.1007/s11292-016-9258-y
   Web of Science ®Google Scholar
• Wilson, H. A. (2014). Can antisocial personality disorder be treated? A meta-analysis examining the effectiveness of treatment in reducing recidivism for individuals diagnosed with ASPD. International Journal of Forensic Mental Health, 13(1), 36–46. doi:10.1080/14999013.2014.890682
   Web of Science ®Google Scholar
• Wilson, H. A., & Hoge, R. D. (2013). The effect of youth diversion programs on recidivism: A meta-analytic review. Criminal Justice and Behavior, 40(5), 497–518. doi:10.1177/0093854812451089
   Web of Science ®Google Scholar
• Wilson, S. J., & Lipsey, M. (2006). The effects of school-based social information processing interventions on aggressive behavior, part I: Universal Programs. Campbell Systematic Reviews, 2(1), 1–42. doi:10.4073/csr.2006.5
   Google Scholar
• Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. Newbury Park, CA: Sage.
   Google Scholar
• Wolfe, S. E., & Lawson, S. G. (2020). The organizationaljustice effect among criminal justice employees: A meta-analysis. Criminology. In press. doi:10.1111/1745-9125.12251.
   Google Scholar
• Wong, J. S., & Balemba, S. (2016). Resisting during sexual assault: A meta-analysis of the effects on injury. Aggression and Violent Behavior, 28, 1–11. doi:10.1016/j.avb.2016.03.010
   Web of Science ®Google Scholar
• Wong, J. S., & Balemba, S. (2018). The effect of victim resistance on rape completion: a meta-analysis. Trauma, Violence, & Abuse, 19(3), 352–365. doi:10.1177/1524838016663934
   PubMed Web of Science ®Google Scholar
• Wong, J. S., Bouchard, J., Lee, C., & Gushue, K. (2019). Examining the effects of day reporting centers on recidivism: A meta-analysis. Journal of Offender Rehabilitation, 58(3), 240–260. doi:10.1080/10509674.2019.1583300
   Web of Science ®Google Scholar
• Wong, J. S., Bouchard, J., Gravel, J., Bouchard, M., & Morselli, C. (2016). Can at-risk youth be diverted from crime? A meta-analysis of restorative diversion programs. Criminal Justice and Behavior, 43(10), 1310–1329. doi:10.1177/0093854816640835
   Web of Science ®Google Scholar
• Wong, J. S., & Gravel, J. (2018). Do sex offenders have higher levels of testosterone? Results from a meta-analysis. Sexual Abuse: A Journal of Research and Treatment, 30(2), 147–168. doi:10.1177/1079063216637857
   PubMed Web of Science ®Google Scholar
• Wood, J. L., Wong, J. S., Gravel, J., Bouchard, M., Descormiers, K., & Morselli, C. (2016). Promises kept? A meta-analysis of gang membership prevention programs. Journal of Criminological Research, Policy and Practice, 2(2), 134–147. doi:10.1108/JCRPP-06-2015-0018
   Web of Science ®Google Scholar
• Zane, S. N., Welsh, B. C., & Mears, D. P. (2016). Juvenile transfer and the specific deterrence hypothesis: Systematic review and meta‐analysis. Criminology & Public Policy, 15(3), 901–925. doi:10.1111/1745-9133.12222
   Web of Science ®Google Scholar