Beyond Cohen's d: Alternative Effect Size Measures for Between-Subject Designs

Abstract

Given the long history of discussion of issues surrounding statistical testing and effect size indices and various attempts by the American Psychological Association and by the American Educational Research Association to encourage the reporting of effect size, most journals in education and psychology have witnessed an increase in effect size reporting since 1999. Yet, effect size was often reported in three indices, namely, the unadjusted R ², Cohen's d, and η² with a simple labeling of small, medium, or large, according to Cohen's (1969) criteria. In this article, the authors present several alternatives to Cohen's d to help researchers conceptualize effect size beyond standardized mean differences for between-subject designs with two groups. The alternative effect size estimators are organized into a typology and are empirically contrasted with Cohen's d in terms of purposes, usages, statistical properties, interpretability, and the potential for meta-analysis. Several sound alternatives are identified to supplement the reporting of Cohen's d. The article concludes with a discussion of the choice of standardizers, the importance of assumptions, and the possibility of extending sound alternative effect size indices to other research contexts.

Keywords:

• effect size
• meta-analysis
• robust
• statistical assumptions
• typology

Acknowledgments

This research was supported in part by two Maris M. Proffitt and Mary Higgins Proffitt Endowment Grants of Indiana University, awarded to C.-Y. J. Peng, and H.-M. Chiang and C.-Y. J. Peng, respectively.

Notes

(See pp. 2–8 of the Appendix, available at https://oncourse.iu.edu/access/content/user/peng/Appendix.Beyond%20Cohen%20d.Peng%2BChen.final.docx Statistical properties of these estimators [e.g., bias, precision in confidence interval (CI), potential for meta-analysis, interpretability] are summarized from publications and conference papers; they are described on pp. 12–27 of the Appendix.