Sweating the small stuff in educational psychology: how effect size and power reporting failed to change from 1969 to 1999, and what that means for the future of changing practicesFootnote¹

1. I would like to thank William R. Christensen II, Jason S. Gunter, and Cheryl Murdock for help with data extraction and earlier drafts of this paper. Some of this research was performed while the author was at the University of Oklahoma. Readers interested in incorporating desirable practices into their research may wish to refer to Best Practices in Quantitative Methods (Osborne, in press).

References

• Boring , E.G. 1919 . Mathematical vs. scientific importance . Psychological Bulletin , 16 : 335 – 338 .
   Google Scholar
• Cohen , J. 1962 . The statistical power of abnormal‐social psychological research: A review . Journal of Abnormal and Social Psychology , 65 : 145 – 153 .
   PubMed Web of Science ®Google Scholar
• Cohen , J. 1988 . Statistical power analysis for the behavioral sciences , (2nd ed.) , Hillsdale , NJ : Erlbaum .
   Google Scholar
• Cohen , J. 1992 . A power primer . Psychological Bulletin , 112 : 155 – 159 .
   PubMed Web of Science ®Google Scholar
• Crandall , C.S. , Olson , L.M. and Sklar , D.P. 2001 . Mortality reduction with air bag and seat belt use in head‐on passenger car collisions . American Journal of Epidemiology , 153 : 219 – 224 .
   PubMed Web of Science ®Google Scholar
• Deemer , W.L. 1947 . The power of the t test and the estimation of required sample size . Journal of Educational Psychology , 38 : 329 – 342 .
   PubMedGoogle Scholar
• Fidler , F. 2005 . From statistical significance to effect estimation: Statistical reform in psychology, medicine, and ecology Doctoral dissertation, University of Melbourne. Retrieved from http://www.botany. unimelb.edu.au/envisci/fiona/fidlerphd_aug06.pdf, 2 April 2007
   Google Scholar
• Fidler , F. and Cumming , G. 2008 . “ The new stats: Attitudes for the twenty‐first century ” . In Best practices in quantitative methods , Edited by: Osborne , J.W. Thousand Oaks , CA : Sage Publishing .
   Google Scholar
• Iramaneerat , C. , Smith , E.V. Jr. and Smith , R.M. “ An introduction to Rasch measurement ” . In Best practices in quantitative methods , Edited by: Osborne , J.W. Thousand Oaks , CA : Sage Publishing . (in press)
   Google Scholar
• Killeen , P.R. 2008 . “ Replication statistics as a replacement for significance testing, best practices in scientific decision‐making ” . In Best practices in quantitative methods , Edited by: Osborne , J. Thousand Oaks , CA : Sage Publishing .
   Google Scholar
• Osborne , J.W. and Overbay , A. 2004 . The power of outliers (and why researchers should ALWAYS check for them) . Practical Assessment, Research, and Evaluation , 9 (6) Retrieved from http://pareonline.net/getvn.asp?v=9&n=6, 2 April 2007
   Google Scholar
• Osborne , J.W. and Waters , E. 2002 . Four assumptions of multiple regression that researchers should always test . Practical Assessment, Research, and Evaluation , 8 (2) Retrieved from http://pareonline.net/getvn.asp?v=8&n=2, 2 April 2007
   Google Scholar
• Rossi , J.S. 1990 . Statistical power of psychological research: What have we gained in 20 years? . Journal of Counseling and Clinical Psychology , 58 : 646 – 656 .
   PubMed Web of Science ®Google Scholar
• Rozeboom , W.W. 1997 . “ Good science is abductive, not hypothetico‐deductive ” . In What if there were no significance tests? , Edited by: Harlow , L.L. , Mulaik , S.A. and Steiger , J.H. 335 – 392 . Mahwah , NJ : Erlbaum .
   Google Scholar
• Sedlmeier , P. and Gigerenzer , G. 1989 . Do studies of statistical power have an effect on the power of studies? . Psychological Bulletin , 105 : 309 – 316 .
   Web of Science ®Google Scholar
• Thompson , B. 1993 . The use of statistical significance tests in research: Bootstrap and other alternatives . Journal of Experimental Education , 61 : 361 – 377 .
   Web of Science ®Google Scholar
• Thompson , B. 2002a . What future quantitative social science research could look like: Confidence intervals for effect sizes . Educational Researcher , 31 (3) : 24 – 31 .
   Google Scholar
• Thompson , B. 2002b . “Statistical,” “practical,” and “clinical”: How many kinds of significance do counselors need to consider? . Journal of Counseling and Development , 80 : 64 – 71 .
   Web of Science ®Google Scholar
• Thompson , B. “ Computing and interpreting effect sizes, confidence intervals, and confidence intervals for effect sizes ” . In Best practices in quantitative methods , Edited by: Osborne , J.W. Thousand Oaks , CA : Sage Publishing . (in press)
   Google Scholar
• Tukey , J.W. 1991 . The philosophy of multiple comparisons . Statistical Science , 6 : 100 – 116 .
   Google Scholar
• Wilcox , R.R. 2005a . Introduction to robust estimation and hypothesis testing , (2nd ed.) , San Diego , CA : Academic Press .
   Google Scholar
• Wilkinson , L. and the APA Task Force on Statistical Inference . 1999 . Statistical methods in psychology journals: Guidelines and explanations . American Psychologist , 54 : 594 – 604 .
   Web of Science ®Google Scholar
• Williams , A.F. and Lund , A.K. 1986 . Seat belt use laws and occupant crash protection in the United States . American Journal of Public Health , 76 : 1438 – 1442 .
   PubMed Web of Science ®Google Scholar