Guide for calculating and interpreting effect sizes and confidence intervals in intellectual and developmental disability research studies

References

• Aberson, C. (2002). Interpreting null results: Improving presentation and conclusions with confidence intervals. Journal of Articles in Support of the Null Hypothesis, 1(3), 36–42. Retrieved from http://www.jasnh.com/
   Google Scholar
• Aberson, C. L. (2010). Applied power analysis for the behavioral sciences. New York, NY: Routledge.
   Google Scholar
• Arbuthnot, J. (1710). An argument for divine providence, taken from the constant regularity observ'd in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27, 186–190. doi: 10.1098/rstl.1710.0011
   Google Scholar
• Bax, L. (2011). MIX 2.0: Professional software for meta-analysis in Excel (Version 2.0.1.4) [Computer software]. BiostatXL. Retrieved from http://www.meta-analysis-made-easy.com/index.html
   Google Scholar
• Bax, L., Yu, L.-M., Ikeda, N., Moons, K. G. M. (2007). A systematic comparison of software dedicated to meta-analysis of causal studies. BMC Medical Research Methodology, 7, 40. doi: 10.1186/1471-2288-7-40
   PubMed Web of Science ®Google Scholar
• Bax, L., Yu, L.-M., Ikeda, N., Tsuruta, H., Moons, K. G. M. (2006). Development and validation of MIX: Comprehensive free software for meta-analysis of causal research data. BMC Medical Research Methodology, 6, 50. doi: 10.1186/1471-2288-6-50
   PubMedGoogle Scholar
• Biostat. (2006). Comprehensive Meta-Analysis (Version 2) [Computer software]. Retrieved from http://www.meta-analysis.com
   Google Scholar
• Borenstein, M., Hedges, LV., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley.
   Google Scholar
• Bower, B. (1997). Null science: Psychology's statistical status quo draws fire. Science News, 151, 356–357. doi: 10.2307/4018441
   Google Scholar
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Cohen, J. (1994). The earth is round (p <.05). American Psychologist, 49, 997–1003.
   Web of Science ®Google Scholar
• Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY: Russell Sage Foundation.
   Google Scholar
• Cumming, G. (2006, July). Understanding replication: Confidence intervals, p values, and what's likely to happen next time. Paper presented at the 7th International Conference on the Teaching of Statistics, Salvador, Brazil. Retrieved from http://www.stat.auckland.ac.nz/∼iase/publications.php?show=17
   Google Scholar
• Cumming, G. (2011). ESCI: Exploratory software for confidence intervals [Computer software]. Melbourne, Australia: La Trobe University. Retrieved from http://www.latrobe.edu.au/psy/esci
   Google Scholar
• Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge.
   Google Scholar
• Cumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532–574.
   Web of Science ®Google Scholar
• Cumming, G., Williams, J., & Fidler, F. (2004). Replication and researchers’ understanding of confidence intervals and standard error bars. Understanding Statistics, 3, 299–311. doi: 10.1207/s15328031us0304_5
   Google Scholar
• D'Andrade, R., & Dart, J. (1990). The interpretation of r versus r2 or why percent of variance accounted for is a poor measure of size of effect. Journal of Quantitative Anthropology, 2, 47–59. Retrieved from http://www.quantitativeanthropology.org/index.php?journal = QA
   Google Scholar
• Demsar, J. (2008). On the appropriateness of statistical tests in machine learning. Retrieved from http://www.site.uottawa.ca/ICML08WS/papers/J_Demsar.pdf
   Google Scholar
• Denis, D. J. (2003). Alternatives to null hypothesis significance testing. Theory and Science, 4(1), 127–156.
   Google Scholar
• Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction With Life Scale. Journal of Personality Assessment, 49, 71–75. doi: 10.1207/s15327752jpa4901_13
   PubMed Web of Science ®Google Scholar
• Dunst, C. J., Hamby, DW., & Trivette, C. M. (2007). Guidelines for calculating effect sizes for practice-based research syntheses (Winterberry Research Perspectives Vol. 1, No. 3). Asheville, NC: Winterberry Press.
   Google Scholar
• Dunst, CJ., Raab, M., & Trivette, C. M. (in press). Characteristics of naturalistic language intervention strategies. Journal of Speech-Language Pathology and Applied Behavior Analysis.
   Google Scholar
• Ellis, P. D. (2009). Effect size calculators. Retrieved from http://www.polyu.edu.hk/mm/effectsizefaqs/calculator/calculator.html
   Google Scholar
• Ellis, P. D. (2010). The essential guide to effects sizes: Statistical power, meta-analysis, and the interpretation of research results. New York, NY: Cambridge University Press.
   Google Scholar
• Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40, 532–538. doi: 10.1037/a0015808
   Google Scholar
• Fisher, R. A. (1926). The arrangement of field experiments. Journal of the Ministry of Agriculture of Great Britain, 33, 503–513. Retrieved from http://hdl.handle.net/2440/15191
   Google Scholar
• Gardner, J. F., & Carran, D. T. (2005). Attainment of personal outcomes by people with developmental disabilities. Mental Retardation, 43, 157–174. doi: 10.1352/0047-6765(2005)43[157:AOPOBP]2.0.CO;2
   PubMedGoogle Scholar
• Gray, DE., & Holden, W. J. (1992). Psycho-social well-being among the parents of children with autism. Journal of Intellectual & Developmental Disability, 18, 83–93. doi: 10.1080/07263869200034841
   Google Scholar
• Grissom, RJ., & Kim, J. J. (2012). Effect sizes for research: Univariate and multivariate applications (2nd ed.). New York, NY: Routledge Academic.
   Google Scholar
• Harrison, J., Thompson, B., & Vannest, K. J. (2009). Interpreting the evidence for effective interventions to increase the academic performance of students with ADHD: Relevance of the statistical significance controversy. Review of Educational Research, 79, 740–775. doi: 10.3102/0034654309331516
   Web of Science ®Google Scholar
• Hedges, L. V. (2008). What are effect sizes and why do we need them? Child Development Perspectives, 2, 167–171.
   Web of Science ®Google Scholar
• Hemphill, J. F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, 78–79. doi: 10.1037/0003-066X.58.1.78
   PubMed Web of Science ®Google Scholar
• Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172–177.
   Web of Science ®Google Scholar
• Hojat, M., & Xu, G. (2004). A visitor's guide to effect sizes: Statistical significance versus practical (clinical) importance of research findings. Advances in Health Sciences Education, 9, 241–249. doi: 10.1023/B:AHSE.0000038173.00909.f6
   PubMed Web of Science ®Google Scholar
• Hubbard, R., & Lindsay, R. M. (2008). Why p values are not a useful measure of evidence in statistical significance testing. Theory & Psychology, 18, 69–88. doi: 10.1177/0959354307086923
   Google Scholar
• Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746–759. doi: 10.1177/0013164496056005002
   Web of Science ®Google Scholar
• Lewis, M. A., Lewis, C. E., Leake, B., King, B. H., & Lindemann, R. (2002). The quality of health care for adults with developmental disabilities. Public Health Reports, 117, 174–184.
   PubMed Web of Science ®Google Scholar
• Lipsey, MW., & Wilson, D. B. (2001). Practical meta-analysis (Applied Social Research Methods Series Vol. 49). Thousand Oaks, CA: Sage.
   Google Scholar
• Luiz, D., Barnard, A., Knoesen, N., Kotras, N., Horrocks, S., McAlinden, P., … O'Connell, R. (2006). Griffiths Mental Development Scales – Extended Revised: Two to eight years: Administration manual. Oxford, UK: Hogrefe.
   Google Scholar
• Lyons, LC., & Morris, W. A. (2003). The Meta-analysis Calculator. Retrieved from http://www.lyonsmorris.com/ma1/index.cfm
   Google Scholar
• Mactavish, J., Schleien, S., & Tabourne, C. (1997). Patterns of family recreation in families that include children with a developmental disability. Journal of Leisure Research, 29, 21–46.
   Web of Science ®Google Scholar
• Marfo, K. (1991). The maternal directiveness theme in mother-child interaction research: Implications for early intervention. In K. Marfo (Ed.), Early intervention in transition: Current perspectives on programs for handicapped children (pp. 177–203). New York, NY: Praeger.
   Google Scholar
• Marsh, R. W. (1982). The use of serial correlation in the analysis of data from interrupted time series trials with single subjects in educational research. Educational Psychology, 2, 317–320. doi: 10.1080/0144341820020312
   Google Scholar
• McCartney, K., & Rosenthal, R. (2000). Effect size, practical importance, and social policy for children. Child Development, 71, 173–180.
   PubMed Web of Science ®Google Scholar
• McLean, J. E., & Ernest, J. M. (1998). The role of statistical significance testing in educational research. Research in the Schools, 5(2), 15–22. Retrieved from http://www.msera.org/rits.htm
   Google Scholar
• Mogie, M. (2004). In support of null hypothesis significance testing. Proceedings of the Royal Society of London: Series B. Biological Sciences, 271(Suppl. 3), S82–S84. doi: 10.1098/rsbl.2003.0105
   PubMed Web of Science ®Google Scholar
• Morgan, P. L. (2003). Null hypothesis significance testing: Philosophical and practical considerations of a statistical controversy. Exceptionality, 11, 209–221. doi: 10.1207/S15327035EX1104_2
   Google Scholar
• Nakagawa, S., & Cuthill, I. C. (2007). Effect size, confidence interval and statistical significance: A practical guide for biologists. Biological Reviews, 82, 591–605. doi: 10.1111/j.1469-185X.2007.00027.x
   PubMed Web of Science ®Google Scholar
• Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241–286. doi: 10.1006/ceps.2000.1040
   PubMed Web of Science ®Google Scholar
• Psytek. (2005). Effect Size Generator-Pro Version 3.5. Retrieved from http://www.clintools.com/products/esg/effect_size_generator.html
   Google Scholar
• Robinson, DH., & Wainer, H. (2001, December). On the past and future of null hypothesis significance testing (Research Report RR-01-24). Princeton, NJ: Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/RR-01-24-Wainer.pdf
   Google Scholar
• Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231–244). New York, NY: Russell Sage Foundation.
   Google Scholar
• Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. New York, NY: Cambridge University Press.
   Google Scholar
• Rosnow, R. L., Rosenthal, R., & Rubin, D. B. (2000). Contrasts and correlations in effect-size estimation. Psychological Science, 11, 446–453. doi: 10.1111/1467-9280.00287
   PubMed Web of Science ®Google Scholar
• Ryff, C. D., & Keyes, C. L. M. (1995). The structure of psychological well-being revisited. Journal of Personality and Social Psychology, 69, 719–727. doi: 10.1037/0022-3514.69.4.719
   PubMed Web of Science ®Google Scholar
• Ryff, C. D., & Singer, B. (1998). The contours of positive human health. Psychological Inquiry, 9, 1–28. doi: 10.1207/s15327965pli0901_1
   Web of Science ®Google Scholar
• Shadish, W. R. (2010). Commercial and freeware-careware-shareware meta-analysis software. Retrieved from http://faculty.ucmerced.edu/wshadish/Meta-Analysis%20Software.htm
   Google Scholar
• Shadish, WR., Robinson, L., & Lu, C. (1997). ES: Effect Size Calculator. Retrieved from http://www.assess.com/xcart/product.php?productid522&cat527&page51
   Google Scholar
• Shadish, WR., Robinson, L., & Lu, C. (1999). ES: A computer program and manual for effect size calculation. Retrieved from http://faculty.ucmerced.edu/wshadish/es.htm
   Google Scholar
• Shieh, G. (2010). Estimation of the simple correlation coefficient. Behavior Research Methods, 42, 906–917. doi: 10.3758/BRM.42.4.906
   PubMed Web of Science ®Google Scholar
• Skok, A., Harvey, D., & Reddihough, D. (2006). Perceived stress, perceived social support, and wellbeing among mothers of school-aged children with cerebral palsy. Journal of Intellectual & Developmental Disability, 31, 53–57. doi: 10.1080/13668250600561929
   Google Scholar
• Sparrow, SS., Cicchetti, DV., & Balla, D. A. (2005). Vineland Adaptive Behavior Scales: Second edition (Vineland-II). >Circle Pines, MN: American Guidance Service.
   Google Scholar
• Stephens, P. A., Buskirk, S. W., & del Rio, C. M. (2007). Inference in ecology and evolution. Trends in Ecology & Evolution, 22, 192–197. doi: 10.1016/j.tree.2006.12.003
   Google Scholar
• Sun, S., Pan, W., & Wang, L. L. (2010). A comprehensive review of effect size reporting and interpreting practices in academic journals in education and psychology. Journal of Educational Psychology, 102, 989–1004. doi: 10.1037/a0019507
   Web of Science ®Google Scholar
• Thompson, B. (1998). Statistical significance and effect size reporting: Portrait of a possible future. Research in the Schools, 5(2), 33–38.
   Google Scholar
• Thompson, B. (1999). Improving research clarity and usefulness with effect size indices as supplements to statistical significance tests. Exceptional Children, 65, 329–337.
   Web of Science ®Google Scholar
• Thompson, B. (2002a). “Statistical,” “practical,” and “clinical”: How many kinds of significance do counselors need to consider? Journal of Counseling & Development, 80, 64–71.
   Google Scholar
• Thompson, B. (2002b). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31(3), 25–32.
   Google Scholar
• Thompson, B. (2005). Replicability of results. In B. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science. Hoboken, NJ: Wiley.
   Google Scholar
• Thompson, B. (2007). Effect sizes, confidence intervals, and confidence intervals for effect sizes. Psychology in the Schools, 44, 423–432. doi: 10.1002/pits.20234
   Web of Science ®Google Scholar
• Vacha-Haase, T., & Thompson, B. (2004). How to estimate and interpret various effect sizes. Journal of Counseling Psychology, 51, 473–481. doi: 10.1037/0022 - 0167.51.4.473
   Web of Science ®Google Scholar
• Valentine, JC., & Cooper, H. (2003). Effect size substantive interpretation guidelines: Issues in the interpretation of effect sizes. Washington, DC: What Works Clearinghouse.
   Google Scholar
• Wainer, H. (1999). One cheer for null hypothesis significance testing. Psychological Methods, 4, 212–213. doi: 10.1037/1082-989X.4.2.212
   Web of Science ®Google Scholar
• Warren, S. F., & Kaiser, A. P. (1986). Incidental language teaching: A critical review. Journal of Speech and Hearing Disorders, 51, 291–299.
   PubMedGoogle Scholar
• Wechsler, D. (2005). Wechsler Intelligence Scale for Children – Fourth Edition: Australian Standardised Edition (WISC-IV Australian). Sydney, Australia: PsychCorp.
   Google Scholar
• Wilson, D. B. (2001). Practical meta-analysis effect size calculator [Computer software]. Retrieved from http://www.campbellcollaboration.org/resources/effect_size_input.php
   Google Scholar
• Zimet, G. D., Dahlem, N. W., Zimet, S. G., & Farley, G. K. (1988). The Multidimensional Scale of Perceived Social Support. Journal of Personality Assessment, 52, 30–41. doi: 10.1207/s15327752jpa5201_2
   Web of Science ®Google Scholar