Accuracy and Precision of an Effect Size and Its Variance From a Multilevel Model for Cluster Randomized Trials: A Simulation Study

REFERENCES

• Afshartous , D. Determination of sample size for multilevel model design . Paper presented at the annual meeting of the American Educational Research Association . San Francisco , CA . April .
   Google Scholar
• American Psychological Association Publications and Communications Board Working Group on Journal Article Reporting Standards . 2008 . Reporting standards for research in psychology: Why do we need them? What might they be? . American Psychologist , 63 : 839 – 851 . doi: 10.1037/0003–066X.63.9.839
   PubMed Web of Science ®Google Scholar
• Browne , W. J. and Draper , D. 2000 . Implementation and performance issues in Bayesian and likelihood fitting of multilevel models . Computational Statistics , 15 : 391 – 420 . doi: 10.1007/s001800000041
   Web of Science ®Google Scholar
• Busing , F. 1993 . Distribution characteristics of variance estimates in two-level models. Leiden , , The Netherlands : Department of Psychometrics and Research Methodology, Leiden University. . Unpublished manuscript
   Google Scholar
• Campbell , M. K. , Elbourne , D. R. and Altman , D. G. 2004 . CONSORT statement: Extension to cluster randomized trials . British Medical Journal , 328 : 702 – 708 . doi: 10.1136/bmj.328.7441.702
   PubMed Web of Science ®Google Scholar
• Carlson , D. , Borman , G. D. and Robinson , M. 2011 . A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement . Educational Evaluation and Policy Analysis , 33 : 378 – 398 . doi: 10.3102/0162373711412765
   Web of Science ®Google Scholar
• Cohen , J. 1992 . A power primer . Psychological Bulletin , 112 : 155 – 159 . doi: 10.1037/0033–2909.112.1.155
   PubMed Web of Science ®Google Scholar
• Collins , L. M. , Schafer , J. L. and Kim , L. M. 2001 . A comparison of inclusive and restrictive strategies in modern missing data procedures . Psychological Methods , 6 : 330 – 351 . doi: 10.1037//1082–989X.6.4.330
   PubMed Web of Science ®Google Scholar
• Crooks , C. V. , Scott , K. , Ellis , W. and Wolfe , D. A. 2011 . Impact of a universal school-based violence prevention program on violent delinquency: Distinctive benefits for youth with maltreatment histories . Child Abuse and Neglect , 35 : 393 – 400 . doi: 10.1016/j.chiabu.2011.03.002
   PubMed Web of Science ®Google Scholar
• Donner , A. and Klar , N. 2000 . Design and analysis of cluster randomization trials in health research London , , UK : Arnold. .
   Google Scholar
• Feingold , A. 2009 . Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as classical analysis . Psychological Methods , 14 : 43 – 53 . doi: 10.1037/a0014699
   PubMed Web of Science ®Google Scholar
• Fisher , R. A. 1971 . The design of experiments, , 9th ed. London : Macmillan. .
   Google Scholar
• Hedges , L. V. 1981 . Distribution theory for Glass's estimator of effect size and related estimators . Journal of Educational and Behavioral Statistics , 6 : 107 – 128 . doi: 10.3102/10769986006002107
   Google Scholar
• Hedges , L. V. 2007 . Effect sizes in cluster-randomized designs . Journal of Educational and Behavioral Statistics , 32 : 341 – 370 . doi: 10.3102/1076998606298043
   Web of Science ®Google Scholar
• Hedges , L. V. 2009 . “ Effect sizes in nested designs. ” . In The handbook of research synthesis, , 2nd ed. Edited by: Cooper , H. , Hedges , L. V. and Valentine , J. C. 337 – 355 . New York , NY : Russell Sage. .
   Google Scholar
• Hedges , L. V. 2011 . Effect sizes in three-level cluster-randomized experiments . Journal of Educational and Behavioral Statistics , 36 : 346 – 380 . doi: 10.3102/1076998610376617
   Web of Science ®Google Scholar
• Hedges , L. V. and Hedberg , E. 2007 . Intraclass correlations for planning group randomized experiments in rural education . Journal of Research in Rural Education , 29 : 60 – 87 . doi: 10.3102/0162373707299706
   Google Scholar
• Hox , J. J. 2002 . Applied multilevel analysis Amsterdam , , The Netherlands : TT Publishers. .
   Google Scholar
• Hox , J. and Maas , C. 2002 . “ Sample sizes for multilevel modeling. ” . In Social science methodology in the new millennium., , 2nd ed. Edited by: Blasius , J. , Hox , J. , de Leeuw , E. and Schmidt , P. Opladen , RG : Leske & Budrich Verlag (CD-ROM). . Proceeding of the Fifth International Conference on Logic and Methodology
   Google Scholar
• Keihner , A. J. , Meigs , R. , Sugerman , S. , Backman , D. , Garbolino , T. and Mitchell , P. 2011 . The Power Play! Campaign's School Ideas & Resource Kits improve determinants of fruit and vegetable intake and physical activity among fourth- and fifth-grade children . Journal of Education and Behavior , 43 : 122 – 129 . doi: 10.1016/j.jneb.2011.01.010
   PubMed Web of Science ®Google Scholar
• Longford , N. T. 1993 . Random coefficients models New York , NY : Oxford University Press. .
   Google Scholar
• Maas , C. and Hox , J. 2004 . Robustness issues in multilevel regression analysis . Statistica Neerlandica , 58 : 127 – 137 . doi: 10.1046/j.0039–0402.2003.00252.x
   Web of Science ®Google Scholar
• Marais , S. , Jordaan , E. , Viljoen , D. , Olivier , L. , de Waal , J. and Poole , C. 2011 . The effect of brief interventions on the drinking behaviour of pregnant women in a high-risk rural South African community: A cluster randomised trial . Early Child Development and Care , 181 : 463 – 474 . doi: 10.1080/03004430903450392
   Google Scholar
• Mood , A. M. , Graybill , F. A. and Boes , D. C. 1974 . Introduction to the theory of statistics New York , NY : McGraw-Hill. .
   Google Scholar
• Neuman , S. B. , Newman , E. H. and Dwyer , J. 2011 . Educational effects of vocabulary intervention on preschoolers' word knowledge and conceptual development: A cluster-randomized trial . Reading Research Quarterly , 46 : 249 – 272 . doi: 10.1598/RRQ.46.3.3
   Web of Science ®Google Scholar
• O'Mara , A. J. 2009 . Methodological and substantive applications of meta-analysis: Multilevel modelling, simulation, and the construct validation of self-concept , Doctoral thesis UK : University of Oxford. . Retrieved from http://ioe-ac.academia.edu/AlisonOMara/Papers/575529/A_comprehensive_multilevel_model_meta-analysis_of_self-concept_interventions
   Google Scholar
• Penuel , W. R. , Gallagher , L. P. and Moorthy , S. 2011 . Preparing teachers to design sequences of instruction in earth systems science: A comparison of three professional development programs . American Educational Research Journal , 48 : 996 – 1025 . doi: 10.3102/0002831211410864
   Web of Science ®Google Scholar
• Preacher , K. J. and Kelley , K. 2011 . Effect size measures for mediation models: Quantitative strategies for communicating indirect effects . Psychological Methods , 16 : 93 – 115 . doi: 10.1037/a0022658.
   PubMed Web of Science ®Google Scholar
• Randel , B. , Beesley , A. , Apthorp , H. , Clark , T. , Wang , X. , Cicchinelli , L. and Williams , J. 2011 . Classroom assessment for student learning: Impact on elementary school mathematics in the central region Evaluation report for Institute of Education Sciences. . Retrieved from http://ies.ed.gov/ncee/edlabs/regions/central/pdf/REL_20114005.pdf(Publication No. REL 20114005)
   Google Scholar
• Raudenbush , S. W. 1993 . “ Hierarchical linear models and experimental design. ” . In Applied analysis of variance in behavioral science Edited by: Edwards , L. 459 – 496 . New York , NY : Marcel Dekker. .
   Google Scholar
• Raudenbush , S. W. 1997 . Statistical analysis and optimal design for cluster randomized trials . Psychological Methods , 2 : 173 – 185 . doi: 10.1037/1082–989X.2.2.173
   Web of Science ®Google Scholar
• Raudenbush , S. W. and Bryk , A. S. 2002 . Hierarchical linear models: Applications and data analysis methods, , 2nd ed. Thousand Oaks , CA : Sage. .
   Google Scholar
• Raudenbush , S. W. and Liu , X. 2001 . Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change . Psychological Methods , 6 : 387 – 401 . doi: 10.1037/1082–989X.6.4.387
   PubMed Web of Science ®Google Scholar
• Roberts , J. K. Effect size measures for the two-level linear multilevel model . Paper presented at the annual meeting of the American Educational Research Association . San Francisco , CA . April . Retrieved from http://www.hlm-online.com/papers/hlm_effect_size.pdf
   Google Scholar
• Senk , S. L. 2002 . “ The effects of the UCSMP secondary school curriculum on students' achievement. ” . In Standards-based school mathematics curricula: What are they? What do students learn? Edited by: Senk , S. L. and Thompson , D. R. 425 – 456 . Mahwah , NJ : Erlbaum. .
   Google Scholar
• Spybrook , J. , Bloom , H. , Congdon , R. , Hill , C. , Martinez , A. and Raudenbush , S. 2011 . Optimal design for longitudinal and multilevel research: Documentation for the optimal design software version 3.0. Retrieved from www.wtgrantfoundation.org
   Google Scholar
• Stigler , M. , Perry , C. and Arora , M. 2011 . Intermediate outcomes from Project MYTRI: Mobilizing Youth for Tobacco-Related Initiatives in India . Cancer Epidemiology, Biomarkers, & Prevention , 16 : 1050 – 1056 . doi: 10.1158/1055–9965.EPI-06–0929
   Web of Science ®Google Scholar
• van Breukelen , G. , Candel , M. and Berger , M. 2007 . Relative efficiency of unequal cluster sizes for variance component estimation in cluster randomized and multicenter trials . Statistical Methods in Medical Research , 17 : 439 – 458 . doi: 10.1177/0962280206079018
   PubMed Web of Science ®Google Scholar
• White , I. A. and Thomas , J. 2005 . Standardized mean differences in individually-randomized and cluster-randomized trials, with applications to meta-analysis . Clinical Trials , 2 : 141 – 151 . doi: 10.1191/1740774505cn081oa
   PubMed Web of Science ®Google Scholar