Residual Normality Assumption and the Estimation of Multiple Membership Random Effects Models

References

• Aaronson, D., Barrow, L., & Sander, W. (2007). Teacher and student achievement in the Chicago public schools. Journal of Labor Economics, 25(1), 95–135. doi:10.1086/508733
   Web of Science ®Google Scholar
• Bell, B. A., Morgan, G. B., Schoeneberger, J. A., Kromrey, J. D., & Ferron, J. M. (2014). How low can you go? An investigation of the influence of sample size and model complexity on point and interval estimates in two-level linear models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 10, 1–11. doi:10.1027/1614-2241/a000062
   Web of Science ®Google Scholar
• Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31(2), 144–152. doi:10.1111/j.2044-8317.1978.tb00581.x
   Web of Science ®Google Scholar
• Browne, W. J., & Draper, D. (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1(3), 473–514. doi:10.1214/06-BA117
   Web of Science ®Google Scholar
• Carpenter, J. R., Goldstein, H., & Rasbash, J. (2003). A novel bootstrap procedure for assessing the relationship between class size and achievement. Journal of the Royal Statistical Society: Series C (Applied Statistics)), 52(4), 431–443. doi:10.1111/1467-9876.00415
   Web of Science ®Google Scholar
• Chung, H. (2009). The impact of ignoring multiple-membership data structures (Doctoral dissertation). Retrieved from https://repositories.lib.utexas.edu/handle/2152/11672
   Google Scholar
• Chung, H., & Beretvas, S. N. (2012). The impact of ignoring multiple membership data structures in multilevel models. British Journal of Mathematical and Statistical Psychology, 65(2), 185–200. doi:10.1111/j.2044-8317.2011.02023.x
   PubMed Web of Science ®Google Scholar
• Clarke, P. (2008). When can group level clustering be ignored? Multilevel models versus single-level models with sparse data. Journal of Epidemiology and Community Health, 62(8), 752–758. doi:10.1136/jech.2007.060798
   PubMed Web of Science ®Google Scholar
• Clarke, P., & Wheaton, B. (2007). Addressing data sparseness in contextual population research: Using cluster analysis to create synthetic neighborhoods. Sociological Methods and Research, 35(3), 311–351. doi:10.1177/0049124106292362
   Web of Science ®Google Scholar
• Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. doi:10.1037/1082-989X.1.1.16
   Web of Science ®Google Scholar
• Elghafghuf, A., Stryhn, H., & Waldner, C. (2014). A cross-classified and multiple membership Cox model applied to calf mortality data. Preventive Veterinary Medicine, 115(1–2), 29–38. doi:10.1016/j.prevetmed.2014.03.012
   PubMed Web of Science ®Google Scholar
• Fielding, A. (2002). Teaching groups as foci for evaluating performance in cost effectiveness of GCE advanced level provision: Some practical methodological innovations. School Effectiveness and School Improvement, 13(2), 225–246. doi:10.1076/sesi.13.2.225.3435
   Web of Science ®Google Scholar
• Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. doi:10.1037/1082-989X.9.4.466
   PubMed Web of Science ®Google Scholar
• Galindo, J. L. (2015). The impact of weights’ specifications with the multiple membership random effects model (Doctoral dissertation). Retrieved from https://repositories.lib.utexas.edu/handle/2152/31012
   Google Scholar
• Garner, C. L., & Raudenbush, S. W. (1991). Neighborhood effects on educational attainment: A multilevel analysis. Sociology of Education, 64(4), 251–262. doi:10.2307/2112706
   Web of Science ®Google Scholar
• Goldstein, H. (2003). Multilevel modelling of educational data. In D. Courgeau (Ed.), Methodology and epistemology of multilevel analysis (pp. 25–42). New York, NY: Springer.
   Google Scholar
• Goldstein, H. (2011). Bootstrapping in multilevel models. In J. J. Hox & J. K. Roberts (Eds.), Handbook of advanced multilevel analysis (pp. 163–171). New York: Routledge.
   Google Scholar
• Grady, M. W. (2010). Modeling achievement in the presence of student mobility: A growth curve model for multiple membership data (Doctoral dissertation). Retrieved from https://repositories.lib.utexas.edu/handle/2152/ETD-UT-2010-08-1632
   Google Scholar
• Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87. doi:10.3102/0162373707299706
   Web of Science ®Google Scholar
• Hill, P. W., & Goldstein, H. (1998). Multilevel modelling of educational data with cross-classification and missing identification of units. Journal of Education and Behavioral Statistics, 23(2), 117–128. doi:10.3102/10769986023002117
   Web of Science ®Google Scholar
• Hox, J. J. (1998). Multilevel modeling: When and why. In I. Balderjahn, R. Mathar, & M. Schader (Eds.), Classification, data analysis, and data highways (pp. 147–154). Berlin: Springer.
   Google Scholar
• Hox, J. J., Maas, C. J., & Brinkhuis, M. J. (2010). The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica, 64(2), 157–170. doi:10.1111/j.1467-9574.2009.00445.x
   Web of Science ®Google Scholar
• Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability (Vol. 1, No. 1, pp. 221–233).
   Google Scholar
• Kaplan, D. (1989). A study of the sampling variability of the z-values of parameter estimates from misspecified structural equation models. Multivariate Behavioral Research, 24(1), 41–57. doi:10.1207/s15327906mbr2401_3
   PubMed Web of Science ®Google Scholar
• Kreft, I. G. G., & de Leeuw, J. (1998). Introduction to multilevel modeling. London: Sage.
   Google Scholar
• Leckie, G. (2009). The complexity of school and neighbourhood effects and movements of pupils on school differences in models of educational achievement. Journal of the Royal Statistical Society: Series A (Statistics in Society)), 172(3), 537–554. doi:10.1111/j.1467-985X.2008.00577.x
   Web of Science ®Google Scholar
• Leroux, A. J., & Beretvas, S. N. (2018). Estimation of a latent variable regression growth curve model for individuals cross-classified by clusters. Multivariate Behavioral Research, 53(2), 231–246. doi:10.1080/00273171.2017.1418654
   PubMed Web of Science ®Google Scholar
• Leyland, A. H. (2001). Spatial analysis. In A. H. Leyland & H. Goldstein (Eds.), Multilevel modelling of health statistics (pp. 143–157). Chichester, England: John Wiley & Sons, Ltd.
   Google Scholar
• Liao, H., & Chuang, A. (2004). A multilevel investigation of factors influencing employee service performance and customer outcomes. Academy of Management Journal, 47(1), 41–58. doi:10.5465/20159559
   Web of Science ®Google Scholar
• Maas, C. J. M., & Hox, J. J. (2004). Robustness issues in multilevel regression analysis. Statistica Neerlandica, 58(2), 127–137. doi:10.1046/j.0039-0402.2003.00252.x
   Web of Science ®Google Scholar
• Maas, C. J. M., & Hox, J. J. (2004). The influence of violations of assumptions on multilevel parameter estimates and their standard errors. Computational Statistics and Data Analysis, 46(3), 427–440. doi:10.1016/j.csda.2003.08.006
   Web of Science ®Google Scholar
• Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1(3), 86–92. doi:10.1027/1614-2241.1.3.86
   Google Scholar
• McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T. A., & Hamilton, L. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67–101. doi:10.3102/10769986029001067
   PubMed Web of Science ®Google Scholar
• McNeish, D. (2016). On using Bayesian methods to address small sample problems. Structural Equation Modeling: A Multidisciplinary Journal, 23(5), 750–773. doi:10.1080/10705511.2016.1186549
   Web of Science ®Google Scholar
• McNeish, D. (2017). Challenging conventional wisdom for multivariate statistical models with small samples. Review of Educational Research, 87(6), 1117–1151. doi:10.3102/0034654317727727
   Web of Science ®Google Scholar
• McNeish, D. M., & Stapleton, L. M. (2016). Modeling clustered data with very few clusters. Multivariate Behavioral Research, 51(4), 495–518. doi:10.1080/00273171.2016.1167008
   PubMed Web of Science ®Google Scholar
• McNeish, D. M., & Stapleton, L. M. (2016). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295–314. doi:10.1007/s10648-014-9287-x
   Web of Science ®Google Scholar
• Meyers, J. L., & Beretvas, S. N. (2006). The impact of inappropriate modeling of cross-classified data structures. Multivariate Behavioral Research, 41(4), 473–497. doi:10.1207/s15327906mbr4104_3
   PubMed Web of Science ®Google Scholar
• Muijs, D., & Reynolds, D. (2003). Student background and teacher effects on achievement and attain in mathematics: A longitudinal study. Educational Research and Evaluation, 9(3), 289–314. doi:10.1076/edre.9.3.289.15571
   Google Scholar
• R Core Team (2017). R: A language and environment for statistical computing (Version 3.4.1) [Software]. Available from http://www.R-project.org
   Google Scholar
• Raftery, A. E., & Lewis, S. M. (1992). One long run with diagnostics: Implementation strategies for Markov chain Monte Carlo. Statistical Science, 7(4), 493–497.
   Google Scholar
• Rasbash, J., & Browne, W. J. (2001). Modeling non-hierarchical structures. In A. H. Leyland & H. Goldstein (Eds.), Multilevel modelling of health statistics (pp. 93–105). Chichester, England: John Wiley & Sons, Ltd.
   Google Scholar
• Rasbash, J., Steele, F., Browne, W. J., & Goldstein, H. (2016). A user’s guide to MLwiN version 2.36. Bristol: Centre for Multilevel Modelling, University of Bristol.
   Google Scholar
• Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Searle, S. R., Casella, G., & McCulloch, C. E. (1992). Variance components. New York, NY: John Wiley & Sons, Inc.
   Google Scholar
• Seco, G. V., García, M. A., García, M. P. F., & Rojas, P. E. L. (2013). Multilevel bootstrap analysis with assumptions violated. Psicothema, 25, 520–528. doi:10.7334/psicothema2013.58
   PubMed Web of Science ®Google Scholar
• Snijders, T. A., & Bosker, R. J. (1994). Modeled variance in two-level models. Sociological Methods and Research, 22(3), 342–363. doi:10.1177/0049124194022003004
   Web of Science ®Google Scholar
• Snijders, T. A., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London, England: Sage.
   Google Scholar
• Spybrook, J., & Raudenbush, S. W. (2009). An examination of the precision and technical accuracy of the first wave of group-randomized trials funded by the Institute of Education Sciences. Educational Evaluation and Policy Analysis, 31(3), 298–318. doi:10.3102/0162373709339524
   Web of Science ®Google Scholar
• Timmermans, A. C., Snijders, T. A., & Bosker, R. J. (2013). In search of value added in the case of complex school effects. Educational and Psychological Measurement, 73(2), 210–228. doi:10.1177/0013164412460392
   Web of Science ®Google Scholar
• Tourangeau, K., Nord, C., Lê, T., Wallner-Allen, K., Vaden-Kiernan, N., Blaker, L., & Najarian, M. (2017). Early childhood longitudinal study, kindergarten class of 2010–11 (ECLS-K:2011) user’s manual for the ECLS-K:2011 kindergarten–second grade data file and electronic codebook, public version (NCES 2017-285). U.S. Department of Education. Washington, DC: National Center for Education Statistics.
   Google Scholar
• U.S. Department of Education, Institute of Education Services, What Works Clearinghouse (2017). What Works Clearinghouse: Procedures handbook (Version 4.0). Washington, DC. Retrieved from https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_handbook_v4.pdf
   Google Scholar
• U.S. Government Accounting Office (1994). Elementary school children: Many change schools frequently, harming their education (GAO/HEHS publication no. 94-45). Washington, DC: U.S. Government Printing Office.
   Google Scholar
• van der Leeden, R., Meijer, E., & Busing, F. M. T. A. (2008). Resampling multilevel models. In J. de Leeuw & E. Meijer (Eds.), Handbook of multilevel analysis (pp. 401–434). New York, NY: Springer-Verlag.
   Google Scholar
• van de Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M., & van Loey, N. E. (2015). Analyzing small data sets using Bayesian estimation: The case of posttraumatic stress symptoms following mechanical ventilation in burn survivors. European Journal of Psychotraumatology, 6, 1–13. doi:10.3402/ejpt.v6.25216
   Web of Science ®Google Scholar
• Verbeek, M. (2000). A guide to modern econometrics. Chichester, England: John Wiley & Sons, Ltd.
   Google Scholar
• Wang, J., Carpenter, J. R., & Kepler, M. A. (2006). Using SAS to conduct nonparametric residual bootstrap multilevel modeling with a small number of groups. Computer Methods and Programs in Biomedicine, 82(2), 130–143. doi:10.1016/j.cmpb.2006.02.006
   PubMed Web of Science ®Google Scholar
• White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica: Journal of the Econometric Society, 50(1), 1–25. doi:10.2307/1912526
   Web of Science ®Google Scholar
• Wolff Smith, L. J., & Beretvas, S. N. (2014). A comparison of procedures for handling mobility and missing level-2 identifiers in two-level data. International Journal of Quantitative Research in Education, 2(2), 153–174. doi:10.1504/IJQRE.2014.064393
   Google Scholar
• Wolff Smith, L. J., & Beretvas, S. N. (2017). A comparison of techniques for handling and assessing the influence of mobility on student achievement. The Journal of Experimental Education, 85(1), 3–23. doi:10.1080/00220973.2015.1065217
   Web of Science ®Google Scholar
• Zhang, Z., Parker, R., Charlton, C. M., & Browne, W. J. (2016). R2MLwiN: A package to run MLwiN from within R. Journal of Statistical Software, 72, 1–43. doi:10.18637/jss.v072.i10
   Web of Science ®Google Scholar