Evaluating the relative efficiency among robust estimation methods for multilevel factor analysis with categorical data

References

• Asparouhov, T., and B. Muthén. 2007. Computationally efficient estimation of multilevel high-dimensional latent variable models. Paper presented at Joint Statistical Meeting Section on Statistics in Epidemiology, 2531–5. https://www.statmodel.com/download/JSM2007000746.pdf.
   Google Scholar
• Babakus, E., C. E. Ferguson, and K. G. Joreskog. 1987. The sensitivity of confirmatory maximum likelihood factor analysis to violations of measurement scale and distributional assumptions. Journal of Marketing Research 24 (2):222–8. doi:10.2307/3151512.
   Web of Science ®Google Scholar
• Bandalos, D. L. 2014. Relative performance of categorical diagonally weighted least squares and robust maximum likelihood estimation. Structural Equation Modeling: A Multidisciplinary Journal 21 (1):102–16. doi:10.1080/10705511.2014.859510.
   Web of Science ®Google Scholar
• Bandalos, D. L., and P. Gagné. 2012. Simulation methods in structural equation modeling. In Handbook of structural equation modeling, ed. R. H. Hoyle, 92–108. New York: Guilford Press.
   Google Scholar
• Bandalos, D. L., and W. Leite. 2013. The use of Monte Carlo studies in structural equation modeling research. In Structural equation modeling: A second course, eds. G. R. Hancock and R. O. Mueller, 2nd ed., 625–66. Charlotte, NC: Information Age Publishing.
   Google Scholar
• Cohen, J. 1988. Statistical power analysis for the behavioral sciences, 2nd ed. Mahwah, NJ: Erlbaum.
   Google Scholar
• Depaoli, S., and J. P. Clifton. 2015. A Bayesian approach to multilevel structural equation modeling with continuous and dichotomous outcomes. Structural Equation Modeling: A Multidisciplinary Journal 22 (3):327–51. doi:10.1080/10705511.2014.937849.
   Web of Science ®Google Scholar
• DeVellis, R. F. 2016. Scale development: Theory and applications, 4th ed. Thousand Oaks, CA: SAGE.
   Google Scholar
• DiStefano, C. 2002. The impact of categorization with confirmatory factor analysis. Structural Equation Modeling: A Multidisciplinary Journal 9 (3):327–46. doi:10.1207/S15328007SEM0903_2.
   Web of Science ®Google Scholar
• DiStefano, C., and B. Hess. 2005. Using confirmatory factor analysis for construct validation: An empirical review. Journal of Psychoeducational Assessment 23 (3):225–41. doi:10.1177/073428290502300303.
   Web of Science ®Google Scholar
• DiStefano, C., and G. B. Morgan. 2014. A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling: A Multidisciplinary Journal 21 (3):425–38. doi:10.1080/10705511.2014.915373.
   Web of Science ®Google Scholar
• Dolan, C. V. 1994. Factor analysis of variables with 2, 3, 5 and 7 response categories: A comparison of categorical variable estimators using simulated data. British Journal of Mathematical and Statistical Psychology 47 (2):309–26. doi:10.1111/j.2044-8317.1994.tb01039.x.
   Web of Science ®Google Scholar
• Flora, D. B., and P. J. Curran. 2004. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods 9 (4):466–91. doi:10.1037/1082-989X.9.4.466.
   PubMed Web of Science ®Google Scholar
• Forero, C. G., A. Maydeu-Olivares, and D. Gallardo-Pujol. 2009. Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS stimulation. Structural Equation Modeling: A Multidisciplinary Journal 16 (4):625–41. doi:10.1080/10705510903203573.
   Web of Science ®Google Scholar
• Fowler, F. J. 2013. Survey research methods. 5th ed. Thousand Oaks, CA: SAGE.
   Google Scholar
• Goldstein, H., and R. P. McDonald. 1988. A general model for the analysis of multilevel data. Psychometrika 53 (4):455–67. doi:10.1007/BF02294400.
   Web of Science ®Google Scholar
• Hallquist, M. N., and J. F. Wiley. 2018. MplusAutomation: An R package for facilitating large-scale latent variable analyses in Mplus. Structural Equation Modeling: A Multidisciplinary Journal 25 (4):621–38. doi:10.1080/10705511.2017.1402334.
   PubMed Web of Science ®Google Scholar
• Harwell, M. 2019. A strategy for using bias and RMSE as outcomes in Monte Carlo studies in statistics. Journal of Modern Applied Statistical Methods 17 (2):eP2938. doi:10.22237/jmasm/1551907966.
   Web of Science ®Google Scholar
• Heck, R. H., and S. L. Thomas. 2015. An introduction to multilevel modeling techniques: MLM and SEM approaches using Mplus. 3rd ed. Abingdon: Routledge. doi:10.1177/0146621609344848.
   Google Scholar
• Hedges, L. V., and E. C. Hedberg. 2007. Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis 29 (1):60–87. doi:10.3102/0162373707299706.
   Web of Science ®Google Scholar
• Holtmann, J., T. Koch, K. Lochner, and M. Eid. 2016. A comparison of ML, WLSMV, and Bayesian methods for multilevel structural equation models in small samples: A simulation study. Multivariate Behavioral Research 51 (5):661–80. doi:10.1080/00273171.2016.1208074.
   PubMed Web of Science ®Google Scholar
• Hoogland, J. J., and A. Boomsma. 1998. Robustness studies in covariance structure modeling an overview and a meta-analysis. Sociological Methods and Research 26 (3):329–67. doi:10.1177/0049124198026003003.
   Web of Science ®Google Scholar
• Hox, J., C. J. Maas, and M. J. Brinkhuis. 2010. The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica 64 (2):157–70. doi:10.1111/j.1467-9574.2009.00445.x.
   Web of Science ®Google Scholar
• Hox, J., R. van de Schoot, and S. Matthijsse. 2012. How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Methods 6 (2):87–93. doi:10.18148/srm/2012.v6i2.5033.
   Web of Science ®Google Scholar
• Hox, J. J., and C. J. Maas. 2001. The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling: A Multidisciplinary Journal 8 (2):157–74. doi:10.1207/S15328007SEM0802_1.
   Web of Science ®Google Scholar
• Hsu, H. Y. 2009. Testing the effectiveness of various commonly used fit indices for detecting misspecifications in multilevel structure equation models. Doctoral diss. Texas A&M University.
   Google Scholar
• Hsu, H.-Y., O.-M. Kwok, J. H. Lin, J. H. Lin, and S. Acosta. 2015. Detecting misspecified multilevel structural equation models with common fit indices: A Monte Carlo study. Multivariate Behavioral Research 50 (2):197–215. doi:10.1080/00273171.2014.977429.
   PubMed Web of Science ®Google Scholar
• Kim, E. S., C. Cao, Y. Wang, and D. T. Nguyen. 2017. Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal 24 (4):524–44. doi:10.1080/10705511.2017.1304822.
   Web of Science ®Google Scholar
• Li, C. H. 2016. Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods 48 (3):936–49. doi:10.3758/s13428-015-0619-7.
   PubMed Web of Science ®Google Scholar
• Maxwell, S. E., and H. D. Delaney. 2004. Designing experiments and analyzing data: A model comparison perspective. 2nd ed. Abingdon: Routledge.
   Google Scholar
• Muthén, B. 1978. Contributions to factor analysis of dichotomous variables. Psychometrika 43 (4):551–60. doi:10.1007/BF02293813.
   Web of Science ®Google Scholar
• Muthén, B. 1984. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika 49 (1):115–32. doi:10.1007/BF02294210.
   Web of Science ®Google Scholar
• Muthén, B. 1994. Multilevel covariance structure analysis. Sociological Methods & Research 22 (3):376–98. doi:10.1177/0049124194022003006.
   Web of Science ®Google Scholar
• Muthén, B., and D. Kaplan. 1985. A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology 38 (2):171–89. doi:10.1111/j.2044-8317.1985.tb00832.x.
   Web of Science ®Google Scholar
• Muthén, L., and B. Muthén. 2017. Mplus user’s guide. 8th ed. Los Angeles, CA: Muthén & Muthén.
   Google Scholar
• Navruz, B. 2016. The behaviors of robust weighted least squares estimation techniques for categorical/ordinal data in multilevel CFA models. Doctoral diss. Texas A&M University.
   Google Scholar
• Padgett, R. N. 2019. ML-CFA Monte Carlo simulation results. Texas Data Repository Dataverse. doi:10.18738/T8/VPAH7O.
   Google Scholar
• Padgett, R. N., and G. B. Morgan. 2019. ML-CFA fit: Online supplemental material. https://noah-padgett.github.io/mcfa-fit/
   Google Scholar
• Padgett, R. N., and G. B. Morgan. 2021. Multilevel CFA with ordered categorical data: A simulation study comparing fit indices across robust estimation methods. Structural Equation Modeling: A Multidisciplinary Journal 28 (1):51–8. doi:10.1080/10705511.2020.1759426.
   Web of Science ®Google Scholar
• Paek, I., M. Cui, N. Öztürk Gübeş, and Y. Yang. 2018. Estimation of an IRT model by Mplus for dichotomously scored responses under different estimation methods. Educational and Psychological Measurement 78 (4):569–88. doi:10.1177/0013164417715738.
   PubMed Web of Science ®Google Scholar
• R Core Team. 2018. R: A language and environment for statistical computing. https://www.r-project.org/.
   Google Scholar
• Rabe-Hesketh, S., A. Skrondal, and X. Zheng. 2007. Multilevel structural equation modeling. In Handbook of latent variable and related models, ed. S.-Y. Lee, vol. 1, 209–27. Amsterdam, The Netherlands: Elsevier B.V.
   Google Scholar
• Raudenbush, S. W., and A. S. Bryk. 2002. Hierarchical linear models: Applications and data analysis methods. 2nd ed. Thousand Oaks, CA: SAGE.
   Google Scholar
• Renbarger, R. L., R. N. Padgett, R. G. Cowden, K. Govender, M. Z. Yilmaz, L. M. Scott, A. V. Makhnach, J. S. Novotny, G. Nugent, L. Rosenbaum, et al. 2020. Culturally relevant resilience: A psychometric meta-analysis of the child and youth resilience measure (cyrm). Journal of Research on Adolescence: The Official Journal of the Society for Research on Adolescence 30 (4):896–912. doi:10.1111/jora.12569.
   PubMed Web of Science ®Google Scholar
• Rhemtulla, M., P. É. Brosseau-Liard, and V. Savalei. 2012. When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods 17 (3):354–73. doi:10.1037/a0029315.
   PubMed Web of Science ®Google Scholar
• Ryu, E., and S. G. West. 2009. Level-specific evaluation of model fit in multilevel structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal 16 (4):583–601. doi:10.1080/10705510903203466.
   Web of Science ®Google Scholar
• Satorra, A., and P. M. Bentler. 1994. Corrections to test statistics and standard errors in covariance structure analysis. In Latent variables analysis: Applications for developmental research, eds. A. von Eye and C. C. Clogg, 399–419. Thousand Oaks, CA: SAGE.
   Google Scholar
• Stapleton, L. M. 2013. Multilevel structural equation modeling with complex sample data. In Structural equation modeling: A second course, eds. G. R. Hancock and R. O. Mueller, 2nd ed., 521–62. Charlotte, NC: Information Age Publishing.
   Google Scholar
• Stapleton, L. M., J. S. Yang, and G. R. Hancock. 2016. Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics 41 (5):481–520. doi:10.3102/1076998616646200.
   Web of Science ®Google Scholar
• van de Schoot, R., and M. Miocevic. 2020. Small sample size solutions: A guide for applied researchers and practitioners. Abingdon: Routledge.
   Google Scholar
• van Erp, S., and W. Browne. 2020. Bayesian multilevel structural equation modeling: An investigation into robust prior distributions. OSF Preprint :1–45. doi:10.31219/osf.io/fzyst.
   Google Scholar
• Wu, J.-Y., and O-m Kwok. 2012. Using SEM to analyze complex survey data: A comparison between design-based single-level and model-based multilevel approaches. Structural Equation Modeling: A Multidisciplinary Journal 19 (1):16–35. doi:10.1080/10705511.2012.634703.
   Web of Science ®Google Scholar
• Yang-Wallentin, F., K. G. Jöreskog, and H. Luo. 2010. Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling: A Multidisciplinary Journal 17 (3):392–423. doi:10.1080/10705511.2010.489003.
   Web of Science ®Google Scholar