Estimating the Maximum Likelihood Root Mean Square Error of Approximation (RMSEA) with Non-normal Data: A Monte-Carlo Study

References

• Asparouhov, T., & Muthén, B. (2005). Multivariate statistical modeling with survey data. In Proceedings of the Federal Committee on Statistical Methodology (FCSM) Research Conference. Washington, DC. Retrieved from https://nces.ed.gov/FCSM/pdf/2005FCSM_Asparouhov_Muthen_IIA.pdf
   Google Scholar
• Asparouhov, T., & Muthén, B. (2010). Simple second order chi-square correction scaled chi-square statistics (Technical appendix). Los Angeles, CA.  Retrieved from https://www.statmodel.com/download/WLSMV_new_chi21.pdf
   Google Scholar
• Bearden, W. O., Sharma, S., & Teel, J. E. (1982). Sample size effects on Chi square and other statistics used in evaluating causal models. Journal of Marketing Research, 19, 425. doi:10.2307/3151716
   Web of Science ®Google Scholar
• Bentler, P. M., & Bonett, D. D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. doi:10.1037//0033-2909.88.3.588
   Web of Science ®Google Scholar
• Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
   Google Scholar
• Breivik, E., & Olsson, U. H. (2001). Adding variables to improve fit: The effect of model size on fit assessment in LISREL. In R. Cudeck, K. G. Jöreskog, & S. Du Toit (Eds.), Structural equation modeling: Present and future: A Festschrift in honor of Karl Jöreskog (pp. 169–194). Chicago, IL: SSI International.
   Google Scholar
• Brosseau-Liard, P. E., Savalei, V., & Li, L. (2012). An investigation of the sample performance of two nonnormality corrections for RMSEA. Multivariate Behavioral Research, 47, 904–930. doi:10.1080/00273171.2012.715252
   PubMed Web of Science ®Google Scholar
• Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.
   Google Scholar
• Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., & Paxton, P. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research, 36, 462–494. doi:10.1177/0049124108314720
   PubMed Web of Science ®Google Scholar
• Chou, C.-P., Bentler, P. M., & Satorra, A. (1991). Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: A Monte Carlo study. British Journal of Mathematical and Statistical Psychology, 44, 347–357. doi:10.1111/j.2044-8317.1991.tb00966.x
   PubMed Web of Science ®Google Scholar
• Cudeck, R., & Henly, S. J. (1991). Model selection in covariance structures analysis and the “problem” of sample size: A clarification. Psychological Bulletin, 109, 512–519. doi:10.1037/0033-2909.109.3.512
   PubMed Web of Science ®Google Scholar
• Curran, P., West, S., & Finch, J. (1996). Psychological methods. The Robustness of Test Statistics to Nonnormality and Specification Error in Confirmatory Factor Analysis, 1, 16–29. doi:10.1037/1082-989X.1.1.16
   Google Scholar
• Curran, P. J., Bollen, K. A., Chen, F., Paxton, P., & Kirby, J. B. (2003). Finite sampling properties of the point estimates and confidence intervals of the RMSEA. Sociological Methods & Research, 32, 208–252. doi:10.1177/0049124103256130
   Web of Science ®Google Scholar
• D’Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (2004). Social problem solving: Theory and assessment. In E. C. Chang, T. J. D’Zurilla, & L. J. Sanna (Eds.), Social problem solving: Theory, research, and training (pp. 11–27). Washington, DC: American Psychological Association. doi:10.1037/10805-000
   Google Scholar
• D’Zurilla, T. J., Nezu, A. M., & Maydeu-Olivares, A. (2002). Manual of the social problem-solving inventory-revised. North Tonawanda, NY: Multi-Health Systems, Inc.
   Google Scholar
• Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56–83. doi:10.1080/10705519909540119
   Web of Science ®Google Scholar
• Foldnes, N., & Olsson, U. H. (2015). Correcting too much or too little? The performance of three chi-square corrections. Multivariate Behavioral Research, 50, 533–543. doi:10.1080/00273171.2015.1036964
   PubMed Web of Science ®Google Scholar
• Forero, C. G., & Maydeu-Olivares, A. (2009). Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, 14, 275–299. doi:10.1037/a0015825
   PubMed Web of Science ®Google Scholar
• Hoelter, J. W. (1983). The analysis of covariance structures: Goodness-of-fit indices. Sociological Methods & Research, 11, 325–344. doi:10.1177/0049124183011003003
   Web of Science ®Google Scholar
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118
   Web of Science ®Google Scholar
• Hu, L., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362. doi:10.1037/0033-2909.112.2.351
   PubMed Web of Science ®Google Scholar
• Jackson, D. L., Gillaspy, J. A., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6–23. doi:10.1037/a0014694
   PubMed Web of Science ®Google Scholar
• Kenny, D. A., & McCoach, D. B. (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling, 10, 333–351. doi:10.1207/S15328007SEM1003_1
   Web of Science ®Google Scholar
• Li, L., & Bentler, P. M. (2006). Robust statistical tests for evaluating the hypothesis of close fit of misspecified mean and covariance structural models (UCLA Statistics Preprint No. 506). Los Angeles, CA.
   Google Scholar
• Lomax, R. G., & Schumacker, R. E. (2004). A beginner’s guide to structural equation modeling. (3rd ed., Vol. 200). Mahwah, NJ: Lawrence Erlbaum. doi:10.4135/9781412973380.n17
   Google Scholar
• Maydeu-Olivares, A. (2017a). Assessing the size of model misfit in structural equation models. Psychometrika, 82, 533–558. doi:10.1007/s11336-016-9552-7
   Web of Science ®Google Scholar
• Maydeu-Olivares, A. (2017b). Maximum likelihood estimation of structural equation models for continuous data: Standard errors and goodness of fit. Structural Equation Modeling, 24, 383–394. doi:10.1080/10705511.2016.1269606
   Web of Science ®Google Scholar
• Maydeu-Olivares, A., Coffman, D. L., & Hartmann, W. M. (2007). Asymptotically distribution-free (ADF) interval estimation of coefficient alpha. Psychological Methods, 12, 157–176. doi:10.1037/1082-989X.12.2.157
   PubMed Web of Science ®Google Scholar
• Maydeu-Olivares, A., & D’Zurilla, T. J. (1996). A factor-analytic study of the social problem-solving inventory: An integration of theory and data. Cognitive Therapy and Research, 20, 115–133. doi:10.1007/BF02228030
   Web of Science ®Google Scholar
• Maydeu-Olivares, A., Fairchild, A. J., & Hall, A. G. (2017). Goodness of fit in item factor analysis: Effect of the number of response alternatives. Structural Equation Modeling, 24, 495–505. doi:10.1080/10705511.2017.1289816
   Web of Science ®Google Scholar
• Maydeu-Olivares, A., Shi, D., & Rosseel, Y. (2018). Assessing fit in structural equation models: A Monte-Carlo evaluation of RMSEA versus SRMR confidence intervals and tests of close fit. Structural Equation Modeling, 25, 389–402. doi:10.1080/10705511.2017.1389611
   Web of Science ®Google Scholar
• McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64–82. doi:10.1037//1082-989X.7.1.64
   PubMed Web of Science ®Google Scholar
• Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156. doi:10.1037/0033-2909.105.1.156
   Web of Science ®Google Scholar
• Moshagen, M. (2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling, 19, 86–98. doi:10.1080/10705511.2012.634724
   Web of Science ®Google Scholar
• Muthén, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171–189. doi:10.1111/j.2044-8317.1985.tb00832.x
   Web of Science ®Google Scholar
• Muthén, L. K., & Muthén, B. (2017). MPLUS 8 [Computer program]. Los Angeles, CA: Muthén & Muthén.
   Google Scholar
• Nevitt, J., & Hancock, G. R. (2000). Improving the root mean square error of approximation for nonnormal conditions in structural equation modeling. The Journal of Experimental Education, 68, 251–268. doi:10.1080/00220970009600095
   Web of Science ®Google Scholar
• Rhemtulla, M., Brosseau-Liard, P. É. E., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373. doi:10.1037/a0029315
   PubMed Web of Science ®Google Scholar
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. doi:10.18637/jss.v048.i02
   Web of Science ®Google Scholar
• Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. Von Eye & C. C. Clogg (Eds.), Latent variable analysis. Applications for developmental research (pp. 399–419). Thousand Oaks, CA: Sage.
   Google Scholar
• Savalei, V. (2010). Expected versus observed information in SEM with incomplete normal and nonnormal data. Psychological Methods, 15, 352–367. doi:10.1037/a0020143
   PubMed Web of Science ®Google Scholar
• Savalei, V. (2012). The relationship between root mean square error of approximation and model misspecification in confirmatory factor analysis models. Educational and Psychological Measurement, 72, 910–932. doi:10.1177/0013164412452564
   Web of Science ®Google Scholar
• Savalei, V. (2018). On the computation of the RMSEA and CFI from the mean-and-variance corrected test statistic with nonnormal data in SEM. Multivariate Behavioral Research, 1–11. doi:10.1080/00273171.2018.1455142
   PubMed Web of Science ®Google Scholar
• Shi, D., Lee, T., & Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79, 310–334. doi:10.1177/0013164418783530
   PubMed Web of Science ®Google Scholar
• Shi, D., Lee, T., & Terry, R. A. (2018). Revisiting the model size effect in structural equation modeling. Structural Equation Modeling, 25(1), 21–40. doi:10.1080/10705511.2017.1369088
   Web of Science ®Google Scholar
• Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173–180. doi:10.1207/s15327906mbr2502_4
   PubMed Web of Science ®Google Scholar
• Steiger, J. H., & Lind, J. C. (1980). Statistically-based tests for the number of common factors. Iowa: Paper presented at the Annual Meeting of the Psychometric Society.
   Google Scholar
• Vale, C. D., & Maurelli, V. A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465–471. doi:10.1007/BF02293687
   Web of Science ®Google Scholar
• West, S. G., Finch, J. F., & Curran, P. J. (1995). Structural equation models with nonnormal variables: Problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56–75). Thousand Oaks, CA: Sage.
   Google Scholar
• Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models. Educational and Psychological Measurement, 73, 913–934. doi:10.1177/0013164413495237
   Web of Science ®Google Scholar
• Yuan, K.-H., & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30, 165–200. doi:10.1111/0081-1750.00078
   Web of Science ®Google Scholar