Estimating the Maximum Likelihood Root Mean Square Error of Approximation (RMSEA) with Non-normal Data: A Monte-Carlo Study

Abstract

Recent research has provided formulae for estimating the maximum likelihood (ML) RMSEA when mean or mean and variance, corrections for non-normality are applied to the likelihood ratio test statistic. We investigate by simulation which choice of corrections provides most accurate point RMSEA estimates, confidence intervals, and p-values for a test of close fit under normality, and in the presence of non-normality. We found that, overall, any robust corrections (choices MLM, MLMV, and MLR) provide better results than ML, which assumes normality. When they err, all choices tend to suggest that the model fits more poorly than it really does. Choice MLMV (mean and variance corrections) provided the most accurate RMSEA estimates and p-values for tests of close fit results but its performance decreases as the number of variables being modeled increases.

Keywords:

• Structural equation modeling (SEM)
• RMSEA
• nonnormal data
• fit index

Notes

¹ Some exogenous variables can be dummy variables, as in regression.

² There are two versions of the mean and variance corrected statistic, one proposed by Satorra and Bentler (1994) and another proposed by Asparouhov and Muthén (2010). Here we use the latter, as it is simpler to describe. In practice, differences between both statistics are very small (Foldnes & Olsson, 2015).

³ The sample RMSEA is not an unbiased estimator of the population RMSEA when data are normally distributed. It is a consistent estimator, in other words, it will converge to the population value as sample size increases. With non-normal data, the “naïve” approach results in a quantity that does not converge to the population RMSEA as sample size increases; it converges to a different population quantity.

⁴ For mean corrected statistics, the constant c is generally printed by software programs. For mean and variance corrections it is not currently printed by software programs.

⁵ The results of this additional set of simulations is available from the authors upon request.

⁶ The correct formula was not available at the time.

Additional information

Funding

This research was supported by the National Science Foundation under Grant No. SES-1659936.