Accuracy in Parameter Estimation for the Root Mean Square Error of Approximation: Sample Size Planning for Narrow Confidence Intervals

Abstract

The root mean square error of approximation (RMSEA) is one of the most widely reported measures of misfit/fit in applications of structural equation modeling. When the RMSEA is of interest, so too should be the accompanying confidence interval. A narrow confidence interval reveals that the plausible parameter values are confined to a relatively small range at the specified level of confidence. The accuracy in parameter estimation approach to sample size planning is developed for the RMSEA so that the confidence interval for the population RMSEA will have a width whose expectation is sufficiently narrow. Analytic developments are shown to work well with a Monte Carlo simulation study. Freely available computer software is developed so that the methods discussed can be implemented. The methods are demonstrated for a repeated measures design where the way in which social relationships and initial depression influence coping strategies and later depression are examined.

Notes

¹The symbol ⇊ introduced in Equation 5 is the Phoenician letter lamd, which was a precursor to the Greek letter “Λ/λ” (lambda) and the Latin letter “L/l” (ell; Powell, 1991). Although λ and Λ are sometimes used to denote the noncentrality parameter of the χ² distribution, in general λ and Λ are more often associated with noncentrality parameters from t distributions and F distributions, respectively. Further, we use λ to denote path coefficients in a forthcoming section; the symbol ⇊ is used for the χ² noncentrality parameter to avoid potential confusion.

²The expectation of a χ²(ν, ⇊) variate is ν + ⇊, where ν is the degrees of freedom and ⇊ is the noncentrality parameter. In the present case (N – 1) ~ χ²(ν, ⇊), and thus E[(N – 1)] = ν + (N – 1)F ₀ and Equation 7 follows.

³The RMSEA is not unbiased, but its bias decreases as sample size is increased. Correspondingly, not only is the precision with which ∊ is estimated improved when sample size is increased but also the bias is reduced.

⁴Given ν, N, and 1 – α, the value of w is solely determined by , and we use w = h() to denote such relationship, where h(·) refers to a nonlinear confidence interval formation function (i.e., the confidence interval formation function discussed previously). Using the Taylor expansion to expand h() at E[], one would obtain that h() = h(E[]) + remainder (e.g., Casella & Berger, 2002, p. 241). Taking the expectation on both sides of the equation, it leads to E[h()] ≈ E[h(E[])]. Because E[h()] = E[w] according to the definition of w and E[] is a constant, it follows that E[w] ≈ h(E[]), meaning the expectation of w is approximately equal to the confidence interval width obtained based on E[].

⁵Another way to create misspecified models is to use the Cudeck-Browne procedure (Cudeck & Browne, 1992). This procedure is implemented in the function Sigma.2.SigmaStar() in the MBESS package. See the supplement available at https://repository.library.nd.edu/view/1/AIPE_RMSEA_MBR_Supplement.pdf for detailed documentation of this function and its possible use when performing Monte Carlo simulations.

⁶During the peer review of this article the effectiveness of the sem R package and optimization routine was called into question. To show that this was not specific to the sem package, we performed a Monte Carlo simulation study to obtain for Model 2c using Mplus Muthén & Muthén, 2010). The results were essentially identical with 88.10% of the 10,000 Mplus replications correctly bracketing the population value when confidence intervals for ∊ were formed (recall that the reported value here was 87.80% with 5,000 replications).