Sensitivity of Fit Indices to Misspecification in Growth Curve Models

Abstract

This study investigated the sensitivity of fit indices to model misspecification in within-individual covariance structure, between-individual covariance structure, and marginal mean structure in growth curve models. Five commonly used fit indices were examined, including the likelihood ratio test statistic, root mean square error of approximation, standardized root mean square residual, comparative fit index, and Tucker-Lewis Index. The fit indices were found to have differential sensitivity to different types of misspecification in either the mean or covariance structures with severity of misspecification controlled. No fit index was always more (or less) sensitive to misspecification in the marginal mean structure relative to those in the covariance structure. Specifying the covariance structure to be saturated can substantially improve the sensitivity of fit indices to misspecification in the marginal mean structure; this result might help identify the sources of specification error in a growth curve model. An empirical example of children's growth in math achievement (Wu, West, & Hughes, 2008) was used to illustrate the results.

Notes

^a d = (χ² - df)/(N - 1), where χ² - df is the sample estimate of the noncentrality parameter.

^b r _jk is a standardized residual from the jth row and kth column of the covariance matrix.

¹There are two ways to specify the marginal mean structure. One is to estimate the intercepts of the repeated measures. The other is to estimate the means of the growth parameters. One cannot estimate the intercepts and the means of the growth parameters simultaneously. Usually, researchers freely estimate the mean growth parameters with intercepts fixed at zero, in which case marginal means are functions of mean growth parameters. To saturate the marginal mean structure, the intercepts need to be freely estimated, but the mean growth parameters must be fixed to be zero, in which case no structure is imposed on the marginal means, and the estimated marginal means will be equal to the sample means.

²In some conditions, Heywood cases occur. These represent cases in which the estimated variance for the quadratic rate or the estimated residual variance at Time 1 was negative, common when the variance component is zero or small in the population. Programs simply fix these estimates to zero (as do researchers typically). We calculated the mean and standard deviation of those parameter estimates for which Heywood cases occurred. The mean of the parameter estimates was very close to zero. For example, for the model with severe misspecification in the mean structure and N = 125, negative quadratic variances occurred in 389 cases among the 1,100 replications. However, the mean quadratic rate variance across the 1,100 replications was .005 with SD = .01.

^a df = 9.

^aThere are four types of misspecification in covariance structure: Tao22 = drop the quadratic rate variance; Tao01 = drop the covariance between intercept and linear rate; Sigma = equate the residual variances at Time 1 and Time 5; Rho = drop the autoregressive coefficient. The analyses are based on 1,100 converged replications. Values .01 are underlined.

³Because tao01 is normally freely estimated, we examined how the ANOVA results would be changed by dropping the misspecification of tao01. We conducted a second ANOVA in which the factor of type of misspecification in the covariance matrix had only three levels (tao22, sigma, and rho). The results showed that with tao01 excluded, type of misspecification in the covariance matrix explained slightly more variation in T _ML , RMSEA, CFI, and TLI but much less variation in SRMR (η² dropped from .578 to .146) compared with the results of the original analysis presented in Table 3. However, dropping tao01 did not change the conclusions regarding the main and interaction effects of type, severity of misspecification, and sample size.

⁴See ( − )′ ⁻¹( − ) in Equation 4.

⁵See ln|| − ln|S| + tr ⁻¹ S − p in Equation 4.

⁶There is a trade-off between freeing parameters in the between-individual and within-individual covariance matrices (Kwok et al., 2007). We propose to free as many parameters as possible in the between-individual covariance matrix fixing the remainder to zero. As an increasing number of parameters are freed in the between-individual covariance matrix, simpler error structures with fewer parameters will need to be entertained in the within-subjects covariance matrix or the model will not be identified.