Correcting Too Much or Too Little? The Performance of Three Chi-Square Corrections

Abstract

This simulation study investigates the performance of three test statistics, T₁, T₂, and T₃, used to evaluate structural equation model fit under non normal data conditions. T₁ is the well-known mean-adjusted statistic of Satorra and Bentler. T₂ is the mean-and-variance adjusted statistic of Sattertwaithe type where the degrees of freedom is manipulated. T₃ is a recently proposed version of T₂ that does not manipulate degrees of freedom. Discrepancies between these statistics and their nominal chi-square distribution in terms of errors of Type I and Type II are investigated. All statistics are shown to be sensitive to increasing kurtosis in the data, with Type I error rates often far off the nominal level. Under excess kurtosis true models are generally over-rejected by T₁ and under-rejected by T₂ and T₃, which have similar performance in all conditions. Under misspecification there is a loss of power with increasing kurtosis, especially for T₂ and T₃. The coefficient of variation of the nonzero eigenvalues of a certain matrix is shown to be a reliable indicator for the adequacy of these statistics.

KEYWORDS:

• Satorra-Bentler statistics
• mean and variance adjusted statistics
• asymptotic robustness
• kurtosis

Notes

Although not documented here, we have conducted simulations for models without equality constraints, but with data generated with uncorrelated dependencies. The findings from these simulations were similar in nature to the findings reported in the present paper. Specifically, we found that equality constraints violate AR as markedly as uncorrelated error dependencies. This finding does not support the claim in Savalei (2008, Table 1) that equality constraints is a less devastating violation of AR than error dependencies.