Direct Discrepancy Dynamic Fit Index Cutoffs for Arbitrary Covariance Structure Models

Abstract

Despite the popularity of traditional fit index cutoffs like RMSEA ≤ .06 and CFI ≥ .95, several studies have noted issues with overgeneralizing traditional cutoffs. Computational methods have been proposed to avoid overgeneralization by deriving cutoffs specifically tailored to the characteristics of the model being evaluated. Simulations show favorable performance of these methods; however, these methods support a narrow set of scenarios (e.g., certain models or response scales) and the interpretation of cutoffs is not always standardized, which affects empirical researchers’ ability to confidently and broadly adopt these methods to evaluate model fit. In this paper, we propose an extension to one recently developed computational method—dynamic fit index cutoffs—that (a) permits application to any covariance structure model (e.g., CFA, mediation, bifactor), (b) standardizes interpretation of cutoffs across any covariance structure model, and (c) supports normal, nonnormal, categorical, and missing data. Software is provided to facilitate implementation of the method.

Keywords:

• CFI
• cutoffs
• model fit
• RMSEA

Notes

1 Throughout this paper, we focus on RMSEA and CFI even though other fit indices exist and have been studied. This focus is motivated by a review of empirical studies by Jackson et al. (2009), which found RMSEA and CFI indices to be the most commonly reported (in 65% and 78% of studies, respectively). No other indices were reported more than 46% of the time (TLI).

2 In the context of approximate fit, it is somewhat ambiguous whether the equivalent of effect size is misspecification magnitude (e.g., omitted paths and their magnitude) or a fit index. Though we view misspecification magnitude and fit indices as separate quantities (e.g., RMSEA has parsimony corrections that incorporate information other than pure misspecification magnitude), another perspective may be that Hu and Bentler’s simulation was a sensitivity power analysis where the target quantity is the fit index (viewed as the effect size) and the known quantities are false positive rate, sensitivity, and sample size. In either case, Hu and Bentler’s fit index simulation still fits into the broader power analysis framework.

3 If MAD were set to 0.000, the focus changes from approximate fit to exact fit. Correspondingly, the Flexible Cutoff method described above can be considered a special case of 3DFI where MAD = 0.000.

Additional information

Funding

This work was supported by the Institute of Education Sciences [R305D220003].