On the Behavior of Fit Indices for Adjudicating Between Exploratory Structural Equation and Confirmatory Factor Analysis Models

ABSTRACT

Compared to traditional confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM) has been shown to result in less structural parameter bias when cross-loadings (CLs) are present. However, when model fit is reasonable for CFA (over ESEM), CFA should be preferred on the basis of parsimony. Using simulations, the current study examined the sensitivity of the CFI, RMSEA, and RMSEA_D in correctly adjudicating model fit between ESEM and CFA. Results showed that 1) the magnitude of structural bias was moderated by the sign of the CL, 2) constraining non-zero CL to zero resulted in incorrectly specified CFAs demonstrating good stand-alone fit but were often rejected when compared with ESEM, and 3) CFAs with negligible factor correlation bias <|.10| often failed the model equivalence test while those with non-ignorable bias >|.30| passed. This disconnect is shown to be linked to CL conditions and calls into question historically held beliefs about what constitutes an “ignorable” CL value.

KEYWORDS:

• ESEM
• CFA
• UFA
• fit indices
• structural bias

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/15366367.2023.2264605

Notes

1. The reliability paradox refers to the counterintuitive behavior of some fit indices to show improved model fit as the magnitude of TLs decreases (i.e., when the quality of measurement decreases). See for example Hancock and Mueller (2011).

2. Although we use conventionally accepted thresholds for describing general patterns of fit across design conditions and models, we are not endorsing the use of applying GFI thresholds in substantive applications as cutoff values as is typically done with test statistics (e.g., Maydeu-Olivares & Shi, 2017).

3. We focus on the bias associated with the incorrectly specified CFA model and not the ESEM model because different rotations can result in different FC bias values with ESEM.

4. Due to space constraints, tabled average GFI values across 1,000 replications, by lowest and highest design condition levels, are presented in the Supporting Information.

5. UFA vs. CFA raw difference values are illustrated in the Supporting Information.

6. UFA vs. CFA raw difference values are illustrated in the Supporting Information.

7. This comparison is based on Mplus Target rotation, and a different form of rotation might well give rise to different amounts of bias for a given application of UFA. Although there are an infinite number of possible rotations, our primary focus here is on the ability of GFI measures to detect the measurement model misspecification in CFA relative to UFA models, which is unaffected by type of UFA rotation.