The Impact of “Negligible” Cross-Loadings in Investigations of Measurement Invariance with MGCFA and MGESEM

Abstract

Measurement invariance (MI) is an essential part of validity evidence concerned with ensuring that tests function similarly across groups, contexts, and time. Most evaluations of MI involve multigroup confirmatory factor analyses (MGCFA) that assume simple structure. However, recent research has shown that constraining non-target indicators to zero when cross-loadings are present results in biased estimates of latent variable associations. Using Monte Carlo simulation, we investigate the behavior of fit statistics for identifying non-invariance when the target measurement model is the same for both groups, and the source of non-invariance is the presence and magnitude of non-zero cross-loadings in one group, but not in another. We consider differences between MGCFA and multigroup ESEM (MGESEM), and combined and separate group tests of configural invariance. Implications for applied researchers are provided.

Keywords:

• CFA
• cross-loadings
• ESEM
• measurement invariance
• multigroup analysis

Notes

1 See Robitzsch and Lüdtke (2023) for an alternative perspective on the need to establish measurement invariance prior to group comparisons on the latent variables.

2 See Leitgöb et al. (2023) for discussion of alternative and emerging methods for evaluating measurement invariance.

3 We show Counsell et al. (2020) null specification with ≥ rather than > as given in Yuan and Chan (2016).

4 In our analyses below, we use the unadjusted ${\epsilon }_0$ value following research by Counsell et al. (2020).

5 Although we use conventionally accepted thresholds for describing their sensitivity to invariance, we are not endorsing the use of applying goodness-of-fit index thresholds in substantive applications as cut-off values as is typically done with test statistics (e.g., Maydeu-Olivares & Shi, 2017).