The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models

ABSTRACT

Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.

 :

•  

Notes

¹ In the methodological literature, measurement quality does not necessarily have a strict definition and can be used to refer to validity, reliability, or generalizability of a particular scale. In this article, we use measurement quality to refer to the strength of the standardized factor loadings, which is highly related to reliability.

² We acknowledge that standardized loadings are not interchangeable with reliability, but they are related and might serve as a fair approximation that is concise to report. If it helps contextualize this study, the coefficient H values for the 0.40, 0.70, and 0.90 loading conditions were 0.48, 0.83, and 0.96, respectively. The McDonald's omega values for these conditions were 0.49, 0.83, and 0.96, respectively.