Comparing the Fit of Item Response Theory and Factor Analysis Models

Abstract

Linear factor analysis (FA) models can be reliably tested using test statistics based on residual covariances. We show that the same statistics can be used to reliably test the fit of item response theory (IRT) models for ordinal data (under some conditions). Hence, the fit of an FA model and of an IRT model to the same data set can now be compared. When applied to a binary data set, our experience suggests that IRT and FA models yield similar fits. However, when the data are polytomous ordinal, IRT models yield a better fit because they involve a higher number of parameters. But when fit is assessed using the root mean square error of approximation (RMSEA), similar fits are obtained again. We explain why. These test statistics have little power to distinguish between FA and IRT models; they are unable to detect that linear FA is misspecified when applied to ordinal data generated under an IRT model.

Keywords:

• approximate fit
• attitudes
• categorical data analysis
• goodness-of-fit
• ordinal factor analysis
• personality
• Samejima's graded model
• structural equation modeling

Notes

¹LISREL (Jöreskog & Sörbom, 2008), Mplus (L. Muthén & Muthén, 1998–2010), or EQS (Bentler, 2004) estimate this model when a factor model is fitted to variables declared to be categorical.

²A linear model with additional assumptions that enables the computation of pattern probabilities was introduced in Maydeu-Olivares (2005b), who referred to it as a linear IRT model to distinguish it from the common factor model.

³This model is formally equivalent to Muraki's (1992) generalized partial credit model.

⁴This function is also called category response function. When the items are binary, the conditional probability of endorsing an item, Pr(Y_i = 1|η), is called item response function.

⁵When an IRT model is identified from the covariance matrix then Δ is of full rank. Also, the model can be estimated using only the information contained in the sample covariance matrix.