Discrepancy measures for item fit analysis in item response theory

Abstract

Item response theory (IRT) models are commonly used in educational and psychological testing to assess the (latent) ability of examinees and the effectiveness of the test items in measuring this underlying trait. The focus of this paper is on the assessment of item fit for unidimensional IRT models for dichotomous items using a Bayesian method. This paper will illustrate and compare the effectiveness of several discrepancy measures, used within the posterior predictive model check procedure, in detecting misfitted items. The effectiveness of the different discrepancy measures are illustrated in a simulation study using artificially altered simulated data. Using the best discrepancy measure among those studied, this method was applied to real data coming from a mathematics placement exam.

Keywords:

• item response theory
• Bayesian method
• posterior predictive distribution
• Gibbs sampling
• importance sampling

AMS Subject Classification :

• 62F15
• 91-08
• 62P15
• 91C05