An Investigation of Agreement Between the Item Difficulty Coefficient Calculated in Accordance With Classical Test Theory and Item Response Theory With Bland–Altman Method

Abstract

The purpose of this study is to investigate agreement between item difficulty coefficients calculated relying on classical test theory and item response theory with Bland–Altman method. According to results, although there is a high correlation between Pj and b coefficient estimated with HGLM (hierarchical generalized linear model), 1P, and 3P models, it can be said that there is no agreement between two methods and cannot be used interchangeably. It is observed that the confidence limit is wide according to Bland–Altman graphics. Therefore, it can be said that there is no agreement between item difficulty values obtained from two methods. Bland–Altman method which is used in clinical studies mostly is suggested to be used in the comparison of methods used especially in the evaluation of student performance in education, in agreement studies among specialist considerations especially in terms of providing additional information to the studies in which correlation coefficient is calculated.

Keywords:

• Bland–Altman method
• Classical test theory
• Item difficulty coefficient
• Item response theory.