Multilevel IRT models for the university teaching evaluation

Abstract

In this paper, a generalization of the two-parameter partial credit model (2PL-PCM) and of two special cases, the partial credit model (PCM) and the rating scale model (RSM), with a hierarchical data structure will be presented. Having shown how 2PL-PCM, as with other item response theory (IRT) models, may be read in terms of a generalized linear mixed model (GLMM) with two aggregation levels, a presentation will be given of an extension to the case of measuring the latent trait of individuals aggregated in groups. The use of this Multilevel IRT model will be illustrated via reference to the evaluation of university teaching by students following the courses. The aim is to generate a ranking of teaching on the basis of student satisfaction, so as to give teachers, and those responsible for organizing study courses, a background of information that takes the opinions of the direct target group for university teaching (that is, the students) into account, in the context of improving the teaching courses available.

Keywords:

• generalized linear mixed model
• item response theory (IRT)
• multilevel model
• multilevel IRT model
• student satisfaction
• university teaching evaluation

Acknowledgements

The authors thank the helpful comments received at SMUE (Foggia, Italy, 2008). V.C. was partially supported by grants from ex-murst 60%, 2008.