The use of predicted values for item parameters in item response theory models: an application in intelligence tests

Abstract

In testing, item response theory models are widely used in order to estimate item parameters and individual abilities. However, even unidimensional models require a considerable sample size so that all parameters can be estimated precisely. The introduction of empirical prior information about candidates and items might reduce the number of candidates needed for parameter estimation. Using data for IQ measurement, this work shows how empirical information about items can be used effectively for item calibration and in adaptive testing. First, we propose multivariate regression trees to predict the item parameters based on a set of covariates related to the item-solving process. Afterwards, we compare the item parameter estimation when tree-fitted values are included in the estimation or when they are ignored. Model estimation is fully Bayesian, and is conducted via Markov chain Monte Carlo methods. The results are two-fold: (a) in item calibration, it is shown that the introduction of prior information is effective with short test lengths and small sample sizes and (b) in adaptive testing, it is demonstrated that the use of the tree-fitted values instead of the estimated parameters leads to a moderate increase in the test length, but provides a considerable saving of resources.

Keywords:

• item response theory models
• multivariate regression trees
• item calibration
• adaptive testing
• Bayesian estimation
• intelligence tests

Acknowledgements

This work was partially supported by a research grant from the Italian Ministry of University and Research (MIUR), PRIN 2008 on “Latent structure analysis: new boundaries in statistical methods and models”. The authors thank the PiCompany for the data availability and the two anonymous reviewers for their helpful suggestions in improving the paper.