Bayesian Modal Estimation of the Four-Parameter Item Response Model in Real, Realistic, and Idealized Data Sets

ABSTRACT

In this study, we explored item and person parameter recovery of the four-parameter model (4PM) in over 24,000 real, realistic, and idealized data sets. In the first analyses, we fit the 4PM and three alternative models to data from three Minnesota Multiphasic Personality Inventory-Adolescent form factor scales using Bayesian modal estimation (BME). Our results indicated that the 4PM fits these scales better than simpler item Response Theory (IRT) models. Next, using the parameter estimates from these real data analyses, we estimated 4PM item parameters in 6,000 realistic data sets to establish minimum sample size requirements for accurate item and person parameter recovery. Using a factorial design that crossed discrete levels of item parameters, sample size, and test length, we also fit the 4PM to an additional 18,000 idealized data sets to extend our parameter recovery findings. Our combined results demonstrated that 4PM item parameters and parameter functions (e.g., item response functions) can be accurately estimated using BME in moderate to large samples (N ⩾ 5, 000) and person parameters can be accurately estimated in smaller samples (N ⩾ 1, 000). In the supplemental files, we report annotated code that shows how to estimate 4PM item and person parameters in (Chalmers, 2012).

KEYWORDS:

• Bayesian modal estimation
• four-parameter model
• IRT
• parameter recovery

Notes

¹ MMPI-A Booklet of Abbreviated Items. Copyright 2005 by the Regents of the University of Minnesota. All rights reserved. Used by permission of the University of Minnesota Press.

2 We thank the University of Minnesota Press for kindly providing these data.

3 Like Gelman and Shalizi (2013), in this article we regard the prior distributions of Bayesian analyses as regularization devices to control estimator variability and we are not embracing subjective definitions of probability. Moreover, because our BME analyses include priors on item parameters we interpret all likelihood-based fit indices heuristically and in a later section we explore the sampling distributions of these indices.