Monte Carlo Evaluation of Two-Level Logistic Regression for Assessing Person Fit

Abstract

Person fit is the degree to which an item response model fits for individual examinees. Reise (2000) described how two-level logistic regression can be used to detect heterogeneity in person fit, evaluate potential predictors of person fit heterogeneity, and identify potentially aberrant individuals. The method has apparently never been applied to empirical data or evaluated in a simulation study. The present research applies Reise's method to empirical data obtained from university undergraduates measured on the Fear of Negative Evaluation scale. Additionally, Reise's method is evaluated under conditions varying according to the type of aberrancy, level of test reliability, and scale length. Statistical power to detect aberrancy was highly dependent on manipulated variables, and some results were affected by bias in model parameters that was due to the aberrant responders. Nevertheless, Reise's method generally performed well and detected aberrant individuals either as well as, or better than, the well-established l _z person-fit statistic.

Notes

¹ Data were originally collected by Carol M. Woods as part of a collaborative project with Jonathan S. Abramowitz and David F. Tolin. The FNE data used in the present research have been published previously (Deacon, Abramowitz, Woods, & Tolin, 2003; Tolin, Woods, & Abramowitz, 2003; Woods, Tolin, & Abramowitz, 2004).

² I thank Johnny Loney for writing the JavaScript function.

³ Strandmark and Linn (1987) describe an alternative approach for estimating a PRC with intersecting IRFs, which Reise (2000) refers to as “a direct precursor” to his method (p. 556).

* = statistically significant (α = .05); − = parameter not estimated, or for SFW items and SEs from Model 2a, not trustworthy.

* = statistically significant (α = .05); − = parameter not estimated, or for SEs from Model 2a, not trustworthy.