Regarding Item Parameter Invariance for the Rasch and the 2-Parameter Logistic Models: An Investigation under Finite Non-Representative Sample Calibrations

ABSTRACT

The property of item parameter invariance in item response theory (IRT) plays a pivotal role in the applications of IRT such as test equating. The scope of parameter invariance when using estimates from finite biased samples in the applications of IRT does not appear to be clearly documented in the IRT literature. This article provides information on the extent to which item parameter invariance is observed in samples with the Rasch and 2-parameter model calibrations through simulations, where the behaviors of item parameter estimates were examined under 12 different types of convenient sampling scenarios. The results indicated that the property of item invariance in IRT for dichotomously scored data could hold for the sample item parameter estimates, regardless of biased samples, when the model holds in the data, the number of items in a test is not small, and the sample size is large.

KEYWORDS:

• Item parameter invariance
• item response theory
• finite biased samples

Notes

1. See Seong (1990) and Zwinderman and van den Wollenberg (1990) for studies of the sensitivity and robustness of the MML-normal method. For more discussion of the JML method properties, see Andersen (1973), De Gruijter (1990), Divgi (1986, 1989), Haberman (1977, 2004, 2008), Molenaar (1995), and Van den Wollenberg et al. (1988).

2. In ANOVA, because our simulation procedure produced only the summary statistics per simulation condition, the ANOVA contained a single observation in each cell. Under this single observation per cell design, the full factorial model cannot be estimated. Thus, for ANOVA model 1, the largest model that can be implemented was with first-order between-subjects interactions. For ANOVA models 2–4, no between-subjects interactions can be modeled.

3. Due to relatively small sample sizes in the ANOVA models (and the constraints in ANOVA due to the single observation per cell which renders no interaction effect modeling), it may lack power to detect the significant effects. Thus, the significance results were only used as supplements to the eta-squared measures.