Leveraging Item Parameter Drift to Assess Transfer Effects in Vocabulary Learning

ABSTRACT

Longitudinal models typically emphasize between-person predictors of change but ignore how growth varies within persons because each person contributes only one data point at each time. In contrast, modeling growth with multi-item assessments allows evaluation of how relative item performance may shift over time. While traditionally viewed as a nuisance under the label of “item parameter drift” (IPD), IPD may be of substantive interest if it reflects how learning manifests on different items or subscales at different rates. In this study, we apply the Explanatory Item Response Model to assess IPD in a causal inference context. Simulation results show that when IPD is not accounted for, both parameter estimates and standard errors can be affected. We illustrate with an empirical application to the persistence of transfer effects from a content literacy intervention , revealing how researchers can leverage IPD to achieve a more fine-grained understanding of how vocabulary learning develops over time.

Acknowledgments

The authors wish to thank Douglas Mosher, Jackie Relyea, and the three anonymous reviewers for their helpful comments on this manuscript.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Data and Code Availability

A detailed replication toolkit containing the data and code used in this study (as well as some supplemental analyses) is available at the following URL: https://doi.org/10.7910/DVN/ZF1LKZ

Notes

¹ While outside the scope of this study, prior research has demonstrated that misspecification of the correlation between random effects can create bias in estimated interactions among the fixed effects. See Gilbert, Miratrix, et al. (2024) for a full treatment of this issue.

² The technically oriented reader might notice that, typically, random slopes longitudinal models are not identified with only two time points because each subject’s individual trajectory can be “perfectly” fit by the model (Muthén, 2000). The issue of non-identifiability does not apply here because the cross-classified structure of the data is additive, not multiplicative. That is, there is no interaction between the person and item random effects because such an interaction would be confounded with the error term, whereas the additive case allows for imperfect fit. Thus, such models may provide additional utility in empirical applications when only two time points are available. See O’Connell et al. (2022, pp. 170–171), Hox et al. (2017), and Shi et al. (2010) for a discussion and additional references.

³ We estimated the CFA model using the lavaan program in R (Rosseel, 2012) using the default estimation options, allowing for variable factor loadings by item. We treated the items as continuous because lavaan does not allow for logistic link functions and to obtain the standard fit statistics available in CFA models of continuous indicators. Furthermore, the assessment also included vocabulary words that were taught in Grade 1 MORE lessons and were tested in both Grade 1, halfway through the intervention, and in Grade 3, one year after the conclusion of the intervention. An analogous analysis of these words is included in the OSM and shows a similar pattern of results to the Grade 2 words analyzed here.