Measurement Invariance: Dealing with the Uncertainty in Anchor Item Choice by Model Averaging

ABSTRACT

A core challenge in modeling partial measurement invariance (MI) is choosing reference items as anchors for which MI indeed holds. Many approaches dealing with this issue have been proposed, each making a different assumption about MI and yielding a single set of anchor items. Here, we consider the case where i) partial MI modeling is used for estimating effects, e.g., a group mean difference, and ii) there is no straightforward theoretical reason to choose specific items as anchors. We argue that in this situation the uncertainty of anchor item choice should be considered and propose to use model averaging with a priori defined model weights. The approach allows not only to depict uncertainty in the anchor items choice but also allows to include prior knowledge and beliefs of the researcher. We derive the properties of the approach and illustrate its use with an example on the assessment of obsessive-compulsive disorder.

KEYWORDS:

• Partial measurement invariance
• model averaging
• Bayesian analysis
• item cluster approach

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

¹ For convenience, we provide R syntax reproducing all analysis steps for both, dichotomous and continuous data, in the supplements.

² Further technical details on applying the item cluster approach are given in the Empirical Example section.

³ We ran multi-group 2PL models in STAN for 100000 iterations (warm-up = 50000). There were no signs of non-convergence (maxRhat = 1.001, medianRhat = 1.00001, minEffN = 836, medianEffN = 72052). Both the STAN and R syntax can be found in the Appendix.

⁴ The small difference occurs due to the fact that when assuming full MI all items equally contribute to the estimate, while in the cluster approach each cluster equally contributes to the aggregated estimate; the number of items, however, differs between clusters.

Additional information

Funding

This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) within the Priority Programme 1646: Education as a Lifelong Process [PO1655/2-1].