Measurement Invariance Testing with Many Groups: A Comparison of Five Approaches

Abstract

With the increasing use of international survey data especially in cross-cultural and multinational studies, establishing measurement invariance (MI) across a large number of groups in a study is essential. Testing MI over many groups is methodologically challenging, however. We identified 5 methods for MI testing across many groups (multiple group confirmatory factor analysis, multilevel confirmatory factor analysis, multilevel factor mixture modeling, Bayesian approximate MI testing, and alignment optimization) and explicated the similarities and differences of these approaches in terms of their conceptual models and statistical procedures. A Monte Carlo study was conducted to investigate the efficacy of the 5 methods in detecting measurement noninvariance across many groups using various fit criteria. Generally, the 5 methods showed reasonable performance in identifying the level of invariance if an appropriate fit criterion was used (e.g., Bayesian information criteron with multilevel factor mixture modeling). Finally, general guidelines in selecting an appropriate method are provided.

Keywords:

• approximate measurement invariance
• Bayesian
• multilevel CFA
• multilevel factor mixture modeling

Notes

¹ http://www.usf.edu/education/faculty/faculty-profiles/eun-sook-kim.aspx

² Otherwise, within-level factor means are not estimated and within-level factor variances are constrained equal between classes in Mplus.

³ The factor loading of the first item is freely estimated (not constrained at 1) because factor variance of one class is fixed at one, but the factor loading of this item should be constrained equal between classes. This identification strategy is a hybrid of unit (equal) loading and unit variance methods. Different identification strategies (e.g., unit loading) can be used.

⁴ We also conducted SB LRT, but too many cases had negative chi-square values. The results are not included in this article.

⁵ As proposed in the introduction, we can test the Bayesian approximate invariance for factor loadings first to establish approximate metric invariance and then intercepts to establish approximate scalar invariance. Due to time constraints, we directly investigated approximate scalar invariance.