The Impact of Varying the Number of Measurement Invariance Constraints on the Assessment of Between-Group Differences of Latent Means

Abstract

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.

Keywords:

• factor mean differences
• measurement invariance
• multiple group analysis

Notes

¹ If differences occur in factor loadings across groups, there is little reason to have intercepts that are equivalent across groups (Thompson & Green, 2013). We are unaware of any empirical studies that examine this issue.

² A pilot study indicated that Type I error and power rates obtained by the Wald test and chi-square difference test of factor mean differences were consistent to the third decimal place.