An Empirical Assessment of the Sensitivity of Mixture Models to Changes in Measurement

Abstract

This study explored the extent to which variations in self-report measures across studies can produce differences in the results obtained from mixture models. Data (N = 854) come from a laboratory analogue study of methods for creating commensurate scores of alcohol- and substance-use-related constructs when items differ systematically across participants for any given measure. Items were manipulated according to 4 conditions, corresponding to increasing levels of alteration to item stems, response options, or both. In Study 1, results from latent class analyses (LCAs) of alcohol consequences were compared across the 4 conditions, revealing differences in class enumeration and configuration. In Study 2, results from factor mixture models (FMMs) of alcohol expectancies were compared across 2 of the conditions, revealing differences in patterns and magnitude of the factor loadings and thresholds. The results suggest that even subtle differences in measurement can have substantively meaningful effects on mixture model results.

Keywords:

• factor mixture models
• latent class analysis
• mixture models
• self-report

FUNDING

This work was supported by National Institutes of Health grants F31 DA040334 (Fellow: Veronica T. Cole) and R01 DA034636 (Principal Investigator: Daniel J. Bauer). The content is solely the responsibility of the authors and does not represent the official views of the National Institute on Drug Abuse or the National Institutes of Health.

Notes

¹ Importantly, although the diagnostic classification of AUD changed from DSM–IV to DSM–5, the criteria themselves changed only by the omission of one item and addition of another.

² Importantly, class membership probabilities can be affected by covariates (Huang & Bandeen-Roche, 2004). Preliminary analyses included gender, African American and Asian race, and Hispanic or Latino ethnicity as covariates affecting class membership. However, neither class enumeration nor the LCA solutions themselves (i.e., item endorsement patterns and class prevalence rates) changed with the inclusion of these covariates. Thus, in the interest of parsimony, we exclusively consider an unconditional model here, so class membership probabilities ${\pi }_{ik}$ do not vary over individuals and become overall prevalence rates ${\pi }_k$.

³ This lack of consistency within version raises questions about the use of LMR LRT for class enumeration in general.

⁴ As in the three-class solutions, Euclidean distance was computed on probabilities averaged across samples for a given version (e.g., samples 1x and 1y), with the exception of Version 4, in which only sample 4y was used.

⁵ To determine the optimal partial weak invariance model, item-by-item tests of loading noninvariance across classes were conducted, following the IRT-LR-DIF strategy (Thissen, 2001). In both versions, the resulting partial weak invariance model was still rejected relative to the configural invariance model by the LRT.

Additional information

Funding

This work was supported by National Institutes of Health grants F31 DA040334 (Fellow: Veronica T. Cole) and R01 DA034636 (Principal Investigator: Daniel J. Bauer). The content is solely the responsibility of the authors and does not represent the official views of the National Institute on Drug Abuse or the National Institutes of Health.