Understanding Linkages Among Mixture Models

Abstract

The methodological literature on mixture modeling has rapidly expanded in the past 15 years, and mixture models are increasingly applied in practice. Nonetheless, this literature has historically been diffuse, with different notations, motivations, and parameterizations making mixture models appear disconnected. This pedagogical review facilitates an integrative understanding of mixture models. First, 5 prototypic mixture models are presented in a unified format with incremental complexity while highlighting their mutual reliance on familiar probability laws, common assumptions, and shared aspects of interpretation. Second, 2 recent extensions—hybrid mixtures and parallel-process mixtures—are discussed. Both relax a key assumption of classic mixture models but do so in different ways. Similarities in construction and interpretation among hybrid mixtures and among parallel-process mixtures are emphasized. Third, the combination of both extensions is motivated and illustrated by means of an example on oppositional defiant and depressive symptoms. By clarifying how existing mixture models relate and can be combined, this article bridges past and current developments and provides a foundation for understanding new developments.

Notes

¹For mixture models with discrete outcomes, a probability parameterization, most commonly, or loglinear parameterization, have historically been used (e.g., Biemer, 2011; Collins & Flaherty, 2002; Heinen, 1996; McCutcheon, 2002). For mixture models with continuous outcomes, a logistic parameterization is typically used for the between-class model, as employed here also. For consistency, we also employ a logistic parameterization with discrete outcomes (following, e.g., Humphreys & Janson, 2000; B. O. Muthén, 2001, 2004; Reboussin, Reboussin, Liang, & Anthony, 1998). This logistic parameterization can be used to compute probabilities, implicitly achieves the same constraints as in the popular probability parameterization, and readily expands to accommodate covariates (unlike the probability parameterization; Magidson & Vermunt, 2004).

²Reasons are that (a) there is no formal set of rules (akin to Wright's tracing rules in SEM) allowing mixture model equations to be directly reproduced from their diagrams and (b) diagrams do not fully represent all aspects of the mixture model. For instance, current path diagrams do not communicate how many classes there are, if a particular parameter differs across some but not all classes, if a parameter is fixed to 0 in some classes but estimated in others, or which is the reference class.

³Here we use a shorthand: P(A) represents P(A = a), where a is a realization of random variable A.

⁴This is also sometimes called the chain rule.

⁵Online appendix is available at http://www.vanderbilt.edu/peabody/sterba/appxs.htm

⁶For each outcome in the LPA, Figure 2 depicts class-specific densities weighted by their class probabilities; however, weighting is used here only for ease of visualization. In LPA estimation, weighting is done for the joint outcome density, not individual outcome densities (see Equation (14)).

⁷Even when discrete subpopulations exist, many more response patterns than classes can arise due to measurement error.

⁸When evaluated for a particular response pattern, it is a probability.

⁹Having K = M corresponds with configural invariance of the categorical latent variable across time.

¹⁰The testability of measurement invariance in the conventional LTA contrasts to its untestability in the conventional GBT (unless extended to a second-order GBT, as in Grimm & Ram, 2009).

¹¹If T > 2, it could be possible to regress latent states at time t on prior states at both t – 1 and t – 2.

^a Further constraints are common for parsimony and/or to prevent empirical underidentification (see text). Further constraints are also used to, for instance, impose threshold invariance within state across time in LTA.

¹²Analogously, in continuous latent variable models, the posterior density can be “shrunk” toward the mean of the prior density (Skrondal & Rabe-Hesketh, 2004).

¹³A third approach, not discussed here, is unavailable for LCA and LTA but available for LPA and GBT. It involves estimating residual covariances within class; estimating many such covariances and/or allowing them to differ across class can incur estimation problems (Lubke & Neale, 2006).

¹⁴For instance, in a K = M = 2 LTA, testing forward change (below-diagonal elements of the transition probability matrix = 0) requires fixing α₁ to a large negative number. Testing no-change (off-diagonal elements of transition probability matrix = 0) requires also fixing β₁₁ to a very large positive number.

¹⁵Within class, the number of growth coefficients is (1 + b) × r (where b is the polynomial curve degree and r is the number of regimes). At timepoints where a particular regime goes off-line, its growth coefficients are fixed to 0 (for details see Dolan, Schmittmann, Lubke, & Neale, 2005).