Exploring the Enumeration Accuracy of Cross-Validation Indices in Latent Class Analysis

ABSTRACT

A crucial issue when estimating mixture models is selecting the model with the correct number of latent classes underlying the data, which is commonly referred to as class enumeration. Although cross-validation methods have been suggested with mixture models to help augment the class enumeration process (Masyn, 2013), they have been seldom used. The purpose of this simulation study was to compare the performance of traditionally used single sample enumeration indices with the performance of cross-validation indices when selecting the correct latent class model with binary indicators. Various conditions were manipulated, including the number of indicators, sample size, class separation, mixing proportions, and number of latent classes. The enumeration accuracy of sixteen indices (traditional, cross-validated, and double cross-validated) were documented in the manipulated conditions. The traditional sample-size adjusted BIC index was the most accurate among the indices. The performance of the double cross-validated -2LL was also promising. Recommendations are provided.

KEYWORDS:

• Class enumeration
• latent class analysis
• cross-validation

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website.

Notes

¹ Entropy was included as a class enumeration index in the current study for comparisons to previous studies. As noted by Masyn (2013), however, there can be a high amount of error when assigning certain participants to latent classes even when entropy is close to 1. In addition, posterior classification error may increase with models that have larger numbers of latent classes by chance. As such, entropy was not intended for class enumeration and should not be used for latent class model selection.

² Nested models in structural equation modeling are usually tested with a likelihood ratio test (LRT) as follows: $LRT=-2\left(L{L}_{k-1}-L{L}_k\right),$ which is on a chi-square (${\chi }^2$) distribution with degrees of freedom equal to the difference in the number of model parameters estimated between the two nested models. Statistically significant LRTs indicate that the more complex model should be adopted as the better fitting model. LRTs associated with nested LCA models, however, do not follow a ${\chi }^2$ distribution due to regularity conditions not being met (McLachlan & Peel, 2000). Specifically, the k-1 class model is attained from setting the parameter values of one latent class equal to zero in the k class model, which is at the border of its admissible parameter space rather than in the admissible parameter space (Peugh & Fan, 2013; Tekle, Gudicha, & Vermunt, 2016). Thus, the standard difference testing using the LRT is not applicable due to the ${\chi }^2$ distribution being estimated at its parameter boundary (McLachlan & Peel, 2000).

³ It is important to note that the pseudo single-sample indices were calculated with the two −2LLs saved when fixing estimates from the entire sample to subsamples A and B using Equations 12(12) $C^{\ast }orAI{C}^{\ast }=\sum _{l=1}^L-2L{L}_{kl}+2q{L}^2,$(12) –14(14) $NBI{C}^{\ast }=\sum _{l=1}^L-2L{L}_{kl}+ln\left(\frac{n+2}{24}\right)q{L}^2$(14) . More specifically, the two AICs, BICs, and NBICs computed when fixing estimates from the entire sample to subsamples A and B were not simply saved.

⁴ It is important to note that the double cross-validated AIC, BIC, and NBIC were calculated separately with the −2LLs saved when fixing estimates from subsamples A and B to subsamples B and A, respectively, using Equations 2–4 and subsequently averaged. More specifically, the two AICs, BICs, and NBICs computed when fixing estimates from subsamples A and B to subsamples B and A, respectively, were not simply saved.