Cautions on the Use of Multiple Imputation When Selecting Between Latent Categorical versus Continuous Models for Psychological Constructs

Abstract

Clinical psychology researchers studying adolescents and young adults long have been interested in characterizing the latent categorical (classes/profiles) versus continuous (factors) nature of psychological syndromes. To inform this debate, researchers sometimes compare the fit of finite mixture versus factor analysis models to symptom data. This study explains and evaluates how missing data handling methods can impact results of this important model fit comparison. Via simulation, we assess three missing data-handling methods previously recommended to researchers fitting these models: multiple imputation using a saturated multivariate normal imputation model, multiple imputation using a hypothesized model, or full information maximum likelihood using the EM algorithm (FIML-EM). Results show that, under certain conditions, the method used to handle missing data can interfere with clinical psychologists' ability to accurately discriminate latent classes from continua. For instance, certain imputation methods increase the chance of selecting latent continua when latent classes truly exist. FIML-EM performed best overall. Recommendations for practice are discussed.

Notes

¹ The missing-completely-at-random assumption is that missingness depends neither on observed variables in the model nor on unobserved variables predictive of the outcome(s) and associated with model variables. The missing-at-random assumption is that missingness may depend on observed variables in the model but not on such unobserved variables.

² Some mixture applications report using a single imputation due to lengthy computational times.

³ Two independent chains were used after 50 identical iterations from one chain with a maximum of 50,000 iterations. Selected sensitivity analyses continuing with one chain yielded the same pattern of results. Convergence for MI was monitored using the Gelman-Rubin approach (with Potential Scale Reduction Factor ≤ 1.025 for any single parameter). Selected samples were inspected for adequate mixing and lack of class label-switching. Procedures recommended by Cho, Cohen, and Kim (2011) and Asparouhov and Muthén (2010) were employed until no label switching was observed across chain (during MI), across imputation within sample, or across repeated fitted LPAs within a cell of the simulation design, in empirical and graphical checks.

⁴ In other words, in all cells FIML-EM was used for model fitting. In only one cell, FIML-EM was also used for handling missing ys (all other cells already had imputed-y data by the analysis stage).

⁵ The conventional likelihood ratio test (LRT) developed for MI is not suited for comparing models with different numbers of classes due to the violation of regularity conditions. Conversely, adjusted versions of the LRT (Lo-Mendell-Rubin-LRT) suited for comparing models with different numbers of classes have not been adapted for MI. The bootstrap LRT (and the lesser used selection index in Markon & Krueger, 2006) also have not been adapted for MI and would be computationally prohibitive in the present simulation.