Brief Research Report: Growth Models With Small Samples and Missing Data

ABSTRACT

Small samples are common in growth models due to financial and logistical difficulties of following people longitudinally. For similar reasons, longitudinal studies often contain missing data. Though full information maximum likelihood (FIML) is popular to accommodate missing data, the limited number of studies in this area have found that FIML tends to perform poorly with small-sample growth models. This report demonstrates that the fault lies not with how FIML accommodates missingness but rather with maximum likelihood estimation itself. We discuss how the less popular restricted likelihood form of FIML, along with small-sample-appropriate methods, yields trustworthy estimates for growth models with small samples and missing data. That is, previously reported small sample issues with FIML are attributable to finite sample bias of maximum likelihood estimation not direct likelihood. Estimation issues pertinent to joint multiple imputation and predictive mean matching are also included and discussed.

KEYWORDS:

• Growth model
• missing data
• mixed effect model
• multilevel model
• small sample

Notes

1. This corresponds to the iterative generalized least squares algorithm (IGLS) for obtaining maximum likelihood estimates. There are other algorithms to obtain the maximum likelihood estimates; another common method is the EM algorithm.

2. Asymptotically, treating the fixed effects as known in the calculation of the variance components is negligible because sampling variance approaches zero as sample size approaches infinity and degrees of freedom are less impactful at larger sample sizes.

3. For a full paper outlining the differences between REML and ML that does not rely on equations, readers are referred to McNeish (2017b).

4. Cheung (2013) discusses an REML estimator for SEM in some conditions. This version of REML is based on equivalent models and does not derive REML for SEM but instead transforms an SEM to an MLM and applies the MLM REML equations. Though effective for some models, it is not fully generalizable.

5. Traditionally, direct likelihood and FIML are considered interchangeable terms for the same method (Allison, 2012). However, FIML is technically one type of direct likelihood method. In this paper, we use “FIML” to specifically refer to full information maximum likelihood and “direct likelihood” to refer to the broader class of such methods, which includes but is not limited to FIML.

6. It seems contradictory that we set missing values to the lower 50% of the distribution but achieved 60% missingness. This occurred by setting the entire lower 50% at Time 3 to be missing at Time 4 in addition to dropouts at Time 2 and Time 3 that were not in the lower 50% at Time 3.