Missing data techniques for multilevel data: implications of model misspecification

Abstract

When modeling multilevel data, it is important to accurately represent the interdependence of observations within clusters. Ignoring data clustering may result in parameter misestimation. However, it is not well established to what degree parameter estimates are affected by model misspecification when applying missing data techniques (MDTs) to incomplete multilevel data. We compare the performance of three MDTs with incomplete hierarchical data. We consider the impact of imputation model misspecification on the quality of parameter estimates by employing multiple imputation under assumptions of a normal model (MI/NM) with two-level cross-sectional data when values are missing at random on the dependent variable at rates of 10%, 30%, and 50%. Five criteria are used to compare estimates from MI/NM to estimates from MI assuming a linear mixed model (MI/LMM) and maximum likelihood estimation to the same incomplete data sets. With 10% missing data (MD), techniques performed similarly for fixed-effects estimates, but variance components were biased with MI/NM. Effects of model misspecification worsened at higher rates of MD, with the hierarchical structure of the data markedly underrepresented by biased variance component estimates. MI/LMM and maximum likelihood provided generally accurate and unbiased parameter estimates but performance was negatively affected by increased rates of MD.

Keywords:

• missing data
• multiple imputation
• Monte Carlo
• multilevel data
• maximum likelihood

Acknowledgements

The authors acknowledge Hariharan Swaminathan and Jane Rogers for their valuable contributions to this paper. This project was partially supported by Award Number K01MH087219 from the National Institute of Mental Health. The contents of the paper are solely the responsibility of the authors and do not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.

Notes

Estimation efficiency is often evaluated with respect to the size of the parameter SE 8. Smaller SE represent more efficient parameter estimates.

The assumption required for ML and MI is that the MD process is ignorable 23. MAR is a component of that condition.

Software packages may restrict which variables may have MD, or may require that a distribution for the variables with MD be specified.

With the continuous development of new programs and features, more recent reviews are likely to be available.

Instructions for including auxiliary variables in SEM are outlined in Graham 8.

Bias is often evaluated as the average deviation of the parameter estimate from the population parameter value 8. Smaller deviation values represent less estimation bias.

The efficiency of an estimate based on m imputations relative to an infinite number of imputations is , where γ is the proportion of missing information (24, see also 10 26). The result is interpreted as the relative size of the SE.

Collins et al. 3 use the term SE to represent the standard deviation of parameter estimates. SD is used here.

SE tended to be positively skewed. Therefore, we trimmed 2% of the estimates from the tails of the distributions and reported the mean of the middle 96% of SE.

Other software packages for MI of multilevel data include MLwiN 20 and WinMICE 14.

Additional information

Notes on contributors

Anne C. Black

Present address: Department of Psychiatry, Yale University, VA Connecticut Healthcare System, 950 Campbell Avenue, West Haven, CT 06516, USA.