Auxiliary Variables in Multiple Imputation When Data Are Missing Not at Random

Abstract

Most current implementations of multiple imputation (MI) assume that data are missing at random (MAR), but this assumption is generally untestable. We performed analyses to test the effects of auxiliary variables on MI when the data are missing not at random (MNAR) using simulated data and real data. In the analyses we varied (a) the correlation, (b) the level of missing data, (c) the pattern of missing data, and (d) sample size. Results showed that MI performed adequately without auxiliary variables but they also had a modest impact on bias in the real data and improved efficiency in both data sets. The results of this study suggest that, counter to the concern about the violation of the MAR assumption, MI appears to be quite robust to missing data that are MNAR in analytic situations such as the ones presented here. Further, results can be made even better via the use of auxiliary variables, particularly when efficiency is a primary concern.

Keywords:

• auxiliary variables
• missing data
• missing not at random
• multiple imputation

Notes

¹Restricting our review to articles which appeared between 2000 and 2008 in three major sociology journals (American Journal of Sociology, American Sociological Review, and Social Forces, in alphabetical order), we found that 13% of the total articles (184 articles) utilized MI to deal with missing data while 46% used list-wise deletion.

²Note that terms referring to extra or supplementary variables included in an imputation procedures are different across authors: “additional information” by Rubin (1996); “auxiliary variables,” “inclusive or restrictive” strategies by Collins et al. (2001); and “liberal and conservative strategies” by Demirtas (2004).

³Retrieved from http://www.nsf.gov/pubs/2007/nsf0748/nsf0748_3.pdf

⁴The Appendix displays the results of regression analyses using listwise deleted GSS data. By comparison, listwise deletion produces very negligible differences both in regression estimates and efficiency from MI. In our research conditions, listwise deletion produced reasonable results under MNAR.

⁵Our additional analyses with listwise deletion (see Appendix) show that MI does not outperform the listwise deletion method in many of the models presented here, which is counter to the literature that reports an advantage over listwise deletion across a wide range of conditions (Allison, 2000; White & Carlin, 2010). Although analyses were performed with listwise deleted data, we do not intend to evaluate which one is better than the other since broader conditions should be considered for the comparison of missing data methods (see White & Carlin for a comparison of listwise deletion with MI). A body of literature shows that listwise deletion is not problematic in regression analyses as long as the probability of missing variable does not depend on outcome variable (Allison, 2000, 2001; Little, 1992; Schafer & Graham, 2002). As to the unbiased estimates of regression slopes with listwise deleted data set even under violation of the assumption, Allison (2000) stated that “disproportionate stratified sampling on the independent variables in a regression model does not bias coefficient estimates. A missing data mechanism that depends only on the values of the independent variables is essentially equivalent to stratified sampling” (pp. 6–7). Enders (2010) encapsulated that “this relatively esoteric scenario is the only situation in which listwise deletion is likely to outperform maximum likelihood estimation and multiple imputation with missing not at random data” (p. 4).