A Comparison of Item-Level and Scale-Level Multiple Imputation for Questionnaire Batteries

Abstract

Behavioral science researchers routinely use scale scores that sum or average a set of questionnaire items to address their substantive questions. A researcher applying multiple imputation to incomplete questionnaire data can either impute the incomplete items prior to computing scale scores or impute the scale scores directly from other scale scores. This study used a Monte Carlo simulation to assess the impact of imputation method on the bias and efficiency of scale-level parameter estimates, including scale score means, between-scale correlations, and regression coefficients. Although the choice of imputation approach had no influence on the bias of scale-level parameter estimates, it had a substantial impact on efficiency, such that item-level imputation consistently produced a meaningful power advantage. The simulation results clearly supported the use of item-level imputation. To illustrate the differences between item- and scale-level imputation, we examined predictors of 7th-grade academic self-efficacy in a sample of 595 low-income Mexican Origin adolescents in a planned missingness design. The results of the empirical data analysis were consistent with those of the simulation and also suggested that researchers should be cautious when implementing planned missing data designs that necessitate scale-level imputation.

Notes

¹This confidence interval width applies to main effects and interactions with a single degree of freedom and assumes that the remaining design effects collectively explained 6% of the variance (i.e., a medium effect size). Assuming that the remaining effects explained 14% of the variance (i.e., a large effect size) increased the confidence interval width to .0011.