Bayesian Simultaneous Edit and Imputation for Multivariate Categorical Data

ABSTRACT

In categorical data, it is typically the case that some combinations of variables are theoretically impossible, such as a 3-year-old child who is married or a man who is pregnant. In practice, however, reported values often include such structural zeros due to, for example, respondent mistakes or data processing errors. To purge data of such errors, many statistical organizations use a process known as edit-imputation. The basic idea is first to select reported values to change according to some heuristic or loss function, and second to replace those values with plausible imputations. This two-stage process typically does not fully use information in the data when determining locations of errors, nor does it appropriately reflect uncertainty resulting from the edits and imputations. We present an alternative approach to editing and imputation for categorical microdata with structural zeros that addresses these shortcomings. Specifically, we use a Bayesian hierarchical model that couples a stochastic model for the measurement error process with a Dirichlet process mixture of multinomial distributions for the underlying, error-free values. The latter model is restricted to have support only on the set of theoretically possible combinations. We illustrate this integrated approach to editing and imputation using simulation studies with data from the 2000 U. S. census, and compare it to a two-stage edit-imputation routine. Supplementary material is available online.

KEYWORDS:

• Dirichlet process
• Error
• Fellegi–Holt
• Latent
• Structural zero

Supplementary Materials

The supplementary materials include the description of the algorithm for transforming a collection of table slice definitions into a collection of disjoint definitions. It also include results from additional simulations.

Funding

This research was supported by a grant from the National Science Foundation (SES-11-31897).