Multivariate Density Estimation by Bayesian Sequential Partitioning

Abstract

Consider a class of densities that are piecewise constant functions over partitions of the sample space defined by sequential coordinate partitioning. We introduce a prior distribution for a density in this function class and derive in closed form the marginal posterior distribution of the corresponding partition. A computationally efficient method, based on sequential importance sampling, is presented for the inference of the partition from this posterior distribution. Compared to traditional approaches such as the kernel method or the histogram, the Bayesian sequential partitioning (BSP) method proposed here is capable of providing much more accurate estimates when the sample space is of moderate to high dimension. We illustrate this by simulated as well as real data examples. The examples also demonstrate how BSP can be used to design new classification methods competitive with the state of the art.

Keywords:

• Binary partition
• Classification
• KL divergence
• Sequential importance sampling

Acknowledgments

We thank John C. Mu for help on the algorithm and proof reading. We also thank Ying Lu for proof reading. This work is partially supported by NSF Grant DMS-09-06044 to WHW.