Bayesian Repulsive Gaussian Mixture Model

ABSTRACT

We develop a general class of Bayesian repulsive Gaussian mixture models that encourage well-separated clusters, aiming at reducing potentially redundant components produced by independent priors for locations (such as the Dirichlet process). The asymptotic results for the posterior distribution of the proposed models are derived, including posterior consistency and posterior contraction rate in the context of nonparametric density estimation. More importantly, we show that compared to the independent prior on the component centers, the repulsive prior introduces additional shrinkage effect on the tail probability of the posterior number of components, which serves as a measurement of the model complexity. In addition, a generalized urn model that allows a random number of components and correlated component centers is developed based on the exchangeable partition distribution, which gives rise to the corresponding blocked-collapsed Gibbs sampler for posterior inference. We evaluate the performance and demonstrate the advantages of the proposed methodology through extensive simulation studies and real data analysis. Supplementary materials for this article are available online.

KEYWORDS:

• Blocked-collapsed Gibbs sampler
• Density estimation
• Generalized urn-model
• Model complexity
• Posterior convergence

Supplementary Material

The supplementary material contains the proofs of all technical results in Sections 2, 3, and 4, additional numerical results, and the MATLAB code for implementing the blocked-collapsed Gibbs sampler in Section 4.