Monte Carlo Approximation of Bayes Factors via Mixing With Surrogate Distributions

Abstract

By mixing the target posterior distribution with a surrogate distribution, of which the normalizing constant is tractable, we propose a method for estimating the marginal likelihood using the Wang–Landau algorithm. We show that a faster convergence of the proposed method can be achieved via the momentum acceleration. Two implementation strategies are detailed: (i) facilitating global jumps between the posterior and surrogate distributions via the multiple-try Metropolis (MTM); (ii) constructing the surrogate via the variational approximation. When a surrogate is difficult to come by, we describe a new jumping mechanism for general reversible jump Markov chain Monte Carlo algorithms, which combines the MTM and a directional sampling algorithm. We illustrate the proposed methods on several statistical models, including the log-Gaussian Cox process, the Bayesian Lasso, the logistic regression, and the g-prior Bayesian variable selection. Supplementary materials for this article are available online.

Keywords:

• Marginal likelihood
• Mixture distribution
• Multiple-try Metropolis
• Reversible jump MCMC
• Wang–Landau algorithm

Supplementary Materials

The supplementary materials contain the proof of Proposition 1 and more details of the numerical examples discussed in Section 5.

Acknowledgments

We thank Pierre Jacob for helpful discussions and suggestions. Some of the numerical examples in the paper are implemented based on the R package debiasedhmc (Heng and Jacob 2019).

Notes

1 For numerical stability, we recommend to work on the logarithmic scale.

2 The source code for implementing the numerical examples discussed in Section 5 is available at https://github.com/chenguangdai/BayesFactor-WL/tree/master/MLWL.

Additional information

Funding

This work was partially supported by NSF DMS-1613035, DMS-1712714 and NIH funding: NIGMS R01GM122080.