Efficient Bayesian Synthetic Likelihood With Whitening Transformations

Abstract

Likelihood-free methods are an established approach for performing approximate Bayesian inference for models with intractable likelihood functions. However, they can be computationally demanding. Bayesian synthetic likelihood (BSL) is a popular such method that approximates the likelihood function of the summary statistic with a known, tractable distribution—typically Gaussian—and then performs statistical inference using standard likelihood-based techniques. However, as the number of summary statistics grows, the number of model simulations required to accurately estimate the covariance matrix for this likelihood rapidly increases. This poses a significant challenge for the application of BSL, especially in cases where model simulation is expensive. In this article, we propose whitening BSL (wBSL)—an efficient BSL method that uses approximate whitening transformations to decorrelate the summary statistics at each algorithm iteration. We show empirically that this can reduce the number of model simulations required to implement BSL by more than an order of magnitude, without much loss of accuracy. We explore a range of whitening procedures and demonstrate the performance of wBSL on a range of simulated and real modeling scenarios from ecology and biology. Supplementary materials for this article are available online.

Keywords:

• Approximate Bayesian computation
• Covariance matrix estimation
• likelihood-free inference
• Markov chain Monte Carlo
• Shrinkage estimation

Acknowledgments

The authors thank to Ziwen An for providing some code for the toad and the collective cell spreading models, and the High Performance Computing group at QUT for their computational resources. We thank to the anonymous associate editor and three referees for helpful comments and suggestions to help improve the article.

Supplementary material

The following documents have been provided to support the main article.

Appendices: Contains a proof of Theorem 1 and all additional results, as referred to in the article (.pdf file).

Code: Contains all Matlab code for the MA(2) example, and an example R script that calls upon the BSL package (Code.zip, compressed (zipped) folder).

Additional information

Funding

JWP was supported by a QUT Master of Philosophy Award. SAS was supported by the Australian Research Council under the Future Fellowship scheme (FT170100079). CD was supported by an Australian Research Council Discovery Project (DP200102101). CD, SAS and IWT are also supported by the ARC Centre of Excellence in Mathematical and Statistical Frontiers (ACEMS; CE140100049). CD is grateful to ACEMS for providing funding to visit SAS at UNSW where discussions on this research took place.