Rapid Bayesian Inference for Expensive Stochastic Models

ABSTRACT

Almost all fields of science rely upon statistical inference to estimate unknown parameters in theoretical and computational models. While the performance of modern computer hardware continues to grow, the computational requirements for the simulation of models are growing even faster. This is largely due to the increase in model complexity, often including stochastic dynamics, that is necessary to describe and characterize phenomena observed using modern, high resolution, experimental techniques. Such models are rarely analytically tractable, meaning that extremely large numbers of stochastic simulations are required for parameter inference. In such cases, parameter inference can be practically impossible. In this work, we present new computational Bayesian techniques that accelerate inference for expensive stochastic models by using computationally inexpensive approximations to inform feasible regions in parameter space, and through learning transforms that adjust the biased approximate inferences to closer represent the correct inferences under the expensive stochastic model. Using topical examples from ecology and cell biology, we demonstrate a speed improvement of an order of magnitude without any loss in accuracy. This represents a substantial improvement over current state-of-the-art methods for Bayesian computations when appropriate model approximations are available. Supplementary files for this article are available online.

KEYWORDS:

• Approximate Bayesian computation
• Bayesian inference
• Continuum-limit approximation
• Parameter estimation
• Sequential Monte Carlo

Supplementary Material

Appendix: Supplementary sections including extra technical descriptions, numerical results and datasets. (.pdf file) Software: Snapshot of GitHub repository including example implementations and examples. (.zip file)

Acknowledgments

Computational resources where provided by the eResearch Office, Queensland University of Technology. The authors thank Wang Jin for helpful discussions. D.J.W. acknowledges continued support from the Centre for Data Science at the Queensland University of Technology. D.J.W. is a member of the Australian Research Council Centre of Excellence for Mathematical and Statistical Frontiers. R.E.B. would like to thank the Leverhulme Trust for a Leverhulme Research Fellowship, the Royal Society for a Wolfson Research Merit Award, and the BBSRC for funding via BB/R00816/1.

Software Availability

Numerical examples presented in this work are available from GitHub https://github.com/ProfMJSimpson/Warne_RapidBayesianInference_2019.

Note

¹Throughout, the overbar tilde notation, for example, $\tilde{x}$, is used to refer to the ABC entities related to the approximate model, whereas quantities without the overbar tilde notation, for example, x, are used to refer fo the ABC entities related to the exact model.

Additional information

Funding

This work was supported by the Australian Research Council (DP170100 474). M.J.S. appreciates support from the University of Canterbury Erskine Fellowship.