A Function Emulation Approach for Doubly Intractable Distributions

Abstract

Doubly intractable distributions arise in many settings, for example, in Markov models for point processes and exponential random graph models for networks. Bayesian inference for these models is challenging because they involve intractable normalizing “constants” that are actually functions of the parameters of interest. Although several computational methods have been developed for these models, each can be computationally burdensome or even infeasible for many problems. We propose a novel algorithm that provides computational gains over existing methods by replacing Monte Carlo approximations to the normalizing function with a Gaussian process-based approximation. We provide theoretical justification for this method. We also develop a closely related algorithm that is applicable more broadly to any likelihood function that is expensive to evaluate. We illustrate the application of our methods to challenging simulated and real data examples, including an exponential random graph model, a Markov point process, and a model for infectious disease dynamics. The algorithm shows significant gains in computational efficiency over existing methods, and has the potential for greater gains for more challenging problems. For a random graph model example, we show how this gain in efficiency allows us to carry out accurate Bayesian inference when other algorithms are computationally impractical. Supplementary materials for this article are available online.

Keywords:

• Doubly intractable distributions
• Exponential random graph models
• Gaussian processes
• Importance sampling
• Markov chain Monte Carlo
• Markov point processes

Acknowledgments

The authors are grateful to Galin Jones, Omiros Papaspiliopoulos, and Alexander Mitrophanov for helpful discussions.

Funding

Additional information

Funding

MH and JP were partially supported by the National Science Foundation through NSF-DMS-1418090. MH was partially supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R01GM123007 (NIH funding: R01GM123007).