Pseudo-Marginal Inference for CTMCs on Infinite Spaces via Monotonic Likelihood Approximations

Abstract

Bayesian inference for Continuous-Time Markov chains (CTMCs) on countably infinite spaces is notoriously difficult because evaluating the likelihood exactly is intractable. One way to address this challenge is to first build a nonnegative and unbiased estimate of the likelihood—involving the matrix exponential of finite truncations of the true rate matrix—and then to use the estimates in a pseudo-marginal inference method. In this work, we show that we can dramatically increase the efficiency of this approach by avoiding the computation of exact matrix exponentials. In particular, we develop a general methodology for constructing an unbiased, nonnegative estimate of the likelihood using doubly-monotone matrix exponential approximations. We further develop a novel approximation in this family—the skeletoid—as well as theory regarding its approximation error and how that relates to the variance of the estimates used in pseudo-marginal inference. Experimental results show that our approach yields more efficient posterior inference for a wide variety of CTMCs. Supplementary materials for this article are available online.

KEYWORDS:

• Continuous-time Markov chain
• Markov chain Monte Carlo
• Matrix exponential
• Pseudo-marginal methods
• Reaction networks
• Stochastic processes

Supplementary Materials

The supplementary materials contain implementation details, proofs, and R code.

Michael Smith Foundation for Health Research Scholar Award;Natural Sciences and Engineering Research Council of Canada;Natural Sciences and Engineering Research Council of Canada;

Disclosure Statement

The authors report there are no competing interests to declare.

Funding

Footnotes

Acknowledgments

The authors thank the associate editor and three anonymous referees for their insightful comments, which have substantially improved the quality of this work.

Notes

1 We do this because the mean rate confounds the effect of varying t, given that e^tQ depends on t and Q only through their product. By fixing the mean rate we remove this ambiguity from the results. Note that the choice of unit mean rate is without loss of generality, since, for example, choosing Q with a mean rate of 2 is equivalent to choosing the rate matrix $Q/2$ and doubling t.

2 Code available at https://github.com/ageorgou/roulette/ and Sherlock and Golightly (2019),³ The authors kindly shared their code with us.

Additional information

Funding

A. Bouchard-Côté is supported by a National Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant and Michael Smith Foundation COVID-19 Research Response. T. Campbell is supported by an NSERC Discovery Grant and an NSERC Discovery Launch Supplement.