MCMC Algorithms for Posteriors on Matrix Spaces

Abstract

We study Markov chain Monte Carlo (MCMC) algorithms for target distributions defined on matrix spaces. Such an important sampling problem has yet to be analytically explored. We carry out a major step in covering this gap by developing the proper theoretical framework that allows for the identification of ergodicity properties of typical MCMC algorithms, relevant in such a context. Beyond the standard Random-Walk Metropolis (RWM) and preconditioned Crank–Nicolson (pCN), a contribution of this article in the development of a novel algorithm, termed the “Mixed” pCN (MpCN). RWM and pCN are shown not to be geometrically ergodic for an important class of matrix distributions with heavy tails. In contrast, MpCN is robust across targets with different tail behavior and has very good empirical performance within the class of heavy-tailed distributions. Geometric ergodicity for MpCN is not fully proven in this work, as some remaining drift conditions are quite challenging to obtain owing to the complexity of the state space. We do, however, make a lot of progress toward a proof, and show in detail the last steps left for future work. We illustrate the computational performance of the various algorithms through numerical applications, including calibration on real data of a challenging model arising in financial statistics. Supplementary materials for this article are available online.

Keywords:

• Drift condition
• Matrix-valued stochastic differential equation
• Preconditioned Crank–Nicolson

Supplementary Materials

R and C code for the analysis in Section 6 (code.zip).

Acknowledgments

We thank an anonymous referee and the Associate Editor for suggestions that have greatly improved the contents of this article.

Additional information

Funding

Alexandros Beskos acknowledges support from a Leverhulme Trust Prize grant number PLP-2014-263. Kengo Kamatani acknowledges support from JSPS KAKENHI grant numbers 16K00046, 20H04149 and JST CREST grant number JPMJCR14D7.