Implicit Copula Variational Inference

Abstract

Key to effective generic, or “black-box,” variational inference is the selection of an approximation to the target density that balances accuracy and speed. Copula models are promising options, but calibration of the approximation can be slow for some choices. Smith, Loaiza-Maya, and Nott (2020) suggest using tractable and scalable “implicit copula” models that are formed by element-wise transformation of the target parameters. We propose an adjustment to these transformations that make the approximation invariant to the scale and location of the target density. We also show how a sub-class of elliptical copulas have a generative representation that allows easy application of the re-parameterization trick and efficient first order optimization. We demonstrate the estimation methodology using two statistical models as examples. The first is a mixed effects logistic regression, and the second is a regularized correlation matrix. For the latter, standard Markov chain Monte Carlo estimation methods can be slow or difficult to implement, yet our proposed variational approach provides an effective and scalable estimator. We illustrate by estimating a regularized Gaussian copula model for income inequality in U.S. states between 1917 and 2018. An Online Appendix and MATLAB code to implement the method are available as supplementary materials.

Keyword:

• Elliptical copula
• Factor model approximation
• Re-parameterization trick
• Spherical co-ordinates
• Stochastic gradient ascent
• Variational Bayes

Supplementary Materials

Online Appendix:provides further details on notation used, the derivatives, examples and MATLAB code.

Code and Data Zip File:provides MATLAB code and data to replicate our results, and apply the method to other datasets and/or models.

Acknowledgments

We thank Professor David Nott for discussions on the use of spherical co-ordinates. We also thank two referees, an Associate Editor and the Editor, Professor Galin Jones, for providing comments that improved the manuscript. We are grateful to Professor Mark Frank for placing his inequality data in the public domain.

Disclosure Statement

No potential competing interest was reported by the authors.