Fast copula variational inference

ABSTRACT

Mean-field variational inference, built on fully factorisations, can be efficiently solved; however, it ignores the dependencies between latent variables, resulting in lower performance. To address this, the copula variational inference (CVI) method is proposed by using the well-established copulas to effectively capture posterior dependencies, leading to better approximations. However, it suffers from a computational issue, where the optimisation for big models with massive latent variables is quite time-consuming. This is mainly caused by the expensive sampling when forming noisy Monte Carlo gradients in CVI. For CVI speedup, in this paper we propose a novel fast CVI (abbr. FCVI). In FCVI, we derive the gradient of CVI objective by an expectation of the mean-field factorisation. Therefore, we can achieve a much efficient sampling from the $D$-dimensional mean-field factorisation, enabling to reduce the sampling complexity from $O(D^2)$ to $O(D)$. To evaluate FCVI, we compare it against baseline methods on modelling performance and runtime. Experimental results demonstrate that FCVI is on a par with CVI, but runs much faster.

KEYWORDS:

• Variational inference
• dependency
• copula
• time complexity

Acknowledgments

This work was supported by China Postdoctoral Science Foundation [grant number 2019M661210], National Natural Science Foundation of China (NSFC) [grant number 62006094, 61876071] and State Key Laboratory of applied optics.

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This work was supported by the National Natural Science Foundation of China [61876071,62006094]; China Postdoctoral Science Foundation [2019M661210].