Stratified Stochastic Variational Inference for High-Dimensional Network Factor Model

Abstract

There has been considerable recent interest in Bayesian modeling of high-dimensional networks via latent space approaches. When the number of nodes increases, estimation based on Markov chain Monte Carlo can be extremely slow and show poor mixing, thereby motivating research on alternative algorithms that scale well in high-dimensional settings. In this article, we focus on the latent factor model, a widely used approach for latent space modeling of network data. We develop scalable algorithms to conduct approximate Bayesian inference via stochastic optimization. Leveraging sparse representations of network data, the proposed algorithms show massive computational and storage benefits, and allow to conduct inference in settings with thousands of nodes. An R package with an efficient c++ implementation of the proposed algorithms is provided.

KEYWORDS:

• Bayesian inference
• Sparsity
• Stochastic optimization
• Variational methods

Supplementary Materials

Svilf_supplementary.pdf Supplemental materials containing additional simulation studies with varying latent dimensionality H, parameter γ and with the probit link function.

R-package: R-package svilf implementing the methods described in the article in c++, using convenient R wrappers. The package provides a common interface calling different implementations of SVILF with probit or logit link function, using uniform or adaptive sub-sampling. A tutorial illustrating the main functionalities of the package is also provided.

Acknowledgments

The authors would like to thank Bruno Scarpa, Peter Hoff and Daniele Durante for their comments and suggestions on the main idea of this work. This research used the HPC cluster system of the VERA center at Ca’ Foscari University of Venice.

Additional information

Funding

The work of Emanuele Aliverti was partially funded by miur-prin 2017 project 20177BRJXS, as well as grant no. bird-188753/18 of the University of Padova, Italy.