Bayesian Variational Inference for Exponential Random Graph Models

References

• Atchadé, Y. F., Lartillot, N., and Robert, C. (2013), “Bayesian Computation for Statistical Models With Intractable Normalizing Constants,” Brazilian Journal of Probability and Statistics, 27, 416–436. DOI: 10.1214/11-BJPS174.
   Web of Science ®Google Scholar
• Besag, J. (1974), “Spatial Interaction and the Statistical Analysis of Lattice Systems,” Journal of the Royal Statistical Society, Series B, 36, 192–236. DOI: 10.1111/j.2517-6161.1974.tb00999.x.
   Google Scholar
• Bouranis, L., Friel, N., and Maire, F. (2017), “Efficient Bayesian Inference for Exponential Random Graph Models by Correcting the Pseudo-Posterior Distribution,” Social Networks, 50, 98–108. DOI: 10.1016/j.socnet.2017.03.013.
   Web of Science ®Google Scholar
• Bouranis, L., Friel, N., and Maire, F. (2018), “Bayesian Model Selection for Exponential Random Graph Models via Adjusted Pseudolikelihoods,” Journal of Computational and Graphical Statistics, 27, 516–528.
   Web of Science ®Google Scholar
• Burda, Y., Grosse, R., and Salakhutdinov, R. (2016), “Importance Weighted Autoencoders,” in Proceedings of the 4th International Conference on Learning Representations (ICLR).
   Google Scholar
• Caimo, A., and Friel, N. (2011), “Bayesian Inference for Exponential Random Graph Models,” Social Networks, 33, 41–55. DOI: 10.1016/j.socnet.2010.09.004.
   Web of Science ®Google Scholar
• Caimo, A., and Friel, N. (2013), “Bayesian Model Selection for Exponential Random Graph Models,” Social Networks, 35, 11–24.
   Web of Science ®Google Scholar
• Caimo, A., and Friel, N. (2014), “Bergm: Bayesian Exponential Random Graphs in R,” Journal of Statistical Software, 61, 1–25.
   Web of Science ®Google Scholar
• Chen, J., and Luss, R. (2019), “Stochastic Gradient Descent With Biased but Consistent Gradient Estimators,” arXiv no. 1807.11880.
   Google Scholar
• Chen, J., Ma, T., and Xiao, C. (2018), “Fastgcn: Fast Learning With Graph Convolutional Networks via Importance Sampling.” arXiv no. 1801.10247.
   Google Scholar
• Chib, S., and Jeliazkov, I. (2001), “Marginal Likelihood From the Metropolis-Hastings Output,” Journal of the American Statistical Association, 96, 270–281. DOI: 10.1198/016214501750332848.
   Web of Science ®Google Scholar
• Gal, Y. (2016), “Uncertainty in Deep Learning,” PhD thesis, University of Cambridge.
   Google Scholar
• Geyer, C. J., and Thompson, E. A. (1992), “Constrained Monte Carlo Maximum Likelihood for Dependent Data,” Journal of the Royal Statistical Society, Series B, 54, 657–699. DOI: 10.1111/j.2517-6161.1992.tb01443.x.
   Google Scholar
• Hummel, R. M., Hunter, D. R., and Handcock, M. S. (2012), “Improving Simulation-Based Algorithms for Fitting ERGMs,” Journal of Computational and Graphical Statistics, 21, 920–939. DOI: 10.1080/10618600.2012.679224.
   PubMed Web of Science ®Google Scholar
• Hunter, D. R., Goodreau, S. M., and Handcock, M. S. (2008), “Goodness of Fit of Social Network Models,” Journal of the American Statistical Association, 103, 248–258. DOI: 10.1198/016214507000000446.
   Web of Science ®Google Scholar
• Hunter, D. R., Handcock, M. S., Butts, C. T., Goodreau, S. M., and Morris, M. (2008), “ergm: A Package to Fit, Simulate and Diagnose Exponential-Family Models for Networks,” Journal of Statistical Software, 24, 1–29. DOI: 10.18637/jss.v024.i03.
   PubMedGoogle Scholar
• Kass, R. E., and Raftery, A. E. (1995), “Bayes Factors,” Journal of the American Statistical Association, 90, 773–795. DOI: 10.1080/01621459.1995.10476572.
   Web of Science ®Google Scholar
• Kingma, D. P., and Ba, J. (2014), “Adam: A Method for Stochastic Optimization,” arXiv no. 1412.6980.
   Google Scholar
• Kingma, D. P., and Welling, M. (2014), “Auto-Encoding Variational Bayes,” in Proceedings of the 2nd International Conference on Learning Representations (ICLR).
   Google Scholar
• Knowles, D. A., and Minka, T. (2011), “Non-Conjugate Variational Message Passing for Multinomial and Binary Regression,” in Advances in Neural Information Processing Systems (Vol. 24), eds. J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, Curran Associates, Inc., pp. 1701–1709.
   Google Scholar
• Kong, A., Liu, J. S., and Wong, W. H. (1994), “Sequential Imputations and Bayesian Missing Data Problems,” Journal of the American Statistical Association, 89, 278–288. DOI: 10.1080/01621459.1994.10476469.
   Web of Science ®Google Scholar
• Le, T. A., Kosiorek, A. R., Siddharth, N., Teh, Y. W., and Wood, F. (2019), “Revisiting Reweighted Wake-Sleep for Models With Stochastic Control Flow,” in Uncertainty in Artificial Intelligence (to appear).
   Google Scholar
• Liang, F. (2010), “A Double Metropolis-Hastings Sampler for Spatial Models With Intractable Normalizing Constants,” Journal of Statistical Computation and Simulation, 80, 1007–1022. DOI: 10.1080/00949650902882162.
   Web of Science ®Google Scholar
• Liang, F., Jin, I. H., Song, Q., and Liu, J. S. (2016), “An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants,” Journal of the American Statistical Association, 111, 377–393. DOI: 10.1080/01621459.2015.1009072.
   Web of Science ®Google Scholar
• Liu, J. S. (2004), Monte Carlo Strategies in Scientific Computing, New York: Springer-Verlag.
   Google Scholar
• Liu, Q., and Pierce, D. A. (1994), “A Note on Gauss-Hermite Quadrature,” Biometrika, 81, 624–629. DOI: 10.2307/2337136.
   Web of Science ®Google Scholar
• Lyne, A.-M., Girolami, M., Atchad, Y., Strathmann, H., and Simpson, D. (2015), “On Russian Roulette Estimates for Bayesian Inference With Doubly-Intractable Likelihoods,” Statistical Science, 30, 443–467. DOI: 10.1214/15-STS523.
   Web of Science ®Google Scholar
• Martino, L., Elvira, V., and Louzada, F. (2017), “Effective Sample Size for Importance Sampling Based on Discrepancy Measures,” Signal Processing, 131, 386–401. DOI: 10.1016/j.sigpro.2016.08.025.
   Web of Science ®Google Scholar
• Møller, J., Pettitt, A. N., Reeves, R., and Berthelsen, K. K. (2006), “An Efficient Markov Chain Monte Carlo Method for Distributions With Intractable Normalising Constants,” Biometrika, 93, 451–458. DOI: 10.1093/biomet/93.2.451.
   Web of Science ®Google Scholar
• Murray, I., Ghahramani, Z., and MacKay, D. J. C. (2006), “MCMC for Doubly-Intractable Distributions,” in Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence, UAI’06, Arlington, VA: AUAI Press, pp. 359–366.
   Google Scholar
• Neal, R. M. (2001), “Annealed Importance Sampling,” Statistics and Computing, 11, 125–139. DOI: 10.1023/A:1008923215028.
   Web of Science ®Google Scholar
• Ormerod, J. T., and Wand, M. P. (2012), “Gaussian Variational Approximate Inference for Generalized Linear Mixed Models,” Journal of Computational and Graphical Statistics, 21, 2–17. DOI: 10.1198/jcgs.2011.09118.
   Web of Science ®Google Scholar
• Park, J., and Haran, M. (2018), “Bayesian Inference in the Presence of Intractable Normalizing Functions,” Journal of the American Statistical Association, 113, 1372–1390. DOI: 10.1080/01621459.2018.1448824.
   Web of Science ®Google Scholar
• Pearson, M., and Michell, L. (2000), “Smoke Rings: Social Network Analysis of Friendship Groups, Smoking and Drug-Taking,” Drugs: Education, Prevention and Policy, 7, 21–37. DOI: 10.1080/713660095.
   Web of Science ®Google Scholar
• Rezende, D. J., Mohamed, S., and Wierstra, D. (2014), “Stochastic Backpropagation and Approximate Inference in Deep Generative Models,” in Proceedings of the 31st International Conference on Machine Learning, JMLR Workshop and Conference Proceedings, eds. E. P. Xing and T. Jebara, pp. 1278–1286.
   Google Scholar
• Robbins, H., and Monro, S. (1951), “A Stochastic Approximation Method,” The Annals of Mathematical Statistics, 22, 400–407. DOI: 10.1214/aoms/1177729586.
   Google Scholar
• Roeder, G., Wu, Y., and Duvenaud, D. K. (2017), “Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference,” in Advances in Neural Information Processing Systems (Vol. 30), eds. I. Guyon, U. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett.
   Google Scholar
• Salgado, H., Santos-Zavaleta, A., Gama-Castro, S., Millán-Zárate, D., Díaz-Peredo, E., Sánchez-Solano, F., Pérez-Rueda, E., Bonavides-Martínez, C., and Collado-Vides, J. (2001), “Regulondb (Version 3.2): Transcriptional Regulation and Operon Organization in Escherichia coli K-12,” Nucleic Acids Research, 29, 72–74. DOI: 10.1093/nar/29.1.72.
   PubMed Web of Science ®Google Scholar
• Saul, Z. M., and Filkov, V. (2007), “Exploring Biological Network Structure Using Exponential Random Graph Models,” Bioinformatics, 23, 2604–2611. DOI: 10.1093/bioinformatics/btm370.
   PubMed Web of Science ®Google Scholar
• Shen-Orr, S., Milo, R., Mangan, S., and Alon, U. (2002), “Network Motifs in the Transcriptional Regulation Network of Escherichia coli,” Nature Genetics, 31, 64–68. DOI: 10.1038/ng881.
   PubMed Web of Science ®Google Scholar
• Snijders, T. A. (2002), “Markov Chain Monte Carlo Estimation of Exponential Random Graph Models,” Journal of Social Structure, 3, 1–40.
   Google Scholar
• Spall, J. C. (2003), Introduction to Stochastic Search and Optimization: Estimation, Simulation and Control, Hoboken, NJ: Wiley.
   Google Scholar
• Strauss, D., and Ikeda, M. (1990), “Pseudolikelihood Estimation for Social Networks,” Journal of the American Statistical Association, 85, 204–212. DOI: 10.1080/01621459.1990.10475327.
   Web of Science ®Google Scholar
• Tadić, V. B., and Doucet, A. (2017), “Asymptotic Bias of Stochastic Gradient Search,” Annals of Applied Probability, 27, 3255–3304. DOI: 10.1214/16-AAP1272.
   Web of Science ®Google Scholar
• Tan, L. S. L. (2020), “Use of Model Reparametrization to Improve Variational Bayes,” arXiv no. 1805.07267.
   Google Scholar
• Tan, L. S. L., and Nott, D. J. (2013), “Variational Inference for Generalized Linear Mixed Models Using Partially Non-Centered Parametrizations,” Statistical Science, 28, 168–188. DOI: 10.1214/13-STS418.
   Web of Science ®Google Scholar
• Tan, L. S. L., and Nott, D. J. (2018), “Gaussian Variational Approximation With Sparse Precision Matrices,” Statistics and Computing, 28, 259–275.
   Web of Science ®Google Scholar
• Titsias, M., and Lázaro-Gredilla, M. (2014), “Doubly Stochastic Variational Bayes for Non-Conjugate Inference,” in Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1971–1979.
   Google Scholar
• Tran, M.-N., Nott, D. J., and Kohn, R. (2017), “Variational Bayes With Intractable Likelihood,” Journal of Computational and Graphical Statistics, 26, 873–882. DOI: 10.1080/10618600.2017.1330205.
   Web of Science ®Google Scholar
• van Duijn, M. A., Gile, K. J., and Handcock, M. S. (2009), “A Framework for the Comparison of Maximum Pseudo-Likelihood and Maximum Likelihood Estimation of Exponential Family Random Graph Models,” Social Networks, 31, 52–62. DOI: 10.1016/j.socnet.2008.10.003.
   PubMed Web of Science ®Google Scholar
• Wand, M. P. (2014), “Fully Simplified Multivariate Normal Updates in Non-Conjugate Variational Message Passing,” Journal of Machine Learning Research, 15, 1351–1369.
   Web of Science ®Google Scholar
• Williams, R. J. (1992), “Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning,” Machine Learning, 8, 229–256. DOI: 10.1007/BF00992696.
   Web of Science ®Google Scholar
• Winn, J., and Bishop, C. M. (2005), “Variational Message Passing,” Journal of Machine Learning Research, 6, 661–694.
   Web of Science ®Google Scholar
• Xu, M., Quiroz, M., Kohn, R., and Sisson, S. A. (2019), “Variance Reduction Properties of the Reparameterization Trick,” in Proceedings of Machine Learning Research, Proceedings of Machine Learning Research (PMLR) (Vol. 89), eds. K. Chaudhuri and M. Sugiyama, pp. 2711–2720.
   Google Scholar
• Zachary, W. W. (1977), “An Information Flow Model for Conflict and Fission in Small Groups,” Journal of Anthropological Research, 33, 452–473. DOI: 10.1086/jar.33.4.3629752.
   Web of Science ®Google Scholar