High-Dimensional Copula Variational Approximation Through Transformation

References

• Aas, K., Czado, C., Frigessi, A., and Bakken, H. (2009), “Pair-Copula Constructions of Multiple Dependence,” Insurance: Mathematics and Economics, 44, 182–198. DOI: 10.1016/j.insmatheco.2007.02.001.
   Web of Science ®Google Scholar
• Archer, E., Park, I. M., Buesing, L., Cunningham, J., and Paninski, L. (2016), “Black Box Variational Inference for State Space Models,” arXiv no. 1511.07367.
   Google Scholar
• Azzalini, A., and Capitanio, A. (2003), “Distributions Generated by Perturbation of Symmetry With Emphasis on a Multivariate Skew t-Distribution,” Journal of the Royal Statistical Society, Series B, 65, 367–389. DOI: 10.1111/1467-9868.00391.
   Google Scholar
• Azzalini, A., and Dalla Valle, A. (1996), “The Multivariate Skew-Normal Distribution,” Biometrika, 83, 715–726. DOI: 10.1093/biomet/83.4.715.
   Web of Science ®Google Scholar
• Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017), “Variational Inference: A Review for Statisticians,” Journal of the American Statistical Association, 112, 859–877. DOI: 10.1080/01621459.2017.1285773.
   Web of Science ®Google Scholar
• Bottou, L. (2010), “Large-Scale Machine Learning With Stochastic Gradient Descent,” in Proceedings of the 19th International Conference on Computational Statistics (COMPSTAT’2010), eds. Y. Lechevallier and G. Saporta, Springer, pp.177–187.
   Google Scholar
• Carpenter, B., Gelman, A., Hoffman, M., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. (2017), “Stan: A Probabilistic Programming Language,” Journal of Statistical Software, 76, 1–32. DOI: 10.18637/jss.v076.i01.
   Web of Science ®Google Scholar
• Challis, E., and Barber, D. (2013), “Gaussian Kullback–Leibler Approximate Inference,” The Journal of Machine Learning Research, 14, 2239–2286.
   Web of Science ®Google Scholar
• Demarta, S., and McNeil, A. J. (2005), “The t Copula and Related Copulas,” International Statistical Review, 73, 111–129. DOI: 10.1111/j.1751-5823.2005.tb00254.x.
   Web of Science ®Google Scholar
• Dinh, L., Sohl-Dickstein, J., and Bengio, S. (2016), “Density Estimation Using Real NVP,” arXiv no. 1605.08803.
   Google Scholar
• Elidan, G. (2010), “Copula Bayesian Networks,” in Advances in Neural Information Processing Systems (Vol. 23), eds. J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta, La Jolla, CA: NIPS Foundation, pp. 559–567.
   Google Scholar
• Fang, H.-B., Fang, K.-T., and Kotz, S. (2002), “The Meta-Elliptical Distributions With Given Marginals,” Journal of Multivariate Analysis, 82, 1–16. DOI: 10.1006/jmva.2001.2017.
   Web of Science ®Google Scholar
• Genest, C., and Nešlehová, J. (2007), “A Primer on Copulas for Count Data,” ASTIN Bulletin: The Journal of the IAA, 37, 475–515. DOI: 10.2143/AST.37.2.2024077.
   Web of Science ®Google Scholar
• Genton, M. G. (2004), Skew-Elliptical Distributions and Their Applications: A Journey Beyond Normality, Boca Raton, FL: CRC Press.
   Google Scholar
• Guo, F., Wang, X., Broderick, T., and Dunson, D. B. (2016), “Boosting Variational Inference,” arXiv no. 1611.05559.
   Google Scholar
• Han, S., Liao, X., Dunson, D., and Carin, L. (2016), “Variational Gaussian Copula Inference,” in Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research (Vol. 51), eds. A. Gretton and C. C. Robert, Cadiz, Spain: PMLR, pp. 829–838.
   Google Scholar
• Headrick, T. C., Kowalchuk, R. K., and Sheng, Y. (2008), “Parametric Probability Densities and Distribution Functions for Tukey g-and-h Transformations and Their Use for Fitting Data,” Applied Mathematical Sciences, 2, 449–462.
   Google Scholar
• Hosmer, D. W., Lemeshow, S., and Sturdivant, R. X. (2013), Applied Logistic Regression (3rd ed.), Hoboken, NJ: Wiley.
   Google Scholar
• Huszár, F. (2017), “Variational Inference Using Implicit Distributions,” arXiv no. 1702.08235.
   Google Scholar
• Kingma, D. P., and Welling, M. (2014), “Auto-Encoding Variational Bayes,” arXiv no. 1312.6114.
   Google Scholar
• Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., and Blei, D. M. (2017), “Automatic Differentiation Variational Inference,” Journal of Machine Learning Research, 18, 430–474.
   Web of Science ®Google Scholar
• Liu, Q., and Wang, D. (2016), “Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm,” in Advances in Neural Information Processing Systems, pp. 2378–2386.
   Google Scholar
• Loaiza-Maya, R., and Smith, M. S. (2019), “Variational Bayes Estimation of Discrete-Margined Copula Models With Application to Time Series,” Journal of Computational and Graphical Statistics, 28, 523–539. DOI: 10.1080/10618600.2018.1562936.
   Web of Science ®Google Scholar
• McNeil, A. J., Frey, R., and Embrechts, P. (2005), Quantitative Risk Management: Concepts, Techniques and Tools, Princeton Series in Finance.
   Google Scholar
• Miller, A. C., Foti, N., and Adams, R. P. (2016), “Variational Boosting: Iteratively Refining Posterior Approximations,” arXiv no. 1611.06585.
   Google Scholar
• Murray, J. S., Dunson, D. B., Carin, L., and Lucas, J. E. (2013), “Bayesian Gaussian Copula Factor Models for Mixed Data,” Journal of the American Statistical Association, 108, 656–665. DOI: 10.1080/01621459.2012.762328.
   PubMed Web of Science ®Google Scholar
• Nelsen, R. B. (2006), An Introduction to Copulas, Springer Series in Statistics, Secaucus, NJ:Springer-Verlag New York, Inc.
   Google Scholar
• Oh, D. H., and Patton, A. J. (2017), “Modeling Dependence in High Dimensions With Factor Copulas,” Journal of Business & Economic Statistics, 35, 139–154. DOI: 10.1080/07350015.2015.1062384.
   Web of Science ®Google Scholar
• Ong, V. M.-H., Nott, D. J., and Smith, M. S. (2018), “Gaussian Variational Approximation With a Factor Covariance Structure,” Journal of Computational and Graphical Statistics, 27, 465–478. DOI: 10.1080/10618600.2017.1390472.
   Web of Science ®Google Scholar
• Opper, M., and Archambeau, C. (2009), “The Variational Gaussian Approximation Revisited,” Neural Computation, 21, 786–792. DOI: 10.1162/neco.2008.08-07-592.
   PubMed Web of Science ®Google Scholar
• Ormerod, J. T. (2011), “Skew-Normal Variational Approximations for Bayesian Inference,” Technical Report, School of Mathematics and Statistics, University of Sydney.
   Google Scholar
• Ormerod, J. T., and Wand, M. P. (2010), “Explaining Variational Approximations,” The American Statistician, 64, 140–153. DOI: 10.1198/tast.2010.09058.
   Web of Science ®Google Scholar
• Peters, G., Chen, W., and Gerlach, R. (2016), “Estimating Quantile Families of Loss Distributions for Non-Life Insurance Modelling via L-Moments,” Risks, 4, 14. DOI: 10.3390/risks4020014.
   Web of Science ®Google Scholar
• Quiroz, M., Nott, D. J., and Kohn, R. (2018), “Gaussian Variational Approximation for High-Dimensional State Space Models,” arXiv no. 1801.07873.
   Google Scholar
• Ranganath, R., Tran, D., and Blei, D. (2016), “Hierarchical Variational Models,” in Proceedings of the 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research (Vol. 48), M. F. Balcan and K. Q. Weinberger, New York, NY: PMLR, pp. 324–333.
   Google Scholar
• Rezende, D. J., and Mohamed, S. (2015), “Variational Inference With Normalizing Flows,” in Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research (Vol. 37), eds. F. Bach and D. Blei, Lille, France: PMLR, pp. 1530–1538.
   Google Scholar
• Rezende, D. J., Mohamed, S., and Wierstra, D. (2014), “Stochastic Backpropagation and Approximate Inference in Deep Generative Models,” in Proceedings of the 31st International Conference on Machine Learning, Proceedings of Machine Learning Research (Vol. 32), eds. E. P. Xing and T. Jebara, Bejing, China: PMLR, pp. 1278–1286.
   Google Scholar
• Robbins, H., and Monro, S. (1951), “A Stochastic Approximation Method,” The Annals of Mathematical Statistics, 22, 400–407. DOI: 10.1214/aoms/1177729586.
   Google Scholar
• Rue, H., Martino, S., and Chopin, N. (2009), “Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations,” Journal of the Royal Statistical Society, Series B, 71, 319–392. DOI: 10.1111/j.1467-9868.2008.00700.x.
   Web of Science ®Google Scholar
• Salimans, T., and Knowles, D. A. (2013), “Fixed-Form Variational Posterior Approximation Through Stochastic Linear Regression,” Bayesian Analysis, 8, 837–882. DOI: 10.1214/13-BA858.
   Web of Science ®Google Scholar
• Seeger, M. (2000), “Bayesian Model Selection for Support Vector Machines, Gaussian Processes and Other Kernel Classifiers,” in Advances in Neural Information Processing Systems (Vol. 12), eds. S. A. Solla, T. K. Leen, and K. Müller, Cambridge, MA: MIT Press, pp. 603–609.
   Google Scholar
• Smith, M. S. (2015), “Copula Modelling of Dependence in Multivariate Time Series,” International Journal of Forecasting, 31, 815–833. DOI: 10.1016/j.ijforecast.2014.04.003.
   Web of Science ®Google Scholar
• Smith, M. S., Gan, Q., and Kohn, R. J. (2012), “Modelling Dependence Using Skew t Copulas: Bayesian Inference and Applications,” Journal of Applied Econometrics, 27, 500–522. DOI: 10.1002/jae.1215.
   Web of Science ®Google Scholar
• Smith, M. S., and Khaled, M. (2012), “Estimation of Copula Models With Discrete Margins via Bayesian Data Augmentation,” Journal of the American Statistical Association, 107, 290–303. DOI: 10.1080/01621459.2011.644501.
   Web of Science ®Google Scholar
• Smith, M. S., and Maneesoonthorn, W. (2018), “Inversion Copulas From Nonlinear State Space Models With an Application to Inflation Forecasting,” International Journal of Forecasting, 34, 389–407. DOI: 10.1016/j.ijforecast.2018.01.002.
   Web of Science ®Google Scholar
• Song, X.-K. P. (2000), “Multivariate Dispersion Models Generated From Gaussian Copula,” Scandinavian Journal of Statistics, 27, 305–320.
   Web of Science ®Google Scholar
• Spantini, A., Bigoni, D., and Marzouk, Y. (2018), “Inference via Low-Dimensional Couplings,” Journal of Machine Learning Research, 19, 2639–2709.
   Web of Science ®Google Scholar
• Su, Q., Liao, X., Chen, C., and Carin, L. (2016), “Nonlinear Statistical Learning With Truncated Gaussian Graphical Models,” in ICML, JMLR Workshop and Conference Proceedings (Vol. 48), JMLR.org, pp. 1948–1957.
   Google Scholar
• Tan, L. S. L., and Nott, D. J. (2018), “Gaussian Variational Approximation With Sparse Precision Matrices,” Statistics and Computing, 28, 259–275. DOI: 10.1007/s11222-017-9729-7.
   Web of Science ®Google Scholar
• Titsias, M., and Lázaro-Gredilla, M. (2014), “Doubly Stochastic Variational Bayes for Non-Conjugate Inference,” in, Proceedings of the 31st International Conference on Machine Learning, Proceedings of Machine Learning Research (Vol. 32), eds. E. P. Xing and T. Jebara, Beijing, China: PMLR, pp. 1971–1979.
   Google Scholar
• Tran, D., Blei, D. M., and Airoldi, E. M. (2015), “Copula Variational Inference,” in Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7–12, 2015, Montreal, Quebec, Canada, pp. 3564–3572.
   Google Scholar
• Tukey, T. W. (1977), “Modern Techniques in Data Analysis,” in NSP-Sponsored Regional Research Conference at Southeastern Massachusetts University, North Dartmouth, MA.
   Google Scholar
• Xu, M., Quiroz, M., Kohn, R., and Sisson, S. A. (2018), “On Some Variance Reduction Properties of the Reparameterization Trick,” arXiv no. 1809.10330.
   Google Scholar
• Yeo, I.-K., and Johnson, R. A. (2000), “A New Family of Power Transformations to Improve Normality or Symmetry,” Biometrika, 87, 954–959. DOI: 10.1093/biomet/87.4.954.
   Web of Science ®Google Scholar
• Yoshiba, T. (2018), “Maximum Likelihood Estimation of Skew-t Copulas With Its Applications to Stock Returns,” Journal of Statistical Computation and Simulation, 88, 2489–2506. DOI: 10.1080/00949655.2018.1469631.
   Web of Science ®Google Scholar
• Zeiler, M. D. (2012), “ADADELTA: An Adaptive Learning Rate Method,” arXiv no. 1212.5701.
   Google Scholar