Variational Bayes for Fast and Accurate Empirical Likelihood Inference

References

• Bahdanau, D., Cho, K., and Yoshua, B. (2015), “Neural Machine Translation by Jointly Learning to Align and Translate,” in Proceedings of the 3rd International Conference on Learning Representations.
   Google Scholar
• Bedoui, A., and Lazar, N. A. (2020), “Bayesian Empirical Likelihood for Ridge and Lasso Regression,” Computational Statistics and Data Analysis, 145, 106917. DOI: 10.1016/j.csda.2020.106917.
   Web of Science ®Google Scholar
• Betancourt, M. (2017), “A Conceptual Introduction to Hamiltonian Monte Carlo,” arXiv.
   Google Scholar
• Betancourt, M., Byrne, S., and Girolami, M. (2015), “Optimising the Integrator Step Size for Hamiltonian Monte Carlo,” arXiv.
   Google Scholar
• Blei, D. M., and Jordan, M. I. (2006),” Variational Inference for Dirichlet Processes,” Bayesian Analysis, 1, 121–143. DOI: 10.1214/06-BA104.
   Web of Science ®Google Scholar
• Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017), “Variational Inference: A Review for Statisticians,” Journal of American Statistical Association, 112, 859–877. DOI: 10.1080/01621459.2017.1285773.
   Web of Science ®Google Scholar
• Bottou, L. (2010), “Large-Scale Machine Learning with Stochastic Gradient Descent, in Proceedings of COMPSTAT’2010.
   Google Scholar
• Braides, A. (2006), “A Handbook of Γ-Convergence,” in Handbook of Differential Equations, Stationary Partial Differential Equations (Vol. 3), eds. M. Chipot, and P. Quittner,pp. 101–213, Amsterdam: North-Holland.
   Google Scholar
• Chambers, J. M., and Hastie, T. J. (1991), Statistical Models in S, Pacific Grove, CA: Wadsworth and Brooks.
   Google Scholar
• Chan, N. H., and Ling, S. (2006), “Empirical Likelihood for Garch Models,” Econometric Theory, 22, 403–428. DOI: 10.1017/S0266466606060208.
   Web of Science ®Google Scholar
• Chaudhuri, S., and Ghosh, M. (2011), “Empirical Likelihood for Small Area Estimation,” Biometrika, 98, 473–480. DOI: 10.1093/biomet/asr004.
   Web of Science ®Google Scholar
• Chaudhuri, S., Ghosh, S., Nott, D. J., and Pham, K. C. (2018), “An Easy-to-Use Empirical Likelihood ABC Method,” arXiv.
   Google Scholar
• Chaudhuri, S., Mondal, D., and Yin, T. (2017), “Hamiltonian Monte Carlo Sampling in Bayesian Empirical Likelihood Computation,” Journal of the Royal Statistical Society, Series B, 79, 293–320. DOI: 10.1111/rssb.12164.
   Google Scholar
• Chen, J., Variyath, A. M., and Abraham, B. (2008), “Adjusted Empirical Likelihood and its Properties,” Journal of Computational and Graphical Statistics, 17, 426–443.
   Web of Science ®Google Scholar
• Chernozhukov, V., and Hong, H. (2003), “An MCMC Approach to Classical Estimation,” Journal of Econometrics, 115, 293–346.
   Web of Science ®Google Scholar
• Chib, S., Shin, M., and Simoni, A. (2018), “Bayesian Estimation and Comparison of Moment Condition Models,” Journal of the American Statistical Association, 113, 1656–1668. DOI: 10.1080/01621459.2017.1358172.
   Web of Science ®Google Scholar
• Dal Maso, G. (1993), An Introduction to Γ-Convergence, Boston: Springer.
   Google Scholar
• Derigs, U. (1988), “Solving Non-bipartite Match Problems via Shortest Path Techniques,” Annals of Operations Research, 13, 225–261. DOI: 10.1007/BF02288324.
   Google Scholar
• Duane, S., Kennedy, A. D., Pendleton, B. J., and Roweth, D. (1987), “Hybrid Monte Carlo,” Physics Letters B, 195, 216–222. DOI: 10.1016/0370-2693(87)91197-X.
   Web of Science ®Google Scholar
• Fang, K. T., and Mukerjee, R. (2006), “Empirical-Type Likelihoods Allowing Posterior Credible Sets with Frequentist Validity: Higher-Order Asymptotics,” Biometrika, 93, 723–733. DOI: 10.1093/biomet/93.3.723.
   Web of Science ®Google Scholar
• Gal, Y. (2016), “Uncertainty in Deep Learning,” Ph.D. thesis, University of Cambridge.
   Google Scholar
• Hastings, W. K. (1970), “Monte Carlo Sampling Methods using Markov Chains and their Applications,” Biometrika, 57, 97–109. DOI: 10.1093/biomet/57.1.97.
   Web of Science ®Google Scholar
• Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J. (2013), “Stochastic Variational Inference,” Journal of Machine Learning Research, 14, 1303–1347.
   Web of Science ®Google Scholar
• Hoffman, M. D., and Gelman, A. (2014), “The No-U-Turn Sampler: Adaptive Setting Path Lengths in Hamiltonian Monte Carlo,” Journal of Machine Learning Research, 15, 1593–1623.
   Web of Science ®Google Scholar
• Ji, C., Shen, H., and West, M. (2010), “Bounded Approximations for Marginal Likelihoods,” Technical Report. Institute of Decision Sciences, Duke University.
   Google Scholar
• Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., and Saul, L. K. (1999), “An Introduction to Variational Methods for Graphical Models,” Machine Learning, 37, 183–233. DOI: 10.1023/A:1007665907178.
   Web of Science ®Google Scholar
• Kleijn, B. J. K., and van der Vaart, A. W. (2012), “The Bernstein-von-Mises Theorem under Misspecification,” Electronic Journal of Statistics, 6, 354–381. DOI: 10.1214/12-EJS675.
   Web of Science ®Google Scholar
• Kolaczyk, E. D. (1994), “Empirical Likelihood for Generalized Linear Models,” Statistica Sinica, 4, 199–218.
   Web of Science ®Google Scholar
• Lahiri, S. N., and Mukhopadhyay, S. (2012), “A Penalized Empirical Likelihood Method in High Dimensions,” Annals of Statistics, 40, 2511–2540.
   Web of Science ®Google Scholar
• Lai, W., Chen, R., Chen, Y., and Koch, T. (2022), “Variational Bayesian Inference for Network Autoregression Models,” Computational Statistics and Data Analysis, 169, 107406. DOI: 10.1016/j.csda.2021.107406.
   Web of Science ®Google Scholar
• Lazar, N. A. (2003), “Bayesian Empirical Likelihood,” Biometrika, 90, 319–326. DOI: 10.1093/biomet/90.2.319.
   Web of Science ®Google Scholar
• Liu, Y., and Chen, J. (2010), “Adjusted Empirical Likelihood with High-Order Precision,” Annals of Statistics, 38, 1341–1362.
   Web of Science ®Google Scholar
• Mengersen, K. L., Pudlo, P., and Robert, C. P. (2013), “Bayesian Computation via Empirical Likelihood,” Proceeedings of the National Academy of Sciences, 110, 1321–1326. DOI: 10.1073/pnas.1208827110.
   PubMed Web of Science ®Google Scholar
• Monahan, J. F., and Boos, D. D. (1992), “Proper Likelihoods for Bayesian Analysis,” Biometrika 79, 271–278. DOI: 10.1093/biomet/79.2.271.
   Web of Science ®Google Scholar
• Neal, R. M. (2006), “Bayesian Learning for Neural Networks PhD thesis,” Ph.D. thesis, University of Toronto.
   Google Scholar
• Ormerod, J. T., and Wand, M. P. (2010), “Explaining Variational Approximations,” The American Statistician, 64, 140–153. DOI: 10.1198/tast.2010.09058.
   Web of Science ®Google Scholar
• Owen, A. (1988), “Empirical Likelihood Ratio Confidence Intervals for a Single Functional,” Biometrika, 75, 237–249. DOI: 10.1093/biomet/75.2.237.
   Web of Science ®Google Scholar
• Owen, A. (1990), “Empirical Likelihood Ratio Confidence Regions,” The Annals of Statistics, 18, 90–120.
   Web of Science ®Google Scholar
• Park, T., and Casella, G. (2008), “The Bayesian Lasso,” Journal of the American Statistical Association, 103, 681–686. DOI: 10.1198/016214508000000337.
   Web of Science ®Google Scholar
• Peng, H., and Schick, A. (2018), “Maximum Empirical Likelihood Estimation and Related Topics,” Electronic Journal of Statistics, 12, 2962–2994. DOI: 10.1214/18-EJS1471.
   Web of Science ®Google Scholar
• Qin, J., and Lawless, J. (1994), “Empirical Likelihood and General Estimating Equations,” The Annals of Statistics, 22, 300–325. DOI: 10.1214/aos/1176325370.
   Web of Science ®Google Scholar
• Quiroz, M., Nott, D. J., and Kohn, R. (2018), “Gaussian Variational Approximation for High-Dimensional State Space Models,” arXiv.
   Google Scholar
• Rao, J. N. K., and Wu, C. (2010), “Bayesian Pseudo-Empirical-Likelihood Intervals for Complex Surveys,” Journal of the Royal Statistical Society, Series B, 72, 533–544.
   Google Scholar
• Robbins, H., and Monro, S. (1951), “A Stochastic Approximation Method,” Annals of Mathematical Statistics, 22, 400–407.
   Google Scholar
• Robert, C. P. (2015), “The Metropolis-Hastings Algorithm”, arXiv.
   Google Scholar
• Roeder, G., Wu, Y., and Duvenaud, D. (2017), “Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference, in 31st Conference on Neural Information Processing Systems.
   Google Scholar
• Rosenbaum, P. R. (2005), “An Exact Distribution-Free Test Comparing Two Multivariate Distributions based on Adjacency,” Journal of the Royal Statistical Society, Series B, 67, 515–530.
   Google Scholar
• Ruiz, F. R., Titsias, M. K., and Blei, D. (2016), “The Generalized Reparameterization Gradient,” in Advances in Neural Information Processing Systems, pp. 460–468.
   Google Scholar
• Schennach, S. M. (2005), “Bayesian Exponentially Tilted Empirical Likelihood,” Biometrika, 92, 31–46.
   Web of Science ®Google Scholar
• Shurman, J. (2006), “Approximation by Smooth Functions,” in Calculus and Analysis in Euclidean Space, pp. 343–373, Cham: Springer.
   Google Scholar
• Tan, L. S. L. (2021), “Use of Model Reparameterization to Improve Variational Bayes,” Journal of the Royal Statistical Society, Series B, 83, 30–57.
   Google Scholar
• Tan, L. S. L., and Nott, D. J. (2018), “Gaussian Variational Approximation with Sparse Precision Matrices,” Statistics and Computing, 28, 259–275.
   Web of Science ®Google Scholar
• Teh, Y. W., Newman, D., and Welling, M. (2007), “A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation,” in Advances in Neural Information Processing Systems (Vol. 19), pp. 1353–1360, MIT Press.
   Google Scholar
• Titsias, M. K., and Lázaro-Gredilla, M. (2014), “Doubly Stochastic Variational Bayes for Non-conjugate Inference,” in Proceedings of the 31st International Conference on Machine Learning.
   Google Scholar
• Tsao, M., and Wu, F. (2014), “Extended Empirical Likelihood for Estimating Equations,” Biometrika, 101, 703–710.
   Web of Science ®Google Scholar
• Vexler, A., Tao, G., and Hutson, A. D. (2014), “Posterior Expectation based on Empirical Likelihood,” Biometrika, 101, 711–718.
   Web of Science ®Google Scholar
• Vexler, A., Zou, L., and Hutson, A. D. (2015), “Data-Driven Confidence Interval Estimation Incorporating Prior Information with an Adjustment for Skewed Data,” The American Statistician, 70, 243–249.
   Web of Science ®Google Scholar
• Vexler, A., Zou, L., and Hutson, A. D. (2019), “The Empirical Likelihood Prior Applied to Bias Reduction of General Estimating Equations,” Computational Statistics and Data Analysis, 138, 96–106. DOI: 10.1016/j.csda.2019.04.001.
   PubMed Web of Science ®Google Scholar
• Wang, Y., and Blei, D. M. (2019), “Frequentist Consistency of Variational Bayes,” Journal of the American Statistical Association, 114, 1147–1161.
   Web of Science ®Google Scholar
• Wu, C., Stoehr, J., and Robert, C. P. (2019), “Faster Hamiltonian Monte Carlo by Learning Leapfrog Scale,” arXiv.
   Google Scholar
• Xu, M., Quiroz, M., Kohn, R., and Sisson, S. A. (2019), “Variance Reduction Properties of the Reparameterization Trick,” in Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), (Vol. 89), 2711–2720.
   Google Scholar
• Yang, Y., and He, X. (2012), “Bayesian Empirical Likelihood for Quantile Regression,” Annals of Statistics, 40, 1102–1131.
   Web of Science ®Google Scholar
• Yiu, A., Goudie, R. J. B., and Tom, B. D. M. (2020), “Inference under Unequal Probability Sampling with the Bayesian Exponentially Tilted Empirical Likelihood,” Biometrika, 107, 857–873. DOI: 10.1093/biomet/asaa028.
   PubMed Web of Science ®Google Scholar
• Zhang, F. (2020), “Convergence Rates of Variational Posterior Distributions,” The Annals of Statistics, 48, 2180–2207.
   Web of Science ®Google Scholar
• Zhao, P., Ghosh, M., Rao, J. N. K., and Wu, C. (2020), “Bayesian Empirical Likelihood Inference with Complex Survey Data,” Journal of the Royal Statistical Society, Series B, 82, 155–174.
   Google Scholar