Distributed Bayesian Inference in Linear Mixed-Effects Models

References

• Abrevaya, J. (2006), “Estimating the Effect of Smoking on Birth Outcomes Using a Matched Panel Data Approach,” Journal of Applied Econometrics, 21, 489–519. DOI: https://doi.org/10.1002/jae.851.
   Web of Science ®Google Scholar
• Agueh, M., and Carlier, G. (2011), “Barycenters in the Wasserstein Space,” SIAM Journal on Mathematical Analysis, 43, 904–924. DOI: https://doi.org/10.1137/100805741.
   Web of Science ®Google Scholar
• Ahn, S., Korattikara, A., and Welling, M. (2012), “Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring,” in Proceedings of the 29th International Conference on Machine Learning.
   Google Scholar
• Alquier, P., Friel, N., Everitt, R., and Boland, A. (2016), “Noisy Monte Carlo: Convergence of Markov Chains With Approximate Transition Kernels,” Statistics and Computing, 26, 29–47. DOI: https://doi.org/10.1007/s11222-014-9521-x.
   Web of Science ®Google Scholar
• Álvarez-Esteban, P. C., del Barrio, E., Cuesta-Albertos, J., and Matrán, C. (2016), “A Fixed-Point Approach to Barycenters in Wasserstein Space,” Journal of Mathematical Analysis and Applications, 441, 744–762. DOI: https://doi.org/10.1016/j.jmaa.2016.04.045.
   Web of Science ®Google Scholar
• Arjovsky, M., Chintala, S., and Bottou, L. (2017), “Wasserstein Gan,” arXiv no. 1701.07875.
   Google Scholar
• Bardenet, R., Doucet, A., and Holmes, C. (2017), “On Markov Chain Monte Carlo Methods for Tall Data,” The Journal of Machine Learning Research, 18, 1515–1557.
   Web of Science ®Google Scholar
• Bernton, E., Jacob, P. E., Gerber, M., and Robert, C. P. (2017), “Inference in Generative Models Using the Wasserstein Distance,” arXiv no. 1701.05146.
   Google Scholar
• Bhatia, R. (2009), Positive Definite Matrices, Princeton, NJ: Princeton University Press.
   Google Scholar
• Campbell, T., and Broderick, T. (2018), “Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent,” arXiv no. 1802.01737.
   Google Scholar
• Castillo, I., Schmidt-Hieber, J., and Van der Vaart, A. (2015), “Bayesian Linear Regression With Sparse Priors,” The Annals of Statistics, 43, 1986–2018. DOI: https://doi.org/10.1214/15-AOS1334.
   Web of Science ®Google Scholar
• Claici, S., and Solomon, J. (2018), “Wasserstein Coresets for Lipschitz Costs,” arXiv no. 1805.07412.
   Google Scholar
• Dicker, L. H., and Erdogdu, M. A. (2017), “Flexible Results for Quadratic Forms With Applications to Variance Components Estimation,” The Annals of Statistics, 45, 386–414. DOI: https://doi.org/10.1214/16-AOS1456.
   Web of Science ®Google Scholar
• Fournier, N., and Guillin, A. (2015), “On the Rate of Convergence in Wasserstein Distance of the Empirical Measure,” Probability Theory and Related Fields, 162, 707–738. DOI: https://doi.org/10.1007/s00440-014-0583-7.
   Web of Science ®Google Scholar
• Gao, K., and Owen, A. (2017), “Efficient Moment Calculations for Variance Components in Large Unbalanced Crossed Random Effects Models,” Electronic Journal of Statistics, 11, 1235–1296. DOI: https://doi.org/10.1214/17-EJS1236.
   Web of Science ®Google Scholar
• Gelman, A., and Hill, J. (2007), Data Analysis Using Regression and Multilevel/Hierarchical Models (Vol. 3), New York: Cambridge University Press.
   Google Scholar
• Gelman, A., Vehtari, A., Jylänki, P., Robert, C., Chopin, N., and Cunningham, J. P. (2014), “Expectation Propagation as a Way of Life,” arXiv no. 1412.4869.
   Google Scholar
• Giordano, R., Broderick, T., and Jordan, M. I. (2017), “Covariances, Robustness, and Variational Bayes,” arXiv no. 1709.02536.
   Google Scholar
• Guhaniyogi, R., Li, C., Savitsky, T. D., and Srivastava, S. (2017), “Distributed Kriging for Massive Data,” arXiv preprint.
   Google Scholar
• Jiang, J., Li, C., Paul, D., Yang, C., and Zhao, H. (2016), “On High-Dimensional Misspecified Mixed Model Analysis in Genome-Wide Association Study,” The Annals of Statistics, 44, 2127–2160. DOI: https://doi.org/10.1214/15-AOS1421.
   Web of Science ®Google Scholar
• Johndrow, J. E., Mattingly, J. C., Mukherjee, S., and Dunson, D. B. (2015), “Approximations of Markov Chains and High-Dimensional Bayesian Inference,” arXiv no. 1508.03387v1.
   Google Scholar
• Johndrow, J. E., Smith, A., Pillai, N., and Dunson, D. B. (2019), “MCMC for Imbalanced Categorical Data,” Journal of the American Statistical Association, 114, 1394–1403. DOI: https://doi.org/10.1080/01621459.2018.1505626.
   Web of Science ®Google Scholar
• Korattikara, A., Chen, Y., and Welling, M. (2014), “Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget,” in Proceedings of the 31st International Conference on Machine Learning, pp. 181–189.
   Google Scholar
• Kucukelbir, A., Ranganath, R., Gelman, A., and Blei, D. (2015), “Automatic Variational Inference in Stan,” in Advances in Neural Information Processing Systems, pp. 568–576.
   Google Scholar
• Lan, S., Zhou, B., and Shahbaba, B. (2014), “Spherical Hamiltonian Monte Carlo for Constrained Target Distributions,” in JMLR Workshop and Conference Proceedings (Vol. 32), NIH Public Access, p. 629.
   Google Scholar
• Lee, C. Y. Y., and Wand, M. P. (2016), “Streamlined Mean Field Variational Bayes for Longitudinal and Multilevel Data Analysis,” Biometrical Journal, 58, 868–895. DOI: https://doi.org/10.1002/bimj.201500007.
   PubMed Web of Science ®Google Scholar
• Li, C., Srivastava, S., and Dunson, D. B. (2017), “Simple, Scalable and Accurate Posterior Interval Estimation,” Biometrika, 104, 665–680. DOI: https://doi.org/10.1093/biomet/asx033.
   Web of Science ®Google Scholar
• Li, J., and Zhang, F. (2018), “Geometry-Sensitive Ensemble Mean Based on Wasserstein Barycenters: Proof-of-Concept on Cloud Simulations,” Journal of Computational and Graphical Statistics, 27, 785–797. DOI: https://doi.org/10.1080/10618600.2018.1448831.
   Web of Science ®Google Scholar
• Liu, J. S. (1994), “The Fraction of Missing Information and Convergence Rate for Data Augmentation,” Computing Science and Statistics, 26, 490–497.
   Google Scholar
• Maclaurin, D., and Adams, R. P. (2015), “Firefly Monte Carlo: Exact MCMC With Subsets of Data,” in Twenty-Fourth International Joint Conference on Artificial Intelligence.
   Google Scholar
• Meng, X.-L., and Van Dyk, D. A. (1999), “Seeking Efficient Data Augmentation Schemes via Conditional and Marginal Augmentation,” Biometrika, 86, 301–320. DOI: https://doi.org/10.1093/biomet/86.2.301.
   Web of Science ®Google Scholar
• Minsker, S., Srivastava, S., Lin, L., and Dunson, D. B. (2014), “Scalable and Robust Bayesian Inference via the Median Posterior,” in Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1656–1664.
   Google Scholar
• Minsker, S., Srivastava, S., Lin, L., and Dunson, D. B. (2017), “Robust and Scalable Bayes via a Median of Subset Posterior Measures,” Journal of Machine Learning Research, 18, 4488–4527.
   Web of Science ®Google Scholar
• Neiswanger, W., Wang, C., and Xing, E. (2014), “Asymptotically Exact, Embarrassingly Parallel MCMC,” in Proceedings of the 30th International Conference on Uncertainty in Artificial Intelligence, pp. 623–632.
   Google Scholar
• Park, T., and Van Dyk, D. A. (2009), “Partially Collapsed Gibbs Samplers: Illustrations and Applications,” Journal of Computational and Graphical Statistics, 18, 283–305. DOI: https://doi.org/10.1198/jcgs.2009.08108.
   Web of Science ®Google Scholar
• Perry, P. O. (2017), “Fast Moment-Based Estimation for Hierarchical Models,” Journal of the Royal Statistical Society, Series B, 79, 267–291. DOI: https://doi.org/10.1111/rssb.12165.
   Google Scholar
• Quiroz, M., Villani, M., Kohn, R., Tran, M.-N., and Dang, K.-D. (2018), “Subsampling MCMC—A Review for the Survey Statistician,” arXiv no. 1807.08409.
   Google Scholar
• Ranganath, R., Tran, D., and Blei, D. (2016), “Hierarchical Variational Models,” in International Conference on Machine Learning, pp. 324–333.
   Google Scholar
• Robert, C. P., and Casella, G. (2004), Monte Carlo Statistical Methods, New York: Springer-Verlag.
   Google Scholar
• Román, J. C., and Hobert, J. P. (2015), “Geometric Ergodicity of Gibbs Samplers for Bayesian General Linear Mixed Models With Proper Priors,” Linear Algebra and Its Applications, 473, 54–77. DOI: https://doi.org/10.1016/j.laa.2013.12.013.
   Web of Science ®Google Scholar
• Rue, H., Martino, S., and Chopin, N. (2009), “Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace approximations,” Journal of the Royal Statistical Society, Series B, 71, 319–392. DOI: https://doi.org/10.1111/j.1467-9868.2008.00700.x.
   Web of Science ®Google Scholar
• Savitsky, T. D., and Srivastava, S. (2018), “Scalable Bayes Under Informative Sampling,” Scandinavian Journal of Statistics, 45, 534–556. DOI: https://doi.org/10.1111/sjos.12312.
   Web of Science ®Google Scholar
• Scott, S. L., Blocker, A. W., Bonassi, F. V., Chipman, H. A., George, E. I., and McCulloch, R. E. (2016), “Bayes and Big Data: The Consensus Monte Carlo Algorithm,” International Journal of Management Science and Engineering Management, 11, 78–88. DOI: https://doi.org/10.1080/17509653.2016.1142191.
   Web of Science ®Google Scholar
• Shahbaba, B., Lan, S., Johnson, W. O., and Neal, R. M. (2014), “Split Hamiltonian Monte Carlo,” Statistics and Computing, 24, 339–349. DOI: https://doi.org/10.1007/s11222-012-9373-1.
   Web of Science ®Google Scholar
• Skovgaard, L. T. (1984), “A Riemannian Geometry of the Multivariate Normal Model,” Scandinavian Journal of Statistics, 11, 211–223.
   Web of Science ®Google Scholar
• Srivastava, S., Cevher, V., Dinh, Q., and Dunson, D. (2015), “WASP: Scalable Bayes via Barycenters of Subset Posteriors,” in Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, pp. 912–920.
   Google Scholar
• Srivastava, S., Li, C., and Dunson, D. B. (2018), “Scalable Bayes via Barycenter in Wasserstein Space,” Journal of Machine Learning Research, 19, 312–346.
   Web of Science ®Google Scholar
• Staib, M., Claici, S., Solomon, J., and Jegelka, S. (2017), “Parallel Streaming Wasserstein Barycenters,” arXiv no. 1705.07443.
   Google Scholar
• Stan Development Team (2017), “Stan: A C++ Library for Probability and Sampling, Version 2.5.0.”
   Google Scholar
• Tan, L. S., and Nott, D. J. (2014), “A Stochastic Variational Framework for Fitting and Diagnosing Generalized Linear Mixed Models,” Bayesian Analysis, 9, 963–1004. DOI: https://doi.org/10.1214/14-BA885.
   Web of Science ®Google Scholar
• van Dyk, D. A. (2000), “Fitting Mixed-Effects Models Using Efficient EM-Type Algorithms,” Journal of Computational and Graphical Statistics, 9, 78–98.
   Web of Science ®Google Scholar
• van Dyk, D. A., and Meng, X.-L. (2001), “The Art of Data Augmentation,” Journal of Computational and Graphical Statistics, 10, 1–50. DOI: https://doi.org/10.1198/10618600152418584.
   Web of Science ®Google Scholar
• van Dyk, D. A., and Park, T. (2008), “Partially Collapsed Gibbs Samplers: Theory and Methods,” Journal of the American Statistical Association, 103, 790–796. DOI: https://doi.org/10.1198/016214508000000409.
   Web of Science ®Google Scholar
• von Brzeski, V., Taddy, M., and Draper, D. (2015), “Causal Inference in Repeated Observational Studies: A Case Study of Ebay Product Releases,” arXiv no. 1509.03940.
   Google Scholar
• Wand, M. P. (2017), “Fast Approximate Inference for Arbitrarily Large Semiparametric Regression Models via Message Passing,” Journal of the American Statistical Association, 112, 137–168. DOI: https://doi.org/10.1080/01621459.2016.1197833.
   Web of Science ®Google Scholar
• Wang, X., and Dunson, D. B. (2013), “Parallel MCMC via Weierstrass Sampler,” arXiv no. 1312.4605.
   Google Scholar
• Wang, X., Guo, F., Heller, K. A., and Dunson, D. B. (2015), “Parallelizing MCMC With Random Partition Trees,” in Advances in Neural Information Processing Systems, pp. 451–459.
   Google Scholar
• Welling, M., and Teh, Y. W. (2011), “Bayesian Learning via Stochastic Gradient Langevin Dynamics,” in Proceedings of the 28th International Conference on Machine Learning, pp. 681–688.
   Google Scholar
• Xue, J., and Liang, F. (2019), “Double-Parallel Monte Carlo for Bayesian Analysis of Big Data,” Statistics and Computing, 29, 23–32. DOI: https://doi.org/10.1007/s11222-017-9791-1.
   PubMed Web of Science ®Google Scholar