Distributed Bayesian Inference in Linear Mixed-Effects Models

Abstract

Linear mixed-effects models play a fundamental role in statistical methodology. A variety of Markov chain Monte Carlo (MCMC) algorithms exist for fitting these models, but they are inefficient in massive data settings because every iteration of any such MCMC algorithm passes through the full data. Many divide-and-conquer methods have been proposed to solve this problem, but they lack theoretical guarantees, impose restrictive assumptions, or have complex computational algorithms. Our focus is one such method called the Wasserstein Posterior (WASP), which has become popular due to its optimal theoretical properties under general assumptions. Unfortunately, practical implementation of the WASP either requires solving a complex linear program or is limited to one-dimensional parameters. The former method is inefficient and the latter method fails to capture the joint posterior dependence structure of multivariate parameters. We develop a new algorithm for computing the WASP of multivariate parameters that is easy to implement and is useful for computing the WASP in any model where the posterior distribution of parameter belongs to a location-scatter family of probability measures. The algorithm is introduced for linear mixed-effects models with both implementation details and theoretical properties. Our algorithm outperforms the current state-of-the-art method in inference on the functions of the covariance matrix of the random effects across diverse numerical comparisons. Supplemental materials for this article are available online.

Keywords:

• Data augmentation
• Divide-and-conquer
• Location-scatter family
• Monte Carlo error
• Wasserstein distance
• Wasserstein barycenter

Supplementary Materials

Supplementary materials are available online and contain proofs of the theoretical results and additional experimental results. The code used in the experiments is available at https://github.com/blayes/LSWASP.

Division of Mathematical Sciences;

Acknowledgments

We thank the editor, associate editor, and two anonymous referees for their helpful comments and constructive criticisms. Yixiang Xu started working on this project when he was a graduate student at The University of Iowa. Cheng Li, Terrance Savitsky, N. D. Shyamalkumar, and Aixin Tan gave invaluable feedback on an initial draft of the article.

Additional information

Funding

Sanvesh Srivastava’s research is partially supported by grants from the Office of Naval Research (ONR-BAA N000141812741) and the National Science Foundation (DMS-1854667/1854662).