Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior

References

• Armagan, A. , Clyde, M. , and Dunson, D. B. (2011), “Generalized Beta Mixtures of Gaussians,” in Advances in Neural Information Processing Systems , pp. 523–531.
   PubMedGoogle Scholar
• Armagan, A. , Dunson, D. B. , and Lee, J. (2013), “Generalized Double Pareto Shrinkage,” Statistica Sinica , 23, 119.
   PubMed Web of Science ®Google Scholar
• Bai, R. , and Ghosh, M. (2019), “Large-Scale Multiple Hypothesis Testing With the Normal-Beta Prime Prior,” Statistics , 53, 1210–1233.
   Web of Science ®Google Scholar
• Bhadra, A. , Datta, J. , Polson, N. G. , and Willard, B. (2017), “The Horseshoe + Estimator of Ultra-Sparse Signals,” Bayesian Analysis , 12, 1105–1131.
   Web of Science ®Google Scholar
• Bhattacharya, A. , Chakraborty, A. , and Mallick, B. (2016), “Fast Sampling With Gaussian Scale-Mixture Priors in High-Dimensional Regression,” Biometrika , 103, 985–991. DOI: https://doi.org/10.1093/biomet/asw042.
   PubMed Web of Science ®Google Scholar
• Bhattacharya, A. , Pati, D. , Pillai, N. S. , and Dunson, D. B. (2015), “Dirichlet–Laplace Priors for Optimal Shrinkage,” Journal of the American Statistical Association , 110, 1479–1490. DOI: https://doi.org/10.1080/01621459.2014.960967.
   PubMed Web of Science ®Google Scholar
• Bingham, C. (1974), “An Antipodally Symmetric Distribution on the Sphere,” The Annals of Statistics , 2, 1201–1225.
   Web of Science ®Google Scholar
• Bondell, H. D. , and Reich, B. J. (2012), “Consistent High-Dimensional Bayesian Variable Selection via Penalized Credible Regions,” Journal of the American Statistical Association , 107, 1610–1624. DOI: https://doi.org/10.1080/01621459.2012.716344.
   PubMed Web of Science ®Google Scholar
• Brown, P. J. , Fearn, T. , and Vannucci, M. (2001), “Bayesian Wavelet Regression on Curves With Application to a Spectroscopic Calibration Problem,” Journal of the American Statistical Association , 96, 398–408.
   Web of Science ®Google Scholar
• Carbonetto, P. , and Stephens, M. (2012), “Scalable Variational Inference for Bayesian Variable Selection in Regression, and Its Accuracy in Genetic Association Studies,” Bayesian Analysis , 7, 73–107.
   Web of Science ®Google Scholar
• Carvalho, C. M. , Polson, N. G. , and Scott, J. G. (2009), “Handling Sparsity via the Horseshoe,” in International Conference on Artificial Intelligence and Statistics , pp. 73–80.
   Google Scholar
• ——— (2010), “The Horseshoe Estimator for Sparse Signals,” Biometrika , 97, 465–480.
   Web of Science ®Google Scholar
• Castillo, I. , Schmidt-Hieber, J. , and van der Vaart, A. (2015), “Bayesian Linear Regression With Sparse Priors,” The Annals of Statistics , 43, 1986–2018.
   Web of Science ®Google Scholar
• Fallaize, C. J. , and Kypraios, T. (2016), “Exact Bayesian Inference for the Bingham Distribution,” Statistics and Computing , 26, 349–360.
   Web of Science ®Google Scholar
• George, E. I. , and McCulloch, R. E. (1993), “Variable Selection via Gibbs Sampling,” Journal of the American Statistical Association , 88, 881–889.
   Web of Science ®Google Scholar
• Ghosh, J. , and Ghattas, A. E. (2015), “Bayesian Variable Selection Under Collinearity,” The American Statistician , 69, 165–173.
   Web of Science ®Google Scholar
• Griffin, J. E. , and Brown, P. J. (2010), “Inference With Normal-Gamma Prior Distributions in Regression Problems,” Bayesian Analysis , 5, 171–188.
   Web of Science ®Google Scholar
• ——— (2013), “Some Priors for Sparse Regression Modelling,” Bayesian Analysis , 8, 691–702.
   Web of Science ®Google Scholar
• Haario, H. , Saksman, E. , and Tamminen, J. (2001), “An Adaptive Metropolis Algorithm,” Bernoulli , 7, 223–242.
   Web of Science ®Google Scholar
• Hans, C. , Dobra, A. , and West, M. (2007), “Shotgun Stochastic Search for ‘Large p’ Regression,” Journal of the American Statistical Association , 102, 507–516.
   Web of Science ®Google Scholar
• Ishwaran, H. , and Rao, J. S. (2005), “Spike and Slab Variable Selection: Frequentist and Bayesian Strategies,” The Annals of Statistics , 33, 730–773.
   Web of Science ®Google Scholar
• Johndrow, J. , Orenstein, P. , and Bhattacharya, A. (2020), “Scalable Approximate MCMC Algorithms for the Horseshoe Prior,” Journal of Machine Learning Research , 21, 1–61.
   PubMed Web of Science ®Google Scholar
• Johnson, N. , Kotz, S. , and Balakrishnan, N. (1995), Continuous Univariate Distributions (Vol. 2), New York: Wiley.
   Google Scholar
• Kent, J. T. (1982), “The Fisher-Bingham Distribution on the Sphere,” Journal of the Royal Statistical Society, Series B, 44, 71–80.
   Google Scholar
• Kent, J. T. , Ganeiber, A. M. , and Mardia, K. V. (2013), “A New Method to Simulate the Bingham and Related Distributions in Directional Data Analysis With Applications,” arXiv no. 1310.8110.
   Google Scholar
• Logsdon, B. , Hoffman, G. , and Mezey, J. (2010), “A Variational Bayes Algorithm for Fast and Accurate Multiple Locus Genome-Wide Association Analysis,” BMC Bioinformatics , 11, 58. DOI: https://doi.org/10.1186/1471-2105-11-58.
   PubMed Web of Science ®Google Scholar
• Mitchell, T. J. , and Beauchamp, J. J. (1988), “Bayesian Variable Selection in Linear Regression,” Journal of the American Statistical Association , 83, 1023–1032.
   Web of Science ®Google Scholar
• Murray, I. , Ghahramani, Z. , and MacKay, D. (2006), “MCMC for Doubly-Intractable Distributions,” in Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence , AUAI Press, pp. 359–366.
   Google Scholar
• Narisetty, N. N. , and He, X. (2014), “Bayesian Variable Selection With Shrinking and Diffusing Priors,” The Annals of Statistics , 42, 789–817.
   Web of Science ®Google Scholar
• Osborne, B. G. , Fearn, T. , Miller, A. R. , and Douglas, S. (1984), “Application of Near Infrared Reflectance Spectroscopy to the Compositional Analysis of Biscuits and Biscuit Doughs,” Journal of the Science of Food and Agriculture , 35, 99–105.
   Web of Science ®Google Scholar
• Polson, N. G. , and Scott, J. G. (2010), “Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction,” Bayesian Statistics , 9, 501–538.
   Google Scholar
• ——— (2012), “Local Shrinkage Rules, Lévy Processes and Regularized Regression,” Journal of the Royal Statistical Society, Series B, 74, 287–311.
   Google Scholar
• Raskutti, G. , Wainwright, M. J. , and Yu, B. (2011), “Minimax Rates of Estimation for High-Dimensional Linear Regression Over lq -Balls,” IEEE Transactions on Information Theory , 57, 6976–6994.
   Web of Science ®Google Scholar
• Ray, K. , and Szabó, B. (2019), “Variational Bayes for High-Dimensional Linear Regression With Sparse Priors,” Unpublished manuscript, arXiv no. 1904.07150.
   Google Scholar
• Ročková, V. , and George, E. I. (2014), “EMVS: The EM Approach to Bayesian Variable Selection,” Journal of the American Statistical Association , 109, 828–846.
   Web of Science ®Google Scholar
• ——— (2018), “The Spike-and-Slab Lasso,” Journal of the American Statistical Association , 113, 431–444.
   Web of Science ®Google Scholar
• Scott, S. L. , and Varian, H. R. (2014), “Predicting the Present With Bayesian Structural Time Series,” International Journal of Mathematical Modelling and Numerical Optimisation , 5, 4–23.
   Google Scholar
• Seshadri, V. (1997), “Halphen’s Laws,” In Kotz, S. , Read, C. B. , Banks, D. L. Encyclopedia of Statistical Sciences , Update Volume 1. New York: Wiley, pp. 302–306.
   Google Scholar
• Song, Q. , and Liang, F. (2017), “Nearly Optimal Bayesian Shrinkage for High Dimensional Regression,” arXiv no. 1712.08964.
   Google Scholar
• Szakács, G. , Annereau, J.-P. , Lababidi, S. , Shankavaram, U. , Arciello, A. , Bussey, K. J. , Reinhold, W. , Guo, Y. , Kruh, G. D. , and Reimers, M. (2004), “Predicting Drug Sensitivity and Resistance: Profiling ABC Transporter Genes in Cancer Cells,” Cancer Cell , 6, 129–137. DOI: https://doi.org/10.1016/j.ccr.2004.06.026.
   PubMed Web of Science ®Google Scholar
• van der Pas, S. L. , Kleijn, B. J. , and van der Vaart, A. W. (2014), “The Horseshoe Estimator: Posterior Concentration Around Nearly Black Vectors,” Electronic Journal of Statistics , 8, 2585–2618.
   Web of Science ®Google Scholar
• van der Pas, S. , Szabó, B. , and van der Vaart, A. (2017a), “Adaptive Posterior Contraction Rates for the Horseshoe,” Electronic Journal of Statistics , 11, 3196–3225.
   Web of Science ®Google Scholar
• ——— (2017b), “Uncertainty Quantification for the Horseshoe” (with discussion), Bayesian Analysis , 12, 1221–1274.
   Web of Science ®Google Scholar
• Zellner, A. , and Siow, A. (1980), “Posterior Odds Ratios for Selected Regression Hypotheses,” Trabajos de Estadística y de Investigación Operativa , 31, 585–603.
   Google Scholar
• Zhang, Y. , and Bondell, H. D. (2018), “Variable Selection via Penalized Credible Regions With Dirichlet–Laplace Global-Local Shrinkage Priors,” Bayesian Analysis , 13, 823–844.
   Web of Science ®Google Scholar