Bayesian Robustness: A Nonasymptotic Viewpoint

References

• Berger, J. O., Moreno, E., Pericchi, L. R., Jesús Bayarri, M., Bernardo, J. M., Cano, J. A., et al. (1994), “An Overview of Robust Bayesian Analysis,” Test, 3, 5–124.
   Google Scholar
• Bhatia, K., Jain, P., Kamalaruban, P., and Kar, P. (2017), “Consistent Robust Regression,” in Advances in Neural Information Processing Systems, pp. 2110–2119.
   Google Scholar
• Bhatia, K., Jain, P., and Kar, P. (2015), “Robust Regression via Hard Thresholding,” in Advances in Neural Information Processing Systems, pp. 721–729.
   Google Scholar
• Box, G. E. (1953), “Non-Normality and Tests on Variances,” Biometrika, 40, 318–335.
   Web of Science ®Google Scholar
• Cheng, X., and Bartlett, P. (2018), “Convergence of Langevin MCMC in KL-Divergence,” in Proceedings of Algorithmic Learning Theory (Vol. 83), pp. 186–211, PMLR.
   Google Scholar
• Dalalyan, A. S. (2017), “Theoretical Guarantees for Approximate Sampling from Smooth and Log-Concave Densities,” Journal of the Royal Statistical Society, Series B, 79, 651–676.
   Google Scholar
• De Finetti, B. (1961), “The Bayesian Approach to the Rejection of Outliers,” in Proceedings of the Fourth Berkeley Symposium on Probability and Statistics (Vol. 1), pp. 199–210. University of California Press Berkeley.
   Google Scholar
• Diakonikolas, I., Kamath, G., Kane, D., Li, J., Steinhardt, J., and Stewart, A. (2019), “Sever: A Robust Meta-Algorithm for Stochastic Optimization,” in Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 1596–1606, PMLR.
   Google Scholar
• Diakonikolas, I., Kamath, G., Kane, D. M., Li, J., Moitra, A., and Stewart, A. (2016), “Robust Estimators in High Dimensions Without the Computational Intractability,” in IEEE 57th Annual Symposium on Foundations of Computer Science, pp. 655–664, IEEE.
   Google Scholar
• Durmus, A., and Moulines, E. (2017), “Nonasymptotic Convergence Analysis for the Unadjusted Langevin Algorithm,” Annals of Applied Probability, 27, 1551–1587.
   Web of Science ®Google Scholar
• Dwivedi, R., Chen, Y., Wainwright, M. J., and Yu, B. (2018), “Log-Concave Sampling: Metropolis-Hastings Algorithms are Fast!” arXiv preprint arXiv:1801.02309.
   Google Scholar
• Ermak, D. L. (1975), “A Computer Simulation of Charged Particles in Solution. I. Technique and Equilibrium Properties,” Journal of Chemical Physics, 62, 4189–4196.
   Web of Science ®Google Scholar
• Huber, P. J. (1964), “Robust Estimation of a Location Parameter,” The Annals of Mathematical Statistics, 35, 73–101.
   Google Scholar
• Huber, P. J. (1973), “Robust Regression: Asymptotics, Conjectures and Monte Carlo,” The Annals of Statistics, 1, 799–821.
   Web of Science ®Google Scholar
• Huber, P. J. (1973), “The Use of Choquet Capacities in Statistics,” Bulletin of the International Statistical Institute, 45, 181–191.
   Google Scholar
• Huber, P. J. (2011), “Robust Statistics,” in International Encyclopedia of Statistical Science, ed. M. Lovric, pp. 1248–1251, Berlin: Springer.
   Google Scholar
• Klivans, A., Kothari, P. K., and Meka, R. (2018), “Efficient Algorithms for Outlier-Robust Regression,” arXiv preprint arXiv:1803.03241.
   Google Scholar
• Kong, C. T. A. (1987), Multivariate Belief Functions and Graphical Models, Cambridge, MA: Harvard University.
   Google Scholar
• Lai, K. A., Rao, A. B., and Vempala, S. (2016), “Agnostic Estimation of Mean and Covariance,” in IEEE 57th Foundations of Computer Science, pp. 665–674, IEEE.
   Google Scholar
• Lovász, L., and Simonovits, M. (1993), “Random Walks in a Convex Body and an Improved Volume Algorithm,” Random structures and Algorithms, 4, 359–412.
   Web of Science ®Google Scholar
• Ma, Y.-A., Chen, Y., Jin, C., Flammarion, N., and Jordan, M. I. (2018), “Sampling can be Faster than Optimization,” arXiv:1811.08413, 2018.
   Google Scholar
• Mengersen, K. L., and Tweedie, R. L. (1996), “Rates of Convergence of the Hastings and Metropolis Algorithms,” The Annals of Statistics, 24, 101–121.
   Web of Science ®Google Scholar
• Miller, J. W., and Dunson, D. B. (2018), “Robust Bayesian Inference via Coarsening,” Journal of the American Statistical Association, 114, 1113–1125. DOI: 10.1080/01621459.2018.1469995.
   PubMed Web of Science ®Google Scholar
• Minsker, S., Srivastava, S., Lin, L., and Dunson, D. B. (2017), “Robust and Scalable Bayes via a Median of Subset Posterior Measures,” The Journal of Machine Learning Research, 18, 4488–4527.
   Web of Science ®Google Scholar
• Neal, R. M. (2011), “MCMC using Hamiltonian Dynamics,” in Handbook of Markov Chain Monte Carlo, eds. S. Brooks, A. Gelman, G. Jones, and X.-L. Meng, pp. 113–162, New York: Chapman and Hall/CRC.
   Google Scholar
• Oksendal, B. (2003), Stochastic Differential Equations (6th ed.), Berlin: Springer.
   Google Scholar
• Otto, F., and Villani, C. (2000), “Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality,” Journal of Functional Analysis, 173, 361–400.
   Web of Science ®Google Scholar
• Prasad, A., Suggala, A. S., Balakrishnan, S., and Ravikumar, P. (2018), “Robust Estimation via Robust Gradient Estimation,” arXiv preprint arXiv:1802.06485.
   Google Scholar
• Roberts, G., and Rosenthal, J. S. (1998), “Optimal Scaling of Discrete Approximations to Langevin Diffusions,” Journal of the Royal Statistical Society, Series B, 60, 255–268.
   Google Scholar
• Roberts, G., and Tweedie, R. (1996), “Exponential Convergence of Langevin Distributions and their Discrete Approximations,” Bernoulli, 2, 341–363.
   Google Scholar
• Schilling, R. L., and Partzsch, L. (2014), Brownian Motion: An Introduction to Stochastic Processes (2nd ed.), Berlin: De Gruyter.
   Google Scholar
• Song, Q., Sun, Y., Ye, M., and Liang, F. (2020), “Extended Stochastic Gradient Markov Chain Monte Carlo for Large-Scale Bayesian Variable Selection,” Biometrika, 107, 997–1004.
   PubMed Web of Science ®Google Scholar
• Suggala, A. S., Bhatia, K., Ravikumar, P., and Jain, P. (2019), “Adaptive Hard Thresholding for Near-Optimal Consistent Robust Regression,” arXiv preprint arXiv:1903.08192, 2019.
   Google Scholar
• Tukey, J. W. (1960), “A Survey of Sampling from Contaminated Distributions,” in Contributions to Probability and Statistics, ed. I. Oklin, pp. 448–485, Redwood City, CA: Stanford University Press.
   Google Scholar
• Wang, C., and Blei, D. M. (2018), “A General Method for Robust Bayesian Modeling,” Bayesian Analysis, 13, 1163–1191.
   Web of Science ®Google Scholar
• Wang, Y., Kucukelbir, A., and Blei, D. M. (2017), “Robust Probabilistic Modeling with Bayesian Data Reweighting,” in Proceedings of the 34th International Conference on Machine Learning (Vol. 70), pp. 3646–3655, PMLR.
   Google Scholar
• Welling, M., and Teh, Y. W. (2011), “Bayesian Learning via Stochastic Gradient Langevin Dynamics,” in Proceedings of the 28th International Conference on Machine Learning, pp. 681–688, Omnipress.
   Google Scholar