Abstract
We study the problem of robustly estimating the posterior distribution for the setting where observed data can be contaminated with potentially adversarial outliers. We propose Rob-ULA, a robust variant of the Unadjusted Langevin Algorithm (ULA), and provide a finite-sample analysis of its sampling distribution. In particular, we show that after iterations, we can sample from pT such that , where ϵ is the fraction of corruptions and represents the squared 2-Wasserstein distance metric. Our results for the class of posteriors which satisfy log-concavity and smoothness assumptions. We corroborate our theoretical analysis with experiments on both synthetic and real-world datasets for mean estimation, regression and binary classification. Supplementary materials for this article are available online.
Supplementary Materials
The supplementary material for this paper contains the proofs of the main theorems along with additional experiment details.
Acknowledgments
We would like to thank members of SAIL, Stat-learning and InterACT labs at Berkeley for helpful discussions.
Disclosure Statement
The authors report there are no competing interests to declare.
Notes
1 For a pair of distributions p, q with smooth and continuous densities, the log-Sobolev inequality implies that where is the relative Fisher information of p with respect to q.