Privacy-Preserving Parametric Inference: A Case for Robust Statistics

References

• Avella-Medina, M. (2017), “Influence Functions for Penalized M-Estimators,” Bernoulli , 23, 3178–3196.
   Web of Science ®Google Scholar
• Avella-Medina, M. , and Brunel, V. E. (2019), “Differentially Private Sub-Gaussian Location Estimators,” arXiv no. 1906.11923.
   Google Scholar
• Awan, J. , and Slavkovic, A. (2018), “Differentially Private Uniformly Most Powerful Tests for Binomial Data,” in Advances in Neural Information Processing Systems , pp. 4208–4218.
   Google Scholar
• Awan, J. , and Slavkovic, A. (2019), “Differentially Private Inference for Binomial Data,” arXiv no. 1904.00459.
   Google Scholar
• Barrientos, A. F. , Reiter, J. P. , Machanavajjhala, A. , and Chen, Y. (2019), “Differentially Private Significance Tests for Regression Coefficients,” Journal of Computational and Graphical Statistics , 28, 440–453.
   Web of Science ®Google Scholar
• Bassily, R. , Smith, A. , and Thakurta, A. (2014), “Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds,” in IEEE 55th Annual Symposium on Foundations of Computer Science , pp. 464–473.
   Google Scholar
• Belsley, D. A. , Kuh, E. , and Welsch, R. E. (2005), Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , New York: Wiley.
   Google Scholar
• Cai, T. T. , Wang, Y. , and Zhang, L. (2019), “The Cost of Differential Privacy: Optimal Rates of Convergence for Parameter Estimation With Differential Privacy,” manuscript.
   Google Scholar
• Canonne, C. L. , Kamath, G. , McMillan, A. , Smith, A. , and Ullman, J. (2019), “The Structure of Optimal Private Tests for Simple Hypotheses,” in Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pp. 310–321. DOI: https://doi.org/10.1145/3313276.3316336.[]
   Google Scholar
• Canonne, C. L. , Kamath, G. , McMillan, A. , Ullman, J. , and Zakynthinou, L. (2019), “Private Identity Testing for High-Dimensional Distributions,” arXiv no. 1905.11947.
   Google Scholar
• Cantoni, E. , and Ronchetti, E. (2001), “Robust Inference for Generalized Linear Models,” Journal of the American Statistical Association , 96, 1022–1030.
   Web of Science ®Google Scholar
• Catoni, O. (2012), “Challenging the Empirical Mean and Empirical Variance: A Deviation Study,” Annales de l’Institut Henri Poincaré , 48, 1148–1185.
   Web of Science ®Google Scholar
• Chaudhuri, K. , and Monteleoni, C. (2008), “Privacy-Preserving Logistic Regression,” in Advances in Neural Information Processing Systems , pp. 289–296.
   Google Scholar
• Chaudhuri, K. , Monteleoni, C. , and Sarwate, A. D. (2011), “Differentially Private Empirical Risk Minimization,” Journal of Machine Learning Research , 12, 1069–1109.
   PubMed Web of Science ®Google Scholar
• Chaudhuri, K. , and Hsu, D. (2012), “Convergence Rates for Differentially Private Statistical Estimation,” in International Conference on Machine Learning 2012 , pp. 155–186.
   Google Scholar
• Chen, Y. , Barrientos, A. F. , Machanavajjhala, A. , and Reiter, J. P. (2018), “Is My Model Any Good: Differentially Private Regression Diagnostics,” Knowledge and Information Systems , 54, 33–64.
   Web of Science ®Google Scholar
• Christmann, A. , and Steinwart, I. (2007), “Consistency and Robustness of Kernel-Based Regression in Convex Risk Minimization,” Bernoulli , 17, 799–819.
   Web of Science ®Google Scholar
• Clarke, B. R. (1986), “Nonsmooth Analysis and Fréchet Differentiability of M-Functionals,” Probability Theory and Related Fields , 73, 197–819.
   Google Scholar
• Dimitrakakis, C. , Nelson, B. , Mitrokotsa, A. , and Rubinstein, B. I. (2014), “Robust and Private Bayesian Inference,” in International Conference on Algorithmic Learning Theory , pp. 291–305.
   Google Scholar
• Dimitrakakis, C. , Nelson, B. , Mitrokotsa, A. , Zhang, Z. and Rubinstein, B. I. (2014), “Differential Privacy for Bayesian Inference Through Posterior Sampling,” Journal of Machine Learning Research , 18, 343–382.
   Google Scholar
• Duchi, J. C. , Jordan, M. I. , and Wainwright, M. J. (2018), “Minimax Optimal Procedures for Locally Private Estimation,” Journal of the American Statistical Association , 113, 182–215.
   Web of Science ®Google Scholar
• Dwork, C. , McSherry, F. , Nissim, K. , and Smith, A. (2006), “Calibrating Noise to Sensitivity in Private Data Analysis,” in Theory of Cryptography Conference , pp. 265–284.
   Google Scholar
• Dwork, C. , and Lei, J. (2009), “Differential Privacy and Robust Statistics,” in Proceedings of the 41st Annual ACM Symposium on Theory of Computing 2009, pp. 371–380. DOI: https://doi.org/10.1145/1536414.1536466.[]
   Google Scholar
• Dwork, C. , and Roth, A. (2014), “The Algorithmic Foundations of Differential Privacy,” Foundations and Trends [ textregistered] in Theoretical Computer Science , 9, 211–407.
   Google Scholar
• Foulds, J. , Geumlek, J. , Welling, M. , and Chaudhuri, K. (2016), “Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testing,” in Proceedings of the 32nd Conference on Uncertainty and Artificial Intelligence 2016.
   Google Scholar
• Gaboardi, M. , Lim, H. W. , Rogers, R. M. , and Vadhan, S. P. (2016), “Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testing,” in International Conference on Machine Learning 2016, pp. 2111–2120.
   Google Scholar
• Hampel, F. R. (1974), “The Influence Curve and Its Role in Robust Estimation,” Journal of the American Statistical Association , 69, 383–393.
   Web of Science ®Google Scholar
• Hampel, F. R. , Ronchetti, E. M. , Rousseeuw, P. J. , and Stahel, W. A. (1986), Robust Statistics: The Approach Based on Influence Functions , New York: Wiley.
   Google Scholar
• Hampel, F. R. , Rousseeuw, P. J. , and Ronchetti, E. (1981), “The Change-of-Variance Curve and Optimal Redescending M-Estimators,” Journal of the American Statistical Association , 69, 643–648.
   Google Scholar
• He, X. , and Shao, Q. M. (2000), “On Parameters of Increasing Dimension,” Journal of Multivariate Analysis , 73, 120–135.
   Web of Science ®Google Scholar
• He, X. , and Simpson, D. G. (1992), “Robust Direction Estimation,” Annals of Statistics , 20, 351–369.
   Web of Science ®Google Scholar
• Heritier, S. , and Ronchetti, E. (1994), “Robust Bounded-Influence Tests in General Parametric Models,” Journal of the American Statistical Association , 89, 897–904.
   Web of Science ®Google Scholar
• Huber, P. (1964), “Robust Estimation of a Location Parameter,” The Annals of Mathematical Statistics , 35, 73–101.
   Google Scholar
• Huber, P. (1967), “The Behavior of Maximum Likelihood Estimates Under Nonstandard Conditions,” in Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability (Vol. 1), pp. 163–168.
   Google Scholar
• Huber, P. (1981), Robust Statistics , New York: Wiley.
   Google Scholar
• Huber, P. , and Ronchetti, E. (2009), Robust Statistics (2nd ed.), New York: Wiley.
   Google Scholar
• Kiefer, D. , Smith, A. , and Thakurta, A. (2012), “Private Convex Empirical Risk Minimization and High-Dimensional Regression,” in Conference on Learning Theory (Vol. 25), pp. 1–25.
   Google Scholar
• La Vecchia, D. , Ronchetti, E. , and Trojani, F. (2012), “Higher-Order Infinitesimal Robustness,” Journal of the American Statistical Association , 107, 1546–1557.
   Web of Science ®Google Scholar
• Lehmann, E. L. , and Romano, J. P. (2005), Testing Statistical Hypothesis (3rd ed.), New York: Springer.
   Google Scholar
• Lei, J. (2011), “Differentially Private M-Estimators,” in Advances in Neural Information Processing Systems , pp. 361–369.
   Google Scholar
• Liu, F. (2016), “Model-Based Differentially Private Data Synthesis,” arXiv no. 1606.08052.
   Google Scholar
• Maronna, R. , Martin, R. , and Yohai, V. (2006), Robust Statistics: Theory and Methods , New York: Wiley.
   Google Scholar
• McCullagh, P. , and Nelder, J. A. (1989), Generalized Linear Models (2nd ed.), London: Chapman & HAll/CRC.
   Google Scholar
• McSherry, F. , and Talwar, K. (2007), “Mechanism Design via Differential Privacy,” in IEEE Symposium on Foundations of Computer Science 2007.
   Google Scholar
• Narayanan, A. , and Shmatikov, V. (2008), “Robust De-anonymization of Large Sparse Datasets,” in IEEE Symposium on Security and Privacy 2008, pp. 111–125.
   Google Scholar
• Nissim, K , Rashkodnikova, S. , and Smith, A. (2007), “Smooth Sensitivity and Sampling in Private Data Analysis,” in Proceedings of the 39th Annual ACM Symposium on Theory of Computing 2007, pp. 75–84.
   Google Scholar
• Rajkumar, A. , and Argawal, S. (2012), “A Differentially Private Stochastic Gradient Descent Algorithm for Multiparty Classification,” in International Conference on Artificial Intelligence and Statistics , pp. 933–941.
   Google Scholar
• Reiter, J. (2002), “Satisfying Disclosure Restrictions With Synthetic Data Sets,” Journal of Official Statistics , 18, 531–544.
   Google Scholar
• Reiter, J. (2005), “Releasing Multiply-Imputed, Synthetic Public Use Microdata: An Illustration and Empirical Study,” Journal of Royal Statistical Society, Series A, 168, 185 –205.
   Google Scholar
• Ronchetti, E. , and Trojani, F. (2001), “Robust Inference With GMM Estimators,” Journal of Econometrics , 101, 37–69.
   Web of Science ®Google Scholar
• Rubin, D. B. (1993), “Statistical Disclosure Limitation,” Journal of Official Statistics , 9, 461–468.
   Google Scholar
• Salibian-Barrera, M. , Van Aelst, S. , and Yohai, V. J. (2016), “Robust Tests for Linear Regression Models Based on τ-Estimates,” Computational Statistics and Data Analysis , 93, 436–455.
   Web of Science ®Google Scholar
• Sheffet, O. (2017), “Differentially Private Ordinary Least Squares,” in International Conference on Machine Learning , pp. 3105–3114.
   Google Scholar
• Sheffet, O. (2018), “Locally Private Hypothesis Testing,” in International Conference on Machine Learning , pp. 4605–4614.
   Google Scholar
• Smith, A. (2008), “Efficient, Differentially Private Point Estimators,” arXiv no. 0809.4794.
   Google Scholar
• Smith, A. (2011), “Privacy-Preserving Statistical Estimation With Optimal Convergence Rates,” in Proceedings of the 43rd Annual ACM Symposium on Theory of Computing , pp. 813–822.
   Google Scholar
• Sweeney, L. (1997), “Weaving Technology and Policy Together to Maintain Confidentiality,” The Journal of Law, Medicine & Ethics , 25, 98–110.
   PubMed Web of Science ®Google Scholar
• Uhler, C. , Slavkovic, A. , and Fienberg, S. (2013), “Privacy-Preserving Data Sharing for Genome-Wide Association Studies,” Journal of Privacy and Confidentiality , 5, 137–166.
   Google Scholar
• von Mises, R. (1947), “On the Asymptotic Distribution of Differentiable Statistical Functionals,” Annals of Mathematical Statistics , 18, 309–348.
   Google Scholar
• Wang, Y. , Lee, J. , and Kifer, D. (2015), “Revisiting Differentially Private Hypothesis Tests for Categorical Data,” arXiv no. 1511.03376v4.
   Google Scholar
• Wang, Y. X. , Fienberg, S. , and Smola, A. (2015), “Privacy for Free: Posterior Sampling and Stochastic Gradient Descent Monte Carlo,” in International Conference on Machine Learning , pp. 2493–2502.
   Google Scholar
• Wasserman, L. , and Zhou, S. (2010), “A Statistical Framework for Differential Privacy,” Journal of the American Statistical Association , 105, 375–389.
   Web of Science ®Google Scholar
• Wedderburn, R. (1974), “Quasi-Likelihood Functions, Generalized Linear Models, and the Gauss-Newton Method,” Biometrika , 61, 439–447.
   Web of Science ®Google Scholar