Gaining Outlier Resistance With Progressive Quantiles: Fast Algorithms and Theoretical Studies

References

• Alfons, A., Croux, C., and Gelper, S. (2013), “Sparse Least Trimmed Squares Regression for Analyzing High-Dimensional Large Data Sets,” The Annals of Applied Statistics, 7, 226–248. DOI: 10.1214/12-AOAS575.
   Web of Science ®Google Scholar
• Avella-Medina, M. (2017), “Influence Functions for Penalized M-Estimators,” Bernoulli, 23, 3178–3196. DOI: 10.3150/16-BEJ841.
   Web of Science ®Google Scholar
• Avella-Medina, M., and Ronchetti, E. (2018), “Robust and Consistent Variable Selection in High-Dimensional Generalized Linear Models,” Biometrika, 105, 31–44. DOI: 10.1093/biomet/asx070.
   Web of Science ®Google Scholar
• Belloni, A., and Chernozhukov, V. (2011), “l1-Penalized Quantile Regression in High-Dimensional Sparse Models,” The Annals of Statistics, 39, 82–130.
   Web of Science ®Google Scholar
• Bianco, A. M., and Yohai, V. J. (1996), “Robust Estimation in the Logistic Regression Model,” in Robust Statistics, Data Analysis, and Computer Intensive Methods, ed. H. Rieder, New York: Springer, pp. 17–34.
   Google Scholar
• Boyd, S., and Vandenberghe, L. (2004), Convex Optimization, Cambridge: Cambridge University Press.
   Google Scholar
• Bregman, L. M. (1967), “The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming,” USSR Computational Mathematics and Mathematical Physics, 7, 200–217. DOI: 10.1016/0041-5553(67)90040-7.
   Google Scholar
• Bühlmann, P., and Yu, B. (2003), “Boosting With the L2 Loss: Regression and Classification,” Journal of the American Statistical Association, 98, 324–339. DOI: 10.1198/016214503000125.
   Web of Science ®Google Scholar
• Candes, E. J., and Tao, T. (2005), “Decoding by Linear Programming,” IEEE Transactions on Information Theory, 51, 4203–4215. DOI: 10.1109/TIT.2005.858979.
   Web of Science ®Google Scholar
• Cantoni, E., and Ronchetti, E. (2001), “Robust Inference for Generalized Linear Models,” Journal of the American Statistical Association, 96, 1022–1030. DOI: 10.1198/016214501753209004.
   Web of Science ®Google Scholar
• Chapelle, O. (2007), “Training a Support Vector Machine in the Primal,” Neural Computation, 19, 1155–1178. DOI: 10.1162/neco.2007.19.5.1155.
   PubMed Web of Science ®Google Scholar
• Chinchor, N. (1992), “MUC-4 Evaluation Metrics,” in Proceedings of the 4th Conference on Message Understanding, pp. 22–29. DOI: 10.3115/1072064.1072067.
   Google Scholar
• Croux, C., and Haesbroeck, G. (2003), “Implementing the Bianco and Yohai Estimator for Logistic Regression,” Computational Statistics & Data Analysis, 44, 273–295.
   Web of Science ®Google Scholar
• Freue, G. V. C., Kepplinger, D., Salibián-Barrera, M., and Smucler, E. (2019), “Robust Elastic Net Estimators for Variable Selection and Identification of Proteomic Biomarkers,” The Annals of Applied Statistics, 13, 2065–2090. DOI: 10.1214/19-AOAS1269.
   Web of Science ®Google Scholar
• Hadi, A. S., and Luceño, A. (1997), “Maximum Trimmed Likelihood Estimators: A Unified Approach, Examples, and Algorithms,” Computational Statistics & Data Analysis, 25, 251–272.
   Web of Science ®Google Scholar
• Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., and Stahel, W. A. (2011), Robust Statistics: The Approach based on Influence Functions (Vol. 1), Hoboken, NJ: Wiley.
   Google Scholar
• Hastie, T., Tibshirani, R., and Wainwright, M. (2015), Statistical Learning With Sparsity: The Lasso and Generalizations, Boca Raton, FL: Chapman and Hall/CRC.
   Google Scholar
• Huber, P., and Ronchetti, E. (2009), Robust Statistics, Hoboken, NJ: Wiley.
   Google Scholar
• Hunter, D. R., and Lange, K. (2004), “A Tutorial on MM Algorithms,” The American Statistician, 58, 30–37. DOI: 10.1198/0003130042836.
   Web of Science ®Google Scholar
• Khan, J. A., Van Aelst, S., and Zamar, R. H. (2007), “Robust Linear Model Selection Based on Least Angle Regression,” Journal of the American Statistical Association, 102, 1289–1299. DOI: 10.1198/016214507000000950.
   Web of Science ®Google Scholar
• Kurnaz, F. S., Hoffmann, I., and Filzmoser, P. (2018), “Robust and Sparse Estimation Methods for High-Dimensional Linear and Logistic Regression,” Chemometrics and Intelligent Laboratory Systems, 172, 211–222. DOI: 10.1016/j.chemolab.2017.11.017.
   Web of Science ®Google Scholar
• Little, M. A., McSharry, P. E., Roberts, S. J., Costello, D. A., and Moroz, I. M. (2007), “Exploiting Nonlinear Recurrence and Fractal Scaling Properties for Voice Disorder Detection,” Biomedical Engineering Online, 6, 23. DOI: 10.1186/1475-925X-6-23.
   PubMed Web of Science ®Google Scholar
• Loh, P.-L. (2017), “Statistical Consistency and Asymptotic Normality for High-Dimensional Robust M-Estimators,” The Annals of Statistics, 45, 866–896.
   Web of Science ®Google Scholar
• Maronna, R. A., Martin, D. R., and Yohai, V. J. (2006), Robust Statistics: Theory and Methods (Vol. 1), New York: Wiley.
   Google Scholar
• Müller, S., and Welsh, A. (2005), “Outlier Robust Model Selection in Linear Regression,” Journal of the American Statistical Association, 100, 1297–1310. DOI: 10.1198/016214505000000529.
   Web of Science ®Google Scholar
• Nair, V., and Hinton, G. E. (2010), “Rectified Linear Units Improve Restricted Boltzmann Machines,” in Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814.
   Google Scholar
• Needell, D., and Tropp, J. A. (2009), “CoSaMP: Iterative Signal Recovery From Incomplete and Inaccurate Samples,” Applied and Computational Harmonic Analysis, 26, 301–321. DOI: 10.1016/j.acha.2008.07.002.
   Web of Science ®Google Scholar
• Öllerer, V., Croux, C., and Alfons, A. (2015), “The Influence Function of Penalized Regression Estimators,” Statistics, 49, 741–765. DOI: 10.1080/02331888.2014.922563.
   Web of Science ®Google Scholar
• Pregibon, D. (1982), “Resistant Fits for Some Commonly Used Logistic Models With Medical Applications,” Biometrics, 38, 485–498. DOI: 10.2307/2530463.
   PubMed Web of Science ®Google Scholar
• Qin, Y., Li, S., Li, Y., and Yu, Y. (2017), “Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection,” arXiv no. 1708.05439.
   Google Scholar
• Ronchetti, E., Field, C., and Blanchard, W. (1997), “Robust Linear Model Selection by Cross-Validation,” Journal of the American Statistical Association, 92, 1017–1023. DOI: 10.1080/01621459.1997.10474057.
   Web of Science ®Google Scholar
• Rousseeuw, P. J. (1985), “Multivariate Estimation With High Breakdown Point,” Mathematical Statistics and Applications, 8, 283–297.
   Google Scholar
• Rousseeuw, P. J., and Driessen, K. V. (1999), “Computing LTS Regression for Large Data Sets,” Technical Report, Institute of Mathematical Statistics Bulletin.
   Google Scholar
• Rousseeuw, P. J., and Hubert, M. (1997), “Recent Developments in PROGRESS,” Lecture Notes-Monograph Series, 31, 201–214.
   Google Scholar
• Rousseeuw, P. J., and Yohai, V. (1984), “Robust Regression by Means of S-Estimators,” in Robust and Nonlinear Time Series Analysis, eds. J. Franke, W. Härdle, and D. Martin, New York: Springer, pp. 256–272.
   Google Scholar
• Sakar, B. E., Isenkul, M. E., Sakar, C. O., Sertbas, A., Gurgen, F., Delil, S., Apaydin, H., and Kursun, O. (2013), “Collection and Analysis of a Parkinson Speech Dataset With Multiple Types of Sound Recordings,” IEEE Journal of Biomedical and Health Informatics, 17, 828–834. DOI: 10.1109/JBHI.2013.2245674.
   PubMed Web of Science ®Google Scholar
• Salibian-Barrera, M., and Van Aelst, S. (2008), “Robust Model Selection Using Fast and Robust Bootstrap,” Computational Statistics & Data Analysis, 52, 5121–5135.
   Web of Science ®Google Scholar
• She, Y., and Chen, K. (2017), “Robust Reduced-Rank Regression,” Biometrika, 104, 633–647. DOI: 10.1093/biomet/asx032.
   PubMed Web of Science ®Google Scholar
• She, Y., and Owen, A. B. (2011), “Outlier Detection Using Nonconvex Penalized Regression,” Journal of the American Statistical Association, 106, 626–639. DOI: 10.1198/jasa.2011.tm10390.
   Web of Science ®Google Scholar
• She, Y., and Tran, H. (2019), “On Cross-Validation for Sparse Reduced Rank Regression,” Journal of the Royal Statistical Society, Series B, 81, 145–161. DOI: 10.1111/rssb.12295.
   Google Scholar
• She, Y., Wang, J., Li, H., and Wu, D. (2013), “Group Iterative Spectrum Thresholding for Super-Resolution Sparse Spectral Selection,” IEEE Transactions on Signal Processing, 61, 6371–6386. DOI: 10.1109/TSP.2013.2281303.
   Web of Science ®Google Scholar
• She, Y., Wang, Z., and Jin, J. (2020), “Analysis of Generalized Bregman Surrogate Algorithms for Nonsmooth Nonconvex Statistical Learning,” The Annals of Statistics (to appear).
   Google Scholar
• Tibshirani, R. (1996), “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society, Series B, 58, 267–288. DOI: 10.1111/j.2517-6161.1996.tb02080.x.
   Google Scholar
• van de Geer, S. A., and Bühlmann, P. (2009), “On the Conditions Used to Prove Oracle Results for the Lasso,” Electronic Journal of Statistics, 3, 1360–1392. DOI: 10.1214/09-EJS506.
   Web of Science ®Google Scholar
• van der Vaart, A. W. (1998), Asymptotic Statistics, Cambridge: Cambridge University Press.
   Google Scholar
• Vandev, D., and Neykov, N. (1998), “About Regression Estimators With High Breakdown Point,” Statistics: A Journal of Theoretical and Applied Statistics, 32, 111–129. DOI: 10.1080/02331889808802657.
   Web of Science ®Google Scholar
• Zhang, T. (2013), “Multi-Stage Convex Relaxation for Feature Selection,” Bernoulli, 19, 2277–2293. DOI: 10.3150/12-BEJ452.
   Web of Science ®Google Scholar