Robust Ridge Regression for High-Dimensional Data

Abstract

Ridge regression, being based on the minimization of a quadratic loss function, is sensitive to outliers. Current proposals for robust ridge-regression estimators are sensitive to “bad leverage observations,” cannot be employed when the number of predictors p is larger than the number of observations n, and have a low robustness when the ratio p / n is large. In this article a ridge-regression estimate based on repeated M estimation (“MM estimation”) is proposed. It is a penalized regression MM estimator, in which the quadratic loss is replaced by an average of , where r_i are the residuals and the residual scale from an initial estimator, which is a penalized S estimator; and ρ is a bounded function. The MM estimator can be computed for p>n and is robust for large p / n. A fast algorithm is proposed. The advantages of the proposed approach over its competitors are demonstrated through both simulated and real data. Supplemental materials are available online.

Keywords:  :

• MM estimate
• S estimate
• Shrinking