The L₁ Method for Robust Nonparametric Regression

Abstract

Consider the problem of estimating the regression function underlying a set of data that is contaminated by a heavy-tailed error distribution. The two standard approaches to such a problem are each flawed. Robust parametric least squares is appropriate only if there is good reason to believe that the underlying function has some particular form, whereas most nonparametric regression methods are asymptotically equivalent to kernel regression methods, which are not resistant against outliers. Existing algorithms for robustifying nonparametric regression procedures use either nonlinear optimization of an influence function or iterative solution of local polynomial fitting using reweighted least squares. Neither of these two approaches combines computational ease with asymptotic theoretical results. Furthermore, application of the robust procedure has been limited almost exclusively to the case of a single explanatory variable with the response variable. In this article a new hybrid method is proposed that combines nonparametric regression with the L ₁ norm. Applying the L ₁ norm on the regression residuals leads naturally to a robust estimator in any dimension. Unlike diagnostic and influence approaches, the L ₁ metric can handle many outliers, whether isolated or clumped, without any requirement to estimate the scale of the residuals. Despite L ₁'s reputation for being computationally intractable, fitting a polynomial by the least absolute deviations criterion is equivalent to solving a linear program with special structure. By using the L ₁ norm over local neighborhoods, a method that is also nonparametric is constructed. Additionally, the new method generalizes easily to several dimensions. To date, the problem of robust smoothing directly in several dimensions has met with little success, without resorting to robust additive models. A proof of consistency for the L ₁ algorithm is presented, and results from both real and simulated data are shown.

Key Words:

• Absolute error
• Bandwidth selection
• L ₁ regression
• Nonparametric regression
• Robust regression