Regressor space outliers in ridge regression

Abstract

A simple analytical expression is derived for leverage in ridge regression. Leverage is shown to be a monotonically decreasing function of the value of the ridge parameter. This reduction in leverage is greatest for those observations lying substantially in the direction of the minor principal axes. Thus, ridge estimation copes with outliers in regressor space by downweighting their influence. A brief illustration is provided.

Keywords:

• eigenvalues
• eigenvectors
• leverage
• outliers
• ridge
• estimation