Geometry of Ridge Regression Illustrated

Abstract

For tutorial purposes ridge traces are displayed in estimation space for repeated samples from a completely known population. Figures given illustrate the initial advantages accruing to ridge-type shrinkage of the least squares coefficients, especially in some cases of near collinearity. The figures also show that other shrunken estimators may perform better or worse, depending on the parameters and design matrix; and they illustrate the problem of choosing a shrinkage parameter or stopping rule. Thus the figures help motivate results previously established algebraically.

Key Words:

• Collinearity
• Principal component regression
• Ridge regression
• Ridge trace
• Shrunken estimation