A Confidence Bound Approach to Choosing the Biasing Parameter in Ridge Regression

Abstract

Consider the multiple linear regression model Y = Xβ + ∈, ∈ ∼ Ň(0, σ² I) where the matrix S = X' X is ill conditioned. A confidence bound approach is developed for choosing k in the ridge estimator β*(k) = (S + kI)^–1 X' Y as follows: A parameter is defined that is essentially the largest (constant) k one could use and still have β*(k)'s mean squared error (MSE) be less than MSE (where is the usual estimate of β). A procedure is then developed to obtain a lower confidence bound k _γ for , and the estimator β* ≡ β*(k _γ) is considered.

Key Words:

• Multiple linear regression: Multicollinearity
• Ridge regression