Shrinking Towards Subspaces in Multiple Linear Regression

Abstract

The problem of multiple linear regression and the use of shrinkage estimators is considered. It is pointed out that sometimes prior beliefs, about the signs and orders of magnitude of the regression coefftcients, for example, suggest a subspace L such that it is more natural to shrink the least squares estimate towards its projection onto L than towards the origin. An empirical Bayes approach is used to construct such an estimator for an arbitrary design matrix. The estimator is illustrated on the acetylene data previously analyzed by Marquardt and Snee (1975).

KEY WORDS:

• Ridge estimators
• Empirical Bayes
• Collinearity