On the n-dimensional geometry of regression diagnostics

Abstract

The n-dimensional geometry of collinearity and data that are influential in least-squares linear regression is explored. A generalization of vector space dimensionality is introduced to provide an intuitive description of these problems. It is also noted that this new measure of dimensionality plays the role of the usual dimension in a James-Stein like result. Some common regression diagnostics are critically examined in this geometric framework.

Keywords:

• collinearity
• effective dimension,
• influence
• leverage
• linear model
• minimax estimation
• numerical ranks
• ridge regression