Understanding Partial Statistics and Redundancy of Variables in Regression and Discriminant Analysis

Abstract

Unexperienced users of statistical methods are sometimes confused by seeming contradictions between the results obtained by univariate variable-by-variable analyses and those obtained by multivariate methods. This article provides some teaching aids that I found useful for explaining the seemingly paradoxical situations to a mathematically untrained audience. Six typical situations are introduced in graphical form and supplemented with numerical examples, illustrating the notion of redundancy of variables and the meaning of partial statistics arising in multiple regression and discriminant analysis.

Key Words:

• Collinearity
• Correlation
• Mahalanobis distance
• Mean difference
• Multiple correlation
• Partial correlation
• Variable selection