The Generalized Ridge Trace Plot: Visualizing Bias and Precision

Abstract

In ridge regression and related shrinkage methods, the ridge trace plot, a plot of estimated coefficients against a shrinkage parameter, is a common graphical adjunct to help determine a favorable trade-off of bias against precision (inverse variance) of the estimates. However, standard unidimensional versions of this plot are ill-suited for this purpose because they show only bias directly and ignore the multidimensional nature of the problem.

A generalized version of the ridge trace plot is introduced, showing covariance ellipsoids in parameter space, whose centers show bias and whose size and shape show variance and covariance, respectively, in relation to the criteria for which these methods were developed. These provide a direct visualization of both bias and precision. Even two-dimensional bivariate versions of this plot show interesting features not revealed in the standard univariate version. Low-rank versions of this plot, based on an orthogonal transformation of predictor space extend these ideas to larger numbers of predictor variables, by focusing on the dimensions in the space of predictors that are likely to be most informative about the nature of bias and precision. Two well-known datasets are used to illustrate these graphical methods. The genridge package for R implements computation and display.

Key Words:

• Biplot
• Model selection
• Multivariate bootstrap
• Regression shrinkage
• Ridge regression
• Ridge trace plot
• Singular value decomposition
• Variance-shrinkage trade-off

ACKNOWLEDGMENTS

This work was supported by grant OGP0138748 from the National Sciences and Engineering Research Council of Canada to Michael Friendly. The author is grateful to John Fox for critical comments on an initial draft of this article and to the associate editor and two reviewers for suggestions and comments that have helped to sharpen and extend the ideas presented.

Additional information

Notes on contributors

Michael Friendly

Michael Friendly, Professor, Psychology Department, York University, Toronto, ON M3J 1P3 (E-mail: [email protected]).