Graphical group ridge

Abstract

This article introduces a novel method, called Graphical Group Ridge (GG-Ridge), which classifies ridge regression predictors in disjoint groups of conditionally correlated variables and derives different penalties (shrinkage parameters) for these groups of predictors. It combines the ridge regression method with the graphical model for high-dimensional data (i.e. the number of predictors, p, exceeds the number of cases, n) or ill-conditioned data (e.g. in the presence of multicollinearity among predictors). Although ridge regression is very effective with these types of data, its main shortcoming is that it applies the same penalty to all predictors, which can consequently limit the reduction in both the mean square error and the prediction mean square error, and over-shrink some predictors. This issue is addressed by the new method which reduces the mean square errors by assigning different penalties to different groups of predictors. Moreover, it reduces the extent of over-shrinking of predictors as compared to the ridge method, which is a desirable property in many fields such as finance, genetics and climate. The performance of the GG-Ridge method is investigated through two simulation studies and a real data analysis, and the results are compared with those of Ridge regression, and Elastic Net. The results indicate that the GG-Ridge outperforms these two methods in reducing the mean square errors, the prediction mean square error, and the bias of coefficients estimates.

Keywords:

• Graphical model
• unbiased estimation
• non-informative features
• LARS
• Lasso
• ridge regression

2010 Mathematics Subject Classification:

• 62J07

Disclosure statement

No potential conflict of interest was reported by the author(s).