The coefficient of determination in the ridge regression

Abstract

In a linear regression, the coefficient of determination, R², is a relevant measure that represents the percentage of variation in the dependent variable that is explained by a set of independent variables. Thus, it measures the predictive ability of the estimated model. For an ordinary least squares (OLS) estimator, this coefficient is calculated from the decomposition of the sum of squares. However, when the model presents collinearity problems (a strong linear relation between the independent variables), the OLS estimation is unstable, and other estimation methodologies are proposed, with the ridge estimation being the most widely applied. This paper shows that the decomposition of the sum of squares is not verified in the ridge regression and proposes how the coefficient of determination should be calculated in this case.

Keywords:

• Multicollinearity
• Goodness of fit
• Sum of squares decomposition
• Transformation of variables

Notes

1 Where SST is the sum of total squares, SSE is the sum of squares explained and SSR is the sum of squares of errors.