Choice of the ridge factor from the correlation matrix determinant

ABSTRACT

Ridge regression is the alternative method to ordinary least squares, which is mostly applied when a multiple linear regression model presents a worrying degree of collinearity. A relevant topic in ridge regression is the selection of the ridge parameter, and different proposals have been presented in the scientific literature. Since the ridge estimator is biased, its estimation is normally based on the calculation of the mean square error (MSE) without considering (to the best of our knowledge) whether the proposed value for the ridge parameter really mitigates the collinearity. With this goal and different simulations, this paper proposes to estimate the ridge parameter from the determinant of the matrix of correlation of the data, which verifies that the variance inflation factor (VIF) is lower than the traditionally established threshold. The possible relation between the VIF and the determinant of the matrix of correlation is also analysed. Finally, the contribution is illustrated with three real examples.

KEYWORDS:

• Multicollinearity
• ridge regression
• variance inflation factor
• mean square error

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Román Salmerón Gómez  http://orcid.org/0000-0003-2589-4058

Catalina B. García  http://orcid.org/0000-0003-1622-3877

Notes

1 The condition number (CN) of the matrix ${\mathbf{X}}^t\mathbf{X}+k\mathbf{I}$ (k>0) is less than the CN for matrix ${\mathbf{X}}^t\mathbf{X}$ [28]. Its inverse is sounder.

2 Note that ${\mathbf{X}}^t\mathbf{X}=\mathbf{\Gamma }\cdot {\mathbf{D}}_{\lambda }\cdot {\mathbf{\Gamma }}^t$, where $\mathbf{\Gamma }$ are the eigenvector matrix of ${\mathbf{X}}^t\mathbf{X}$; thus, ${\mathbf{\Gamma }}^t={\mathbf{\Gamma }}^{-1}$, and ${\lambda }_1,\dots ,{\lambda }_p$ are its eigenvalues, being ${\mathbf{D}}_{\lambda }=diag({\lambda }_1,\dots ,{\lambda }_p)$.

3 As we have said in the Introduction, we use $1-det\left(\mathbf{R}\right)$ because for p=3, this value coincides with the value of ${\rho }^2$. Then, the results are comparable to the findings in García et al. [6]

4 Dataset extracted from the World Bank website. All data are expressed in logarithms.

5 Dataset available in R-project (longley data).

6 Considering Table 15, the MSE is decreasing if k is lower than 0.00000007707998.

7 The limit of the MSE when k tends to infinity.

8 Situations where the number of cases is reduced (values of 50% and, even, 0%) are not considered.