Study of partial least squares and ridge regression methods

ABSTRACT

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.

KEYWORDS:

• Multicollinearity
• Partial least squares
• Ridge regression

MATHEMATICS SUBJECT CLASSIFICATION:

• 62J05
• 62J07

Acknowledgment

This work is part of the research carried out by ”Grupo Regional de Estudios de Opinión.”

Funding

This research was supported by Universidad del Bío Bío, DIUBB Grant # 141108 3/R and DIUBB Grant # GI151508/EF.