Partial ridge regression under multicollinearity

ABSTRACT

In multiple linear regression analysis, linear dependencies in the regressor variables lead to ill-conditioning known as multicollinearity. Multicollinearity inflates variance of the estimates as well as causes changes in direction of signs of the coefficient estimates leading to unreliable, and many times erroneous inference. Principal components regression and ridge or shrinkage approach have not provided completely satisfactory results in dealing with the multicollinearity. There are host of issues in ridge regression like choosing bias k and stability or consistency of the variances which still remain unresolved. In this paper, a partial ridge regression estimation is proposed, which involves selectively adjusting the ridge constants associated with highly collinear variables to control instability in the variances of coefficient estimates. Results based on synthetic data from simulations, and a real-world data set from the manufacturing industry show that the proposed method outperforms the existing solutions in terms of bias, mean square error, and relative efficiency of the estimated parameters.

KEYWORDS:

• Linear models
• multicollinearity
• ridge regression
• matrix factorization
• MSE

Acknowledgments

The authors would like to thank the referees for their valuable comments and suggestions which has helped in the marked improvement of the paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

M. Gallo http://orcid.org/000-0001-7904-0491