New quantile based ridge M-estimator for linear regression models with multicollinearity and outliers

Abstract

The ordinary least squares and ridge regression estimators in a multiple linear regression model with multicollinearity and y-direction outliers lead to unfavorable results. In order to mitigate such situation, the available literature provides few ridge M-estimators to get precise estimates. The ridge parameter, k, plays a vital role in a bias-variance tradeoff for these estimators. However, for high signal-to-noise ratio and multicollinearity with y-direction outliers, the available methods may not perform well in terms of their mean squared error. In this article, we propose a new quantile based ridge M-estimator. The new estimator gives an automated choice of quantile probability of ridge parameter according to the level of noise and multicollinearity. Based on a simulation study, the new estimator outperforms the ordinary least square estimator, ridge estimator, and other considered ridge M-estimators especially for high multicollinearity, significant error variance, and y-direction outliers. Besides normal distribution, new estimator also performs well for heavy-tailed error distribution. Finally, two real-life examples are used to illustrate the application of the proposed estimator.

Keywords:

• M-estimator
• Mean squared error
• Multicollinearity
• Outliers
• Ridge parameter
• Ridge regression
• Signal-to-noise ratio

MSC 2010 CLASSIFICATION:

• 62J05
• 62J07

Additional information

Funding

Higher Education Commission Pakistan (https://www.hec.gov.pk) Indigenous PhD fellowship Phase-II Batch-III under grant no: 315-11394-2PS3-084 awarded to the first author.