Shrinkage and penalized estimators in weighted least absolute deviations regression models

ABSTRACT

In this paper, we consider the estimation problem of the weighted least absolute deviation (WLAD) regression parameter vector when there are some outliers or heavy-tailed errors in the response and the leverage points in the predictors. We propose the pretest and James–Stein shrinkage WLAD estimators when some of the parameters may be subject to certain restrictions. We derive the asymptotic risk of the pretest and shrinkage WLAD estimators and show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage WLAD estimator is strictly less than the unrestricted WLAD estimator. On the other hand, the risk of the pretest WLAD estimator depends on the validity of the restrictions on the parameters. Furthermore, we study the WLAD absolute shrinkage and selection operator (WLAD-LASSO) and compare its relative performance with the pretest and shrinkage WLAD estimators. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the unrestricted WLAD estimator. A real-life data example using body fat study is used to illustrate the performance of the suggested estimators.

Keywords:

• Asymptotic distributional bias
• asymptotic distributional risk
• Monte Carlo simulation
• outliers
• pretest
• shrinkage
• WLAD-LASSO

Acknowledgements

We express our sincere thanks to the referees and the editor for their constructive and valuable suggestions, which led to an improvement of this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The research of Dr. Shakhawat Hossain was supported by a grant from the Natural Sciences and Engineering Research Council of Canada.