Sparsely restricted penalized estimators

Abstract

Parameters of a linear regression model might be estimated with Ordinary Least Squares (OLS). If there are linear restrictions on model parameters, it is possible to use Restricted Least Squares (RLS). Both OLS and RLS fail when the number of predictors is large and not suitable for model selection in a sparse model. In contrast to OLS and RLS, Least Absolute Shrinkage and Selection Operator (LASSO) and Bridge estimators perform well in sparse models and can be used for model selection and estimation, simultaneously. A Restricted LASSO (RLASSO) estimator is proposed in the literature recently but it doesn’t provide sparse solutions. In this paper, we propose new sparsely restricted penalized estimators called Sparsely Restricted LASSO (SRL) and Sparsely Restricted Bridge (SRB) based on RLASSO and Restricted Bridge (RBridge). We show that SRL and SRB produces sparse solutions while satisfying restrictions of model parameters. We compare aforementioned estimators with Monte Carlo simulations. We also use prostate cancer dataset that is widely used in the literature for a numerical application. Our comparisons show that SRL and SRB outperform the remaining estimators in terms of model selection performance and mean squared error.

Keywords:

• Restricted penalized estimators
• Sparsely restricted bridge
• Sparsely restricted LASSO
• Model selection

Acknowledgments

Authors would like to thank two anonymous referees and participants of 11th International Statistics Days Conference for their helpful comments.

Additional information

Funding

This work was supported by Çukurova University Scientific Research Projects Unit under Grant SBA-2018-10961.