Spline-Lasso in High-Dimensional Linear Regression

Abstract

We consider a high-dimensional linear regression problem, where the covariates (features) are ordered in some meaningful way, and the number of covariates p can be much larger than the sample size n. The fused lasso of Tibshirani et al. is designed especially to tackle this type of problems; it yields sparse coefficients and selects grouped variables, and encourages local constant coefficient profile within each group. However, in some applications, the effects of different features within a group might be different and change smoothly. In this article, we propose a new spline-lasso or more generally, spline-MCP to better capture the different effects within the group. The newly proposed method is very easy to implement since it can be easily turned into a lasso or MCP problem. Simulations show that the method works very effectively both in feature selection and prediction accuracy. A real application is also given to illustrate the benefits of the method. Supplementary materials for this article are available online.

• Oracle inequality
• Spline-lasso
• Spline-MCP
• Variable selection.