Efficient Global Approximation of Generalized Nonlinear ℓ₁-Regularized Solution Paths and Its Applications

Abstract

We consider efficient construction of nonlinear solution paths for general ℓ₁-regularization. Unlike the existing methods that incrementally build the solution path through a combination of local linear approximation and recalibration, we propose an efficient global approximation to the whole solution path. With the loss function approximated by a quadratic spline, we show that the solution path can be computed using a generalized Lars algorithm. The proposed methodology avoids high-dimensional numerical optimization and thus provides faster and more stable computation. The methodology also can be easily extended to more general regularization framework. We illustrate such flexibility with several examples, including a generalization of the elastic net and a new method that effectively exploits the so-called “support vectors” in kernel logistic regression.

Keywords:  :

• ℓ₁-regularization
• LARS
• LASSO
• Solution path
• Support vector pursuit