General Sparse Boosting: Improving Feature Selection of L₂ Boosting by Correlation-Based Penalty Family

Abstract

In high-dimensional setting, componentwise L₂boosting has been used to construct sparse model that performs well, but it tends to select many ineffective variables. Several sparse boosting methods, such as, SparseL₂Boosting and Twin Boosting, have been proposed to improve the variable selection of L₂boosting algorithm. In this article, we propose a new general sparse boosting method (GSBoosting). The relations are established between GSBoosting and other well known regularized variable selection methods in the orthogonal linear model, such as adaptive Lasso, hard thresholds, etc. Simulation results show that GSBoosting has good performance in both prediction and variable selection.

Keywords:

• Adaptive Lasso
• Boosting algorithm
• Model selection
• Sparsity

Mathematics Subject Classification:

• Primary 62608
• Secondary 62H12