Stochastic Stepwise Ensembles for Variable Selection

Abstract

In this article, we advocate the ensemble approach for variable selection. We point out that the stochastic mechanism used to generate the variable-selection ensemble (VSE) must be picked with care. We construct a VSE using a stochastic stepwise algorithm and compare its performance with numerous state-of-the-art algorithms. Supplemental materials for the article are available online.

Key Words:

• False negative rate
• False positive rate
• LASSO
• Ranking
• Stochastic search
• Strength–diversity tradeoff