Selection of Binary Variables and Classification by Boosting

Abstract

We adopt boosting for classification and selection of high-dimensional binary variables for which classical methods based on normality and non singular sample dispersion are inapplicable. Boosting seems particularly well suited for binary variables. We present three methods of which two combine boosting with the relatively classical variable selection methods developed in Wilbur et al. (2002). Our primary interest is variable selection in classification with small misclassification error being used as validation of proposed method for variable selection. Two of the new methods perform uniformly better than Wilbur et al. (2002) in one set of simulated and three real life examples.

Keywords:

• Boosting
• Cross validation
• DNA fingerprints
• High-dimensional data
• Multivariate binary data
• Thresholding
• Variable selection

Mathematics Subject Classification:

• Primary 62H30
• Secondary 62P10

Acknowledgment

We thank an anonymous refree whose careful reading and suggestions have led to an improvement presentation.