Variable Selection for Support Vector Machines

Abstract

Consider using values of variables X ₁, X ₂,…, X _p to classify entities into one of two classes. Kernel-based procedures such as support vector machines (SVMs) are well suited for this task. In general, the classification accuracy of SVMs can be substantially improved if instead of all p candidate variables, a smaller subset of (say m) variables is used. A new two-step approach to variable selection for SVMs is therefore proposed: best variable subsets of size k = 1,2,…, p are first identified, and then a new data-dependent criterion is used to determine a value for m. The new approach is evaluated in a Monte Carlo simulation study, and on a sample of data sets.

Keywords:

• Kernel model dimension
• Kernel target alignment
• Recursive feature elimination
• Variable selection for SVMs

Mathematics Subject Classification:

• 62H30
• 68T10

Acknowledgment

We acknowledge the use of the UCI Machine Learning Data Repository, and the valuable comments of a referee which led to this improved version of the original article.