Regression Methods for Combining Multiple Classifiers

Abstract

As no single classification method outperforms other classification methods under all circumstances, decision-makers may solve a classification problem using several classification methods and examine their performance for classification purposes in the learning set. Based on this performance, better classification methods might be adopted and poor methods might be avoided. However, which single classification method is the best to predict the classification of new observations is still not clear, especially when some methods offer similar classification performance in the learning set. In this article we present various regression and classical methods, which combine several classification methods to predict the classification of new observations. The quality of the combined classifiers is examined on some real data. Nonparametric regression is the best method of combining classifiers.

Keywords:

• Logistic regression
• Nonparametric regression
• Partial least-squares regression
• Regression combining technique
• Stacked regression
• The lasso and the elastic net

Mathematics Subject Classification:

• Primary 62H30
• Secondary 62G08