On the estimation of the expected actual error rate in sample–based multinomial classification

Abstract

Assuming the two class case, several methods for estimating the expected actual error rate in sample–based multinomial classification are compared with respect to bias and mean absolute error. “Smooth” modifications of the estimators of COCHRAN and HOPKINS (1961) and HILLS (1966) are recommended. Some numerical results illustrate the performance of these estimators particularly in the case of sparsely occupied cells in contingency tables

Keywords:

• Primary 62H30,62H17
• secondary 62E25
• Multinomial distributions
• error rates
• posterior probability estimators
• plug–in estimatiors
• Bootstrap
• leaving–one–out
• cross–validation