Comparing Performance of Multiclass Classification Systems with ROC Manifolds: When Volume and Correct Classification Fails

Abstract

Higher dimensional surfaces are used to examine the diagnostic performance of multiclass classification systems. These surfaces are extensions of the ROC curve and are known as ROC surfaces or manifolds. Manifolds may be constructed from either the correct classifications or from the misclassifications of the diagnostic system. Comparisons of the usefulness of each of these ROC manifolds with respect to the performance of the diagnostic system are made with emphasis on inferences from volume under the surface and optimal operating points (thresholds) of the system. Recommendations for when to use each type of ROC manifold and performance measure are discussed.

Keywords:

• Bayes cost
• ROC curve
• ROC manifold
• Sensitivity
• Specificity
• Volume under the surface, VUS
• Youden's Index

Mathematical Subject Classification:

• Primary 62
• 62G05
• Secondary 57M
• 57M99