A Simple Regularization Procedure for Discriminant Analysis

Abstract

Linear and quadratic discriminant analysis are likely to lead to unstable models and poor predictions in the presence of quasicolinearity among variables or in the case of the small sample and high-dimensional setting. A simple regularization procedure is proposed to cope with this problem. It is based on the introduction of a tuning parameter that draws a line between linear or quadratic discriminant analysis that is based on Mahalanobis distance and discriminant analysis based on the identity matrix. The tuning parameter is customized to individual situations by minimizing the cross-validated misclassification risk. The efficiency of the method of analysis in comparison with existing procedures is demonstrated on the basis of a data set and a large simulation study.

Keywords:

• Euclidean distance classifier
• Linear discriminant analysis
• Quadratic discriminant analysis
• Regularized discriminant analysis

Mathematics Subject Classification:

• 62H30