A Model Selection Criterion for Discriminant Analysis of Several Groups When the Dimension is Larger than the Total Sample Size

Abstract

This article deals with a criterion for selection of variables for the multiple group discriminant analysis in high-dimensional data. The variable selection models considered for discriminant analysis in Fujikoshi (1985, 2002) are the ones based on additional information due to Rao (1948, 1970). Our criterion is based on Akaike information criterion (AIC) for this model. The AIC has been successfully used in the literature in model selection when the dimension p is smaller than the sample size N. However, the case when p > N has not been considered in the literature, because MLE can not be estimated corresponding to singularity of the within-group covariance matrix. A popular method used to address the singularity problem in high-dimensional classification is the regularized method, which replaces the within-group sample covariance matrix with a ridge-type covariance estimate to stabilize the estimate. In this article, we propose AIC-type criterion by replacing MLE of the within-group covariance matrix with ridge-type estimator. This idea follows Srivastava and Kubokawa (2008). Simulations revealed that our proposed criterion performs well.

Keywords:

• Akaike information criterion
• Discriminant analysis
• High-dimensional data
• Ridge-type estimator

Mathematics Subject Classification:

• Primary 62H12
• Secondary 62H30

Acknowledgments

The authors would like to thank the referees for suitable comments and careful reading. We are grateful to Professor Yasunori Fujikoshi for his advice and encouragement. In addition, this work was supported by Japan Society for the Promotion of Science (JSPS).