Entropy-based model-free feature screening for ultrahigh-dimensional multiclass classification

Abstract

Most feature screening methods for ultrahigh-dimensional classification explicitly or implicitly assume the covariates are continuous. However, in the practice, it is quite common that both categorical and continuous covariates appear in the data, and applicable feature screening method is very limited. To handle this non-trivial situation, we propose an entropy-based feature screening method, which is model free and provides a unified screening procedure for both categorical and continuous covariates. We establish the sure screening and ranking consistency properties of the proposed procedure. We investigate the finite sample performance of the proposed procedure by simulation studies and illustrate the method by a real data analysis.

Keywords:

• entropy
• feature screening
• information gain
• multiclass classification
• sure screening property

AMS Subject Classification:

• 62H30
• 62G20

Acknowledgements

The authors would like to thank the editor and two anonymous referees for their helpful comments and suggestions.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by Shanghai Nature Science Foundation 15ZR1410300, Program of Shanghai Subject Chief Scientist (14XD1401600), and the 111 Project (B14019).