Variable selection via penalized minimum φ-divergence estimation in logistic regression

Abstract

We propose penalized minimum φ-divergence estimator for parameter estimation and variable selection in logistic regression. Using an appropriate penalty function, we show that penalized φ-divergence estimator has oracle property. With probability tending to 1, penalized φ-divergence estimator identifies the true model and estimates nonzero coefficients as efficiently as if the sparsity of the true model was known in advance. The advantage of penalized φ-divergence estimator is that it produces estimates of nonzero parameters efficiently than penalized maximum likelihood estimator when sample size is small and is equivalent to it for large one. Numerical simulations confirm our findings.

Keywords:

• φ-divergence
• logistic regression
• penalized MLE
• SCAD
• variable selection

Acknowledgements

The authors thank the Editor and anonymous referees whose suggestions led to the significant improvement in this paper.