Krylov subspace solvers for ℓ1 regularized logistic regression method

Abstract

In this paper, we propose an approach based on Krylov subspace methods for the solution of ${\mathcal{\ell }}_1$ regularized logistic regression problem. The main idea is to transform the constrained ${\mathcal{\ell }}_2$-${\mathcal{\ell }}_1$ minimization problem obtained by applying the IRLS method to a ${\mathcal{\ell }}_2$-${\mathcal{\ell }}_2$ one that allow regularization matrices in the usual 2-norm regularization term. The regularization parameter that controls the equilibrium between the minimization of the two terms of the ${\mathcal{\ell }}_2$-${\mathcal{\ell }}_2$ minimization problem can be then chosen inexpensively by solving some reduced minimization problems related to generalized cross-validation (GCV) methods. These reduced problems can be obtained after a few iterations of Krylov subspace based methods. The goal of our simulation study is directed toward the variable selection and the prediction accuracy performance of the proposed method in solving a ${\mathcal{\ell }}_1$ regularized logistic regression problem in large dimensional data with different correlation structures among predictors. Finally, real data are used to confirm the efficiency of the proposed method in terms of the computational cost.

KEYWORDS:

• Logistic regression model
• Krylov subspace methods
• regularization
• ${\mathcal{\ell }}_1$-norm
• variable selection

Acknowledgements

The authors would like to thank the Editor and the referees for their useful comments which resulted in improving the quality of this article.