A coordinate majorization descent algorithm for ℓ₁ penalized learning

Abstract

The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1–22] is an extremely fast implementation of the standard coordinate descent algorithm for solving ℓ₁ penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the ℓ₁ penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.

Keywords:

• coordinate decent
• majorization–minimization
• glmnet
• lasso