On estimation error bounds of the Elastic Net when p ≫ n

Abstract

We study the estimation property of the Elastic Net estimator in high-dimensional linear regression models where the number of parameters p is comparable to or larger than the sample size n. In such a situation, one often assumes sparsity of the true regression coefficient vector ${\beta }^{\ast }\in R^p$, i.e., assuming that ${\beta }^{\ast }$ belongs to an ${\ell }_q$-ball with radius $R_q$, ${\mathbb{B}}_q(R_q)$, for some $q\in [0,1]$. In this paper, we provide ${\ell }_2$-estimation error bounds for the Elastic Net and naive Elastic Net estimators under a unified framework for high-dimensional analysis of M-estimators proposed by Negahban et al. [A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Adv Neural Inf Process Syst. 2009;22:1348–1356]. We show that for both cases of exact sparsity and weak sparsity, under the same conditions on the design matrix, the Elastic Net estimator achieves a slightly better error bound than the Lasso estimator by suitably choosing the tuning parameters.

Keywords:

• Lasso
• naive Elastic Net
• Elastic Net
• model selection consistency
• estimation consistency

Acknowledgments

The authors are grateful to the associate editor, two reviewers, and professor Bin Yu for many insightful comments and helpful suggestions, which led to a substantially improved manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).