Multi-step adaptive elastic-net: reducing false positives in high-dimensional variable selection

Abstract

Regression and variable selection in high-dimensional settings, especially when $p\gg n$ has been a popular research topic in statistical machine learning. In recent years, many successful methods have been developed to tackle this problem. In this paper, we propose the multi-step adaptive elastic-net (MSA-Enet), a multi-step estimation algorithm built upon adaptive elastic-net regularization. The numerical study on simulation data and real-world biological data sets have shown that the MSA-Enet method tends to significantly reduce the number of false-positive variables, while still maintain the estimation accuracy. By analysing the variables eliminated in each step, more insight could be gained about the structure of the correlated variable groups. These properties are desirable in many real-world variable selection and regression problems.

Keywords:

• multi-step shrinkage
• adaptive regularization
• elastic net
• high dimensionality
• variable selection

AMS Subject Classifications:

• 62J05 (primary)
• 62J07 (secondary)

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Nan Xiao http://orcid.org/0000-0002-0250-5673

Additional information

Funding

This work was supported by the National Natural Science Foundation of China under Grant No. 11271374.