In defense of LASSO

Abstract

Although LASSO has been criticized for selecting too many covariates, it is illustrated in this paper that the bigger model chosen by LASSO method is suitable for exploratory research aiming at identifying all potential causes for further scientific investigation. Up to now, all criticisms assume that the covariates are observed without measurement errors, which is not likely to be true in many practical situations. Under measurement errors, the meaning of “relevant covariates” can be ambiguous. In such a situation, some covariates without an association with the response can be “potentially relevant”. The crucial point is that “relevant” and “potentially relevant” covariates cannot be distinguished based on the observed data in the presence of measurement errors. To avoid misinterpretation, both should be included in the model. This means that a bigger model is preferred. To understand the subset of covariates that should be included, a factor model of the covariates is introduced. Furthermore, new consistency theory is established under conditions weaker than those in Meinshausen and Bühlmann to cope with the situations where the preferred subset is not the same as the true model.

Keywords:

• Exploratory research
• LASSO
• penalized likelihood estimation
• prediction error
• variable selection

Additional information

Funding

Chi Tim, Ng’s work is supported by National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No.NRF-2017R1C1B2011652). Woojoo Lee was supported by a Grant from the Next-Generation BioGreen 21 program (Project No. PJ01337701), Rural Development Administration, Republic of Korea. Youngjo Lee’s work is supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. 2019R1A2C1002408).