Robust adaptive variable selection in ultra-high dimensional linear regression models

Abstract

We consider the problem of simultaneous variable selection and parameter estimation in an ultra-high dimensional linear regression model. The adaptive penalty functions are used in this regard to achieve the oracle variable selection property with simpler assumptions and lesser computational burden. Noting the non-robust nature of the usual adaptive procedures (e.g. adaptive LASSO) based on the squared error loss function against data contamination, quite frequent with modern large-scale data sets (e.g. noisy gene expression data, spectra and spectral data), in this paper, we present a new adaptive regularization procedure using a robust loss function based on the density power divergence (DPD) measure under a general class of error distributions. We theoretically prove that the proposed adaptive DPD-LASSO estimator of the regression coefficients is highly robust, consistent, asymptotically normal and leads to robust oracle-consistent variable selection under easily verifiable assumptions. Numerical illustrations are provided for the mostly used normal and heavy-tailed error densities. Finally, the proposal is applied to analyse an interesting spectral dataset, in the field of chemometrics, regarding the electron-probe X-ray microanalysis (EPXMA) of archaeological glass vessels from the 16th and 17th centuries.

Keywords:

• High-dimensional linear regression models
• adaptive LASSO estimator
• non-polynomial dimensionality
• oracle property
• density power divergence

Mathematics Subject Classifications:

• MSC Primary 62F35
• MSEC Secondary62J07

Acknowledgments

Authors are grateful to the anonymous referees and the Editor for their insightful comments and suggestions that have improved the manuscript significantly. This research is supported by the Spanish Grants PGC2018-095 194-B-100 and FPU 19/01824. Research of AG is also partially supported by an INSPIRE Faculty Research Grant and a grant (No. SRG/2020/000072) from SERB, both under the Department of Science and Technology, Government of India, India. M.Jaenada and L.Pardo are members of the Interdisciplinary Mathematics Institute. The authors have no conflict of interest, financial or otherwise.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Department of Science and Technology, Ministry of Science and Technology, India [INSPIRE Faculty Research Grant], Ministerio de Educacion, Cultura y Deporte [FPU19/01824]Ministerio de Universidades, Spain[PGC2018-095 194-B-100], Science and Engineering Research Board [SRG/2020/000072].