Subspace Estimation with Automatic Dimension and Variable Selection in Sufficient Dimension Reduction

Abstract

Sufficient dimension reduction (SDR) methods target finding lower-dimensional representations of a multivariate predictor to preserve all the information about the conditional distribution of the response given the predictor. The reduction is commonly achieved by projecting the predictor onto a low-dimensional subspace. The smallest such subspace is known as the Central Subspace (CS) and is the key parameter of interest for most SDR methods. In this article, we propose a unified and flexible framework for estimating the CS in high dimensions. Our approach generalizes a wide range of model-based and model-free SDR methods to high-dimensional settings, where the CS is assumed to involve only a subset of the predictors. We formulate the problem as a quadratic convex optimization so that the global solution is feasible. The proposed estimation procedure simultaneously achieves the structural dimension selection and coordinate-independent variable selection of the CS. Theoretically, our method achieves dimension selection, variable selection, and subspace estimation consistency at a high convergence rate under mild conditions. We demonstrate the effectiveness and efficiency of our method with extensive simulation studies and real data examples. Supplementary materials for this article are available online.

Keywords:

• Central subspace
• Sufficient dimension reduction
• Variable selection

Supplementary Materials

The R code, additional numerical and theoretical results, and proofs are contained in Supplementary Materials.

Acknowledgments

The authors are grateful to the Editor, Associate Editor and three referees for insightful comments that have led to significant improvements of this article.

Additional information

Funding

Research for this article was supported in part by grants CCF-1908969, DMS-2053697, and DMS-2113590 from the U.S. National Science Foundation.