Simultaneous Inference for High-Dimensional Linear Models

ABSTRACT

This article proposes a bootstrap-assisted procedure to conduct simultaneous inference for high-dimensional sparse linear models based on the recent desparsifying Lasso estimator. Our procedure allows the dimension of the parameter vector of interest to be exponentially larger than sample size, and it automatically accounts for the dependence within the desparsifying Lasso estimator. Moreover, our simultaneous testing method can be naturally coupled with the margin screening to enhance its power in sparse testing with a reduced computational cost, or with the step-down method to provide a strong control for the family-wise error rate. In theory, we prove that our simultaneous testing procedure asymptotically achieves the prespecified significance level, and enjoys certain optimality in terms of its power even when the model errors are non-Gaussian. Our general theory is also useful in studying the support recovery problem. To broaden the applicability, we further extend our main results to generalized linear models with convex loss functions. The effectiveness of our methods is demonstrated via simulation studies. Supplementary materials for this article are available online.

KEYWORDS:

• Desparsifying Lasso
• High-dimensional linear models
• Multiple testing
• Multiplier Bootstrap
• Simultaneous inference
• Support recovery

Acknowledgements

The authors thank the Editor, Associate Editor and reviewers for their constructive comments and helpful suggestions, which substantially improved the article.

Funding

Research Sponsored by NSF CAREER Award DMS-1151692, DMS-1418042, Simons Fellowship in Mathematics, Office of Naval Research (ONR N00014-15-1-2331) and a grant from Indiana Clinical and Translational Sciences Institute. Guang Cheng was on sabbatical at Princeton while part of this work was carried out; he would like to thank the Princeton ORFE department for its hospitality and support.

Notes

1 This resparsifying procedure has the merits that it can improve the l _∞-bounds of Lasso and has l_q -bounds similar to Lasso (under sparsity conditions).