Testing Many Zero Restrictions in a High Dimensional Linear Regression Setting

Abstract

We propose a test of many zero parameter restrictions in a high dimensional linear iid regression model with k $\gg$ n regressors. The test statistic is formed by estimating key parameters one at a time based on many low dimension regression models with nuisance terms. The parsimoniously parameterized models identify whether the original parameter of interest is or is not zero. Estimating fixed low dimension sub-parameters ensures greater estimator accuracy, it does not require a sparsity assumption nor therefore a regularized estimator, it is computationally fast compared to, for example, de-biased Lasso, and using only the largest in a sequence of weighted estimators reduces test statistic complexity and therefore estimation error. We provide a parametric wild bootstrap for p-value computation, and prove the test is consistent and has nontrivial $\sqrt{n/\{\hspace{0.17em}\mathrm{\text{ln}}\hspace{0.17em}(n){\mathcal{M}}_n\}}$-local-to-null power where ${\mathcal{M}}_n$ is the $l_{\infty }$ covariate fourth moment.

Keywords:

• High dimensional inference
• Linear regression
• max-test

Acknowledgments

This manuscript benefitted from the expert commentary and suggestions of two referees and an associate editor.

Disclosure Statement

No potential conflict of interest was reported by the author.

Notes

1 As a testament to the increasing amount of available data, in November 2022 the 27th General Conference on Weights and Measures presented new numerical prefixes handling, among other magnitudes,10³⁰.

2 This may carry over to (semi)nonparametric settings with fixed low dimension ${\delta }_{}$ and infinite dimensional function θ₀. See Chen and Pouzo (2015) for simulation evidence of sieve based bootstrapped t-tests.

3 Using Matlab (with coordinate descent and ADMM algorithms) via a SLURM scheduler on the Longleaf cluster at UNC, with 128 workers on 1 node, and setting n $=$ 100, $k_{\delta }$ $=$ 0, $k_{\theta ,n}$ $=$ 200, and 1000 bootstrap samples, one bootstrapped p-value under H₀ for the DBL max-statistic with 5-fold cross-validation took 335 sec (5.6 min). Increase $k_{\theta ,n}$ to 480, as in our simulation study, yielded a computation time of 8109 sec (135.2 min). The bootstrapped parsimonious max-test, by comparison, took only 4.17 and 5.1 sec, respectively, yielding up to a 1600x computation time gain. Increase n to 250 and set $k_{\theta ,n}$ $=$ 1144, and DBL required 50.21 hr, while our method took just 5.2 sec (roughly $35,000\times$ faster). See Section 4 for complete simulation details.

4 In our design x_t is bounded or Gaussian and is therefore ${\mathcal{L}}_p$-bounded for any p > 0.

5 See Dezeure et al. (2015) for discussion of an R-package hdi (high dimensional inference). At least for the class of hypotheses considered in this article, we anticipate that estimating many low dimension models by least squares will always dominate de-biased Lasso in terms of computation time, irrespective of the software or solver used. A comprehensive comparison across software platforms and numerical solvers, however, is left for subsequent research.

6 We suppress λ, ${\lambda }^{\prime }\lambda$ $=$ 1, when confusion is avoided.