Testing One Hypothesis Multiple Times: The Multidimensional Case

Abstract

The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the “testing one hypothesis multiple times” (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non identifiable parameter is specified, and a global p-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this article, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrarily large significance levels. This leads to an highly generalizable computational tool to perform hypothesis testing under nonstandard regularity conditions. Supplementary materials for this article are available online.

Keywords:

• Euler characteristics
• Graph theory
• Lipschitz–Killing curvatures
• Multidimensional signal search
• Non identifiability in hypothesis testing
• Non nested models

Acknowledgments

The authors thank two anonymous referees and the associate editor, whose comments and suggestions greatly improved the quality and clarity of the article. SA and DvD also thank Jan Conrad for the valuable discussion of the physics problems which motivated this work, and Brandon Anderson who provided the Fermi-LAT datasets used in the analyses.

Notes

1 We are interested in scenarios where local maxima become rarer and rarer as $c\to +\infty$. Hence, we are implicitly assuming that no ridges above c occur.

2 A Gaussian random field $\{Z(\mathit{\theta })\}$ is said to be “suitably regular” if it has almost surely continuous partial derivatives up to the second order, and if the two-tensor field induced by $\{Z(\mathit{\theta })\}$ satisfies the additional mild conditions specified in Definition 3.2 of Taylor and Adler (2003).

3 Notice that the main difference between the mesh ${\mathcal{M}}_k$ (or $\mathcal{M}{\prime }_k$) and the graph ${\mathcal{G}}_k^D$ is that the former depends on the position of its vertices in $\mathbf{\Theta }$ and their distance; whereas the latter only accounts for their connectivity.

4 http://fermi.gsfc.nasa.gov/ssc/data/analysis/software

Additional information

Funding

SA acknowledges support from the Swedish Research Council through a grant with PI: Jan Conrad. Finally, the authors acknowledge support from the Marie-Skodowska-Curie RISE (H2020-MSCA-RISE-2015-691164) grant provided by the European Commission.