An Empirical Bayes Approach to Controlling the False Discovery Exceedance

Abstract

In large-scale multiple hypothesis testing problems, the false discovery exceedance (FDX) provides a desirable alternative to the widely used false discovery rate (FDR) when the false discovery proportion (FDP) is highly variable. We develop an empirical Bayes approach to control the FDX. We show that, for independent hypotheses from a two-group model and dependent hypotheses from a Gaussian model fulfilling the exchangeability condition, an oracle decision rule based on ranking and thresholding the local false discovery rate (lfdr) is optimal in the sense that the power is maximized subject to the FDX constraint. We propose a data-driven FDX procedure that uses carefully designed computational shortcuts to emulate the oracle rule. We investigate the empirical performance of the proposed method using both simulated and real data and study the merits of FDX control through an application for identifying abnormal stock trading strategies.

Keywords:

• Cautious data mining
• Local false discovery rate
• Multiple hypothesis testing
• Poisson binomial distribution
• Trading strategies

Supplementary Materials

The supplementary materials contain the proofs of several propositions omitted in the main text and, in particular, establish the optimal ranking and the asymptotic guarantees. We also provide several illustrations, such as the computational advantage for procedure 2, a counterexample of ranking by lfdr when the tests are not exchangeable, and a numerical study to illustrate the performance of the proposed procedure for the stock trading strategies via simulation.

Acknowledgments

We thank the two anonymous reviewers, Associate Editor and the Editor, whose comments have significantly improved the work and its exposition. Pallavi Basu is grateful to Profs. Sebastian Döhler, Etienne Roquain, and Daniel Yekutieli for insightful discussions and to Mr. Gunashekhar Nandiboyina for parallel computing support. All errors are our own. The views expressed in this article do not necessarily reflect those of the Federal Reserve Bank of Dallas or its staff.

Disclosure Statement

The authors report there are no competing interests to declare.

Additional information

Funding

Part of the work was done while PB was at Tel Aviv University and has received funding from the European Research Council under the European Community’s Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement n ${}^{°}$ [294519]-PSARPS. Her research is partially supported by a MATRICS grant (MTR/2022/000073) from DST-SERB, India.