Admissibility in Partial Conjunction Testing

ABSTRACT

Meta-analysis combines results from multiple studies aiming to increase power in finding their common effect. It would typically reject the null hypothesis of no effect if any one of the studies shows strong significance. The partial conjunction null hypothesis is rejected only when at least r of n component hypotheses are nonnull with r = 1 corresponding to a usual meta-analysis. Compared with meta-analysis, it can encourage replicable findings across studies. A by-product of it when applied to different r values is a confidence interval of r quantifying the proportion of nonnull studies. Benjamini and Heller (2008) provided a valid test for the partial conjunction null by ignoring the r − 1 smallest p-values and applying a valid meta-analysis p-value to the remaining n − r + 1 p-values. We provide sufficient and necessary conditions of admissible combined p-value for the partial conjunction hypothesis among monotone tests. Non-monotone tests always dominate monotone tests but are usually too unreasonable to be used in practice. Based on these findings, we propose a generalized form of Benjamini and Heller’s test which allows usage of various types of meta-analysis p-values, and apply our method to an example in assessing replicable benefit of new anticoagulants across subgroups of patients for stroke prevention.

KEYWORDS:

• Combined p-values
• Meta-analysis
• Power
• Replicable findings
• Subgroup analysis

Additional information

Funding

Jingshu Wang was supported in part by NIH grant R01 HG006137 (no. 2), and the Wharton Dean’s Fund. Art. B Owen was supported in part by NSF grants DMS-1407397 and DMS-1521145.