Smaller p-Values via Indirect Information

Abstract

This article develops p-values for evaluating means of normal populations that make use of indirect or prior information. A p-value of this type is based on a biased frequentist hypothesis test that has optimal average power with respect to a probability distribution that encodes indirect information about the mean parameter, resulting in a smaller p-value if the indirect information is accurate. In a variety of multiparameter settings, we show how to adaptively estimate the indirect information for each mean parameter while still maintaining uniformity of the p-values under their null hypotheses. This is done using a linking model through which indirect information about the mean of one population may be obtained from the data of other populations. Importantly, the linking model does not need to be correct to maintain the uniformity of the p-values under their null hypotheses. This methodology is illustrated in several data analysis scenarios, including small area inference, spatially arranged populations, interactions in linear regression, and generalized linear models. Supplementary materials for this article are available online.

KEYWORDS:

• Empirical Bayes
• Frequentist
• Hierarchical model
• Hypothesis test
• Multiple testing
• Small area estimation

Supplementary Materials

Replication materials, additional numerical examples, and software to compute FAB p-values for parameters in linear and generalized linear models are available at https://pdhoff.github.io/FABInference.