How the Maximal Evidence of P-Values Against Point Null Hypotheses Depends on Sample Size

ABSTRACT

Minimum Bayes factors are commonly used to transform two-sided p-values to lower bounds on the posterior probability of the null hypothesis. Several proposals exist in the literature, but none of them depends on the sample size. However, the evidence of a p-value against a point null hypothesis is known to depend on the sample size. In this article, we consider p-values in the linear model and propose new minimum Bayes factors that depend on sample size and converge to existing bounds as the sample size goes to infinity. It turns out that the maximal evidence of an exact two-sided p-value increases with decreasing sample size. The effect of adjusting minimum Bayes factors for sample size is shown in two applications.

KEYWORDS:

• Evidence
• F-Test
• Minimum Bayes factor
• p-value
• Sample size
• t-Test

Acknowledgments

We thank an Associate Editor and several referees for numerous comments that improved the presentation of the results in this article.

Funding

This work was supported by the Swiss National Science Foundation [project #159715].