Bayes Factors Based on p-Values and Sets of Priors With Restricted Strength

Abstract

This article focuses on the minimum Bayes factor compatible with a p-value, considering a set of priors with restricted strength. The resulting minimum Bayes factor depends on both the strength of the set of priors and the sample size. The results can be used to interpret the evidence for/against the hypothesis provided by a p-value in a way that accounts for the strength of the priors and the sample size. In particular, the results suggest further lowering the p-value cutoff for “statistical significance.”

Keywords:

• Bayes factor
• Posterior probability
• p-Value

Acknowledgments

Thanks to the editor, associate editor, and referees for helpful comments and suggestions. Any errors are mine.

Notes

1 Recall the following standard arguments. Consider a normal prior for θ with mean m and variance s². Then, by standard Bayesian arguments, the posterior for θ based on a virtual sample of M iid observations would have mean $\frac{\frac{M}{{\sigma }^2}{\widehat{\theta }}_M+\frac{1}{s^2}m}{\frac{M}{{\sigma }^2}+\frac{1}{s^2}}$ and variance $\frac{1}{\frac{M}{{\sigma }^2}+\frac{1}{s^2}}=\frac{{\sigma }^2}{M+\frac{{\sigma }^2}{s^2}}=\frac{{\sigma }^2}{M(1+\frac{{\sigma }^2}{s^2}\frac{1}{M})}$, where ${\widehat{\theta }}_M$ is the virtual sample average. If $\frac{{\sigma }^2}{s^2}\frac{1}{M}$ is small, which can arise when s² is large (a weak “initial” prior) and/or when M is large (a large virtual sample), the posterior for θ is normal with variance approximately $\frac{{\sigma }^2}{M}$. Using this “posterior” from a virtual sample as the prior for the “actual” analysis would therefore result in a normal prior with variance $\frac{{\sigma }^2}{M}$ where M is the sample size of the virtual sample.

2 The way the Held and Ott (2016) minBF depends on N is qualitatively different from the current article. As $N\to \infty$, the Held and Ott (2016) minBF approaches the finite Sellke, Bayarri, and Berger (2001) minBF. In the current article, as $N\to \infty$ (for fixed strength of priors), the minBF approaches $\infty$.

3 ${\mathcal{D}}_K$ contains exactly those normal priors with precision no greater than $2\pi K^2=2\pi \frac{N}{{\sigma }^2}{\kappa }^2$. Therefore, the largest M such that ${\mathcal{N}}_{\frac{M}{{\sigma }^2}}\subseteq {\mathcal{D}}_K$ is $\left(2\pi \frac{N}{{\sigma }^2}{\kappa }^2\right){\sigma }^2=2\pi N{\kappa }^2$.