A Decision-Theoretic Formulation of Fisher’s Approach to Testing

Abstract

In Fisher’s interpretation of statistical testing, a test is seen as a ‘screening’ procedure; one either reports some scientific findings, or alternatively gives no firm conclusions. These choices differ fundamentally from hypothesis testing, in the style of Neyman and Pearson, which does not consider a noncommittal response; tests are developed as choices between two complementary hypotheses, typically labeled ‘null’ and ‘alternative.’ The same choices are presented in typical Bayesian tests, where Bayes Factors are used to judge the relative support for a null or alternative model. In this article, we use decision theory to show that Bayesian tests can also describe Fisher-style ‘screening’ procedures, and that such approaches lead directly to Bayesian analogs of the Wald test and two-sided p-value, and to Bayesian tests with frequentist properties that can be determined easily and accurately. In contrast to hypothesis testing, these ‘screening’ decisions do not exhibit the Lindley/Jeffreys paradox, that divides frequentists and Bayesians.

Keywords:  :

• Bayesian inference
• Decision theory
• Testing