Best uses of p-values and complementary measures in medical research: Recent developments in the frequentist and Bayesian frameworks

ABSTRACT

The p-value is a classical proposal of statistical inference, dating back to the seminal contributions by Fisher, Neyman and E. Pearson. However, p-values have been frequently misunderstood and misused in practice, and medical research is not an exception. In recent years, in several statistical and applied journals, a debate erupted about the need of clear guidelines in reporting p-values, which culminated with the publication of the ASA statement in 2016. In this paper, we assess strengths and limitations of p-values and we assert that in applied research the p-value should be supplemented by other measures, such as the Bayes factor, the Bayes false discovery rate and the local Bayes false discovery rate. We also review a recent proposal by Bayarri et al. from a Bayesian perspective that has the advantage of introducing an indicator, the rejection odds, which keeps into account both pre- and post-experimental information, and could also have a straightforward frequentist interpretation. We conduct a delimited numerical study that investigates on the relation of the Bayes factor with its maximum, and of the local Bayes false discovery rate with its minimum under different distributional assumptions and parameter choices. We illustrate the concepts expressed in theory with an example in clinical oncology, namely a randomized trial on the effectiveness of a new chemotherapy for patients with AIDS and Kaposi’s sarcoma.

KEYWORDS:

• P-value
• significance testing
• Bayes factor
• Bayes false discovery rate
• local Bayes false discovery rate
• rejection ratio
• rejection odds