Nonparametric Predictive Inference for Reproducibility of Basic Nonparametric Tests

Abstract

Reproducibility of tests is an important characteristic of the practical relevance of test outcomes. Recently, there has been substantial interest in the reproducibility probability (RP), where not only its estimation but also its actual definition and interpretation are not uniquely determined in the classical frequentist statistics framework. Nonparametric predictive inference (NPI) is a frequentist statistics approach that makes few assumptions, enabled by the use of lower and upper probabilities to quantify uncertainty, and that explicitly focuses on future observations. The explicitly predictive nature of NPI provides a natural formulation for inferences on RP. In this article, we introduce the NPI approach to RP for some basic nonparametric tests.

AMS Subject Classification:

• 60A99
• 62G99
• 62P30

Keywords:

• Lower and upper probability
• Nonparametric predictive inference
• Rank-sum test
• Reproducibility probability
• Sign test
• Signed-rank test

Acknowledgments

We are grateful to two anonymous reviewers who supported our work enthusiastically and provided excellent suggestions to improve the presentation.