Win-probabilities for comparing two binary outcomes

ABSTRACT

This article considers the problem of choosing between two treatments that have binary outcomes with unknown success probabilities p₁ and p₂. The choice is based upon the information provided by two observations X₁ ∼ B(n₁, p₁) and X₂ ∼ B(n₂, p₂) from independent binomial distributions. Standard approaches to this problem utilize basic statistical inference methodologies such as hypothesis tests and confidence intervals for the difference p₁ − p₂ of the success probabilities. However, in this article the analysis of win-probabilities is considered. If X*₁ represents a potential future observation from Treatment 1 while X*₂ represents a potential future observation from Treatment 2, win-probabilities are defined in terms of the comparisons of X*₁ and X*₂. These win-probabilities provide a direct assessment of the relative advantages and disadvantages of choosing either treatment for one future application, and their interpretation can be combined with other factors such as costs, side-effects, and the availabilities of the two treatments. In this article, it is shown how confidence intervals for the win-probabilities can be constructed, and examples of their use are provided. Computer code for the implementation of this new methodology is available from the authors.

KEYWORDS:

• Acceptance set
• Bernoulli probability
• Binomial distribution
• Confidence interval
• Non-inferiority
• Selection
• Win-probability

MATHEMATICS SUBJECT CLASSIFICATION:

• 62F03 Hypothesis testing
• 60E05 Distributions General Theory

Acknowledgments

The authors sincerely thank the reviewer for the helpful and insightful comments and suggestions that have resulted in a much improved version of this manuscript. The authors also thank Dr. Kanda Runapongsa Saikaew and the Computer Center of Khon Kaen University, Thailand, for the dataset used in Example 3.4.

Funding

This research has been supported in part by Srinakharinwirot University (University Grant).