Win-probabilities for comparing two binary outcomes

This article considers the problem of choosing between two treatments that have binary outcomes with unknown success probabilities p₁ and p₂. The choice is based upon the information provided by two observations X₁ ～ B(n₁, p₁) and X₂ ～ B(n₂, p₂) from independent binomial distributions. Standard approaches to this problem utilize basic statistical inference methodologies such as hypothesis tests and confidence intervals for the difference p₁ ? p₂ of the success probabilities. However, in this article the analysis of win-probabilities is considered. If X*₁ represents a potential future observation from Treatment 1 while X*₂ represents a potential future observation from Treatment 2, win-probabilities are defined in terms of the comparisons of X*₁ and X*₂. These win-probabilities provide a direct assessment of the relative advantages and disadvantages of choosing either treatment for one future application, and their interpretation can be combined with other factors such as costs, side-effects, and the availabilities of the two treatments. In this article, it is shown how confidence intervals for the win-probabilities can be constructed, and examples of their use are provided. Computer code for the implementation of this new methodology is available from the authors.