Improved p-Value Tests for Comparing Two Independent Binomial Proportions

Abstract

In the comparison of two independent binomial proportions for small or moderate sample sizes, Fisher's exact test can be conservative in the sense of its actual significance level (or size) being much less than the nominal level. Boschloo (1970) proposed a procedure, which raises the significance level used for the p-value test derived from Fisher's exact test, such that the size of the p-value test is closer to and below the nominal level. Röhmel and Mansmann (1999) used the unconditional approach, called the supremum procedure, to a valid p-value to obtain a modified p-value test, and showed that the modified p-value test is always better than the original one. In this article, we show that Röhmel and Mansmann's procedure is equivalent to Boschloo's procedure. The supremum procedure will then be applied to some well-known p-value procedures appeared in the literature. Numerical studies including two real cases are given to illustrate the advantages of the proposed modified p-value tests. Numerical results also show that the sizes of modified p-value tests are closer to nominal levels than those of the original p-value tests for many cases, especially in case of unbalanced sample sizes.

Keywords:

• Binomial proportions
• Boschloo's test
• Confidence interval p-value
• Fisher's p-value
• Mid p-value
• p-value test

Mathematics Subject Classification:

• 62F03
• 62P10
• 62P25

Acknowledgments

The authors gratefully acknowledge Professor N. Balakrishnan for his very helpful suggestions on an earlier version of this article. The authors also thank the Associate Editor and the referee for their many valuable comments, which substantially improved the article.

Notes

Source: The experimental result in Fries et al. (1993).

Source: Reprinted from Irwin (1935).

Note: For each pair of p-value tests, if the modified p-value test has size closer to α, the superscript of its size is marked by a “○”. If the size is closest to and not greater than a given α level, then it is displayed by boldface.

Note: For each pair of p-value tests, if the modified p-value test has size closer to α, the superscript of its size is marked by a “○”. If the size is closest to and not greater than a given α level, then it is displayed by boldface.