Exact Power of Conditional and Unconditional Tests: Going beyond the 2 × 2 Contingency Table

Abstract

The controversial question of whether one should condition on both margins of a contingency table for exact inference is examined from a fresh, computational perspective. The conditional test is believed to be less powerful than the unconditional test. However, in all previous work the actual hard evidence for this alleged power loss has always been provided by the single 2 × 2 table. In this setting the discreteness of the test statistic confounds the issue since the loss in power is offset by a corresponding reduction in Type 1 error. Although one could overcome discreteness through an auxiliary randomization experiment, this would not resolve the controversy, because such post hoc randomization is unacceptable to the practicing statistician. We overcome the discreteness more naturally by extending the power computations to 2 × 3 contingency tables. In doing so we find that the power advantage of the unconditional test rapidly vanishes. The article also discusses computational difficulties we would encounter if we attempted to extend the unconditional test beyond the 2 × 2 table.

Key Words:

• Ancillarity
• Barnard's test
• Fisher's exact test
• p values