Small-sample power of uncorrected and satterthwaite corrected t tests for comparing binomial proportions

Abstract

When testing the equality of proportions from two independent binomial populations, one may reject the arguments for conditioning all testing and inference on the observed data and choose to employ the exact unconditional test (Suissa and Shuster, 1985), or the t test approximation (D'agostino, Chase, and Belanger, 1988). The achieved size of the t test with a pooled variance has been investigated previously for small sample sizes, but the power has not. Exact calculations, computed via enumeration, of the power of the t test with pooled variance and with Satterthwaite's variance approximation are reported here. The pooled variance approach was superior in both null and nonnull cases. For very small sample sizes, the tests achieved a power of at least .8 only when differences between proportions were at least .45. The calculations allowed demonstrating the accuracy of noncentral F integrals as approximations of empirical power, at least when power is acceptably high. The pooled variance approach provides an acceptably accurate and convenient approximation to the power of the t test approximation of the unconditional test. A similar approach also appears to work well for approximating power of the exact unconditional test.

Keywords:

• noncentral
• two-by-two table
• Fisher's Exact Test
• conditional test
• unconditional test