Sample Size Calculation for Simulation-Based Multiple-Testing Procedures

ABSTRACT

In this article, we present a simple method to calculate sample size and power for a simulation-based multiple testing procedure which gives a sharper critical value than the standard Bonferroni method. The method is especially useful when several highly correlated test statistics are involved in a multiple-testing procedure. The formula for sample size calculation will be useful in designing clinical trials with multiple endpoints or correlated outcomes. We illustrate our method with a quality-of-life study for patients with early stage prostate cancer. Our method can also be used for comparing multiple independent groups.

Key Words:

• Bonferroni
• Global test
• Multiple endpoints
• Multiple testing
• Sample size
• Simulation

ACKNOWLEDGMENT

The authors thank the editor and associate editor for the fair review process and valuable comments.

Notes

Note: L and α denote the number of tests and overall Type I error, respectively. Common correlation ρ is assumed among test statistics except for “ρ = 0.6− − 0.1;,” where “ρ = 0.6− − 0.1; ” denotes the case of varying correlations with ρ _ll′ = 0.7 − l/10 for l = 1, …, 6 and l′ = l + 1, …, 7. The results are based on M = 10,000 random vectors.

Note: Three numbers in parenthesis correspond to the estimated Type I errors with balanced group size of 50, 30, and 10, respectively. Common correlation ρ is assumed among test statistics except for “ρ = 0.6− − 0.1;,” where “ρ = 0.6− − 0.1; ” denotes the case of varying correlations with ρ _ll′ = 0.7 − l/10 for l = 1, …, 6 and l′ = l + 1, …, 7. We used 5000 simulations.

Note: Each entry denotes sample size per treatment group required to achieve the specified power and to detect the effect size of δ. “Multiple” and “Global” represent multiple testing and O'Brien's global testing, respectively. Number in parenthesis is the empirical power computed from 5000 simulation runs.

Note: Each entry denotes sample size per treatment group required to achieve the specified power and to detect the effect size δ. “ρ = 0.6− − 0.1; ” denotes the case of varying correlations with ρ _ll′ = 0.7 − l/10 for l = 1, …, 6, l′ = l + 1, …, 7; “δ = 0.2 − 0.4” represents the case of varying effect sizes with δ _l  = 0.2 for l = 1, 2, 3, 4 and δ _l  = 0.4 for l = 5, 6, 7; and “ρ = 0.5^block” denotes a block diagonal correlation matrix with common ρ = 0.5 within block of sizes 4 and 3. Number in parenthesis is the empirical power computed from 5000 simulation runs.