Accounting for Pilot Study Uncertainty in Sample Size Determination of Randomized Controlled Trials

Abstract

A confirmatory randomized controlled trial can be severely underpowered if the variability and/or effect size estimated from a small-scale pilot study is not taken into account in the sample size calculation. This article reviews and summarizes four existing methods and proposes five new approaches, namely a tolerance probability (TP) based approach and four expected power (EP) based approaches. While the TP-based approach takes into account only sampling variability, the four EP-based approaches incorporate both sampling variability and effect size estimates from external pilot studies. Formulas for power and sample size determination of the proposed approaches are first derived for normally distributed response variables and then extended to binary and time-to-event outcome variables. Simulations are conducted to compare the performance of the proposed methods with two existing methods in terms of sample size inflation and total costs (assuming a fixed cost ratio per subject between external pilot and confirmatory trials). The proposed methods are illustrated in two examples, one with binary endpoint and another with time-to-event endpoint, in terms of study sample size and power. Finally, some recommendations are made based on the degree of uncertainty of sampling variance and effect size and the performance of the methods.

Keywords:

• Effect size
• Expected power
• Minimum total sample size
• Pilot studies
• Sampling variability
• Total cost

Supplementary Materials

All computational methods are implemented with R codes which are provided as supplementary materials.

Acknowledgments

The authors would like to thank the two anonymous referees and the associate editor for their constructive comments that help improve the precision and accuracy of the materials presented in this article.