A Two-Stage Decision Making Approach for Safety Studies

ABSTRACT

For safety studies, two types of hypothesis testing are often considered: detecting a safety signal and ruling out a safety concern. Under the detecting framework, statistical non-significance is often confused with the conclusion that there is no safety concern. Such a conclusion, in the presence of low study power or large variability, is problematic. To overcome the interpretation issue with non-significant results from a detecting hypothesis, we propose a Two-Stage Decision-Making (TSDM) approach for safety studies. It is basically a ruling-out design allowing an interim analysis that applies both detecting and ruling-out criteria at the interim and final stages with a pre-specified alpha spending function. The proposed TSDM approach incorporates both detecting a safety signal and ruling out safety concerns into a single study design to increase the probability of making a definite decision. It is based on the ruling-out framework that utilizes both directions of the confidence interval to make a decision for ruling out unacceptable risk or detecting safety signal at each analysis stage. We assess the proposed TSDM approach by investigating properties such as operational type I error rate, overall study power based on analytical approximations, overall probability of making a decision, and required sample sizes. We conduct Monte Carlo simulations to evaluate such properties regarding various outcome types of confidence intervals and summarize the statistical interpretations and the implications on study design.

KEYWORDS:

• Detecting a safety signal
• ruling out a safety concern
• two-stage decision-making (TSDM) approach
• operational type I error
• study power
• confidence interval
• Monte Carlo simulations
• group sequential design
• alpha spending function

Acknowledgments

The authors would like to thank U.S. Food and Drug Administration (FDA) employees Drs. David Graham, John Yap, Mark Levenson and Stella Grosser, and former FDA employees Drs. Kunthel By, Thomas Ly and Easter Zhao for their constructive inputs and advices during the preparation of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Huang’s work is partly supported through a grant from the U.S. Food and Drug Administration, and by an appointment to the Research Participation Program at the Center for Drug Evaluation and Research administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and the U.S. Food and Drug Administration.