Abstract
Redesigning previously conducted clinical trials is a useful tool for better determining the impact of an alternative design. Shuster and Chang suggest the redesign of clinical trials that do not use sequential monitoring by imposing a sequential monitoring structure and determining what would have happened had such a structure been employed. However, early stopping decisions are complex and involve are intimate understanding of ethical, logistical, and statistical issues. “Second-guessing” a data and safety monitoring board is dangerous without this intimate understanding.
Redesigning an old study using a new design and figuring out what would have happened is always fun and interesting for a design researcher. I give such problems to students for homework, and I often have a section called “Redesigning a Clinical Trial” in my new design papers. Among other things, Professors Shuster and Chang suggest redesigning a clinical trial that did not impose sequential monitoring to figure out what would have happened had group sequential monitoring been employed. Such an analysis has obvious ethical consequences, as the clinical trial volunteers might have been randomized to an inferior treatment, when the trial had already demonstrated efficacy had an interim inspection been planned. In my mind, this is precisely the type of important contribution that design researchers can make; such arguments can be influential in convincing skeptics that complicated or new designs are indeed worth considering for future studies.
But now let me speak as a member of numerous data and safety monitoring committees (DSMCs). The authors state,
Interest in the ability to second-guess researchers and data and safety monitoring committees has increased in recent years …. The ability to second-guess after the fact will put these committees on notice, since their interim decisions can be scrutinized once the study is published.
A DSMC is charged with making some of the most difficult decisions during the course of a clinical trial; among these are (1) whether to impose a sequential monitoring plan during the clinical trial, and (2) whether to stop early if the boundary is crossed. Often times the deliberation on these two issues is heated, and involves many factors inherent to the specific study. With regard to decision (1), the authors state,
There may be valid reasons why investigators may be unwilling to adopt a sequential plan for their study: it may be impractical, patient outcome results may come in too late to be of any use for interim analysis, the investigators may distrust proportional hazards in a survival analysis setting, the consequences of good vs. poor results may not affect the health of present and future participants, or the study may be too small to consider sequential designs.
I could add some more reasons, but this is a sufficient assortment. With regard to decision (2), there may be more gained by continuing a trial, in terms of accruing additional safety information, information on important secondary outcomes, gaining additional precision (as the authors mention), etc., than would be gained by stopping early, even if the boundary is crossed. Such a cost-benefit analysis, deeply weighing the risk to patients, is exactly the duty with which a DSMC is charged. Therefore, “second-guessing” a DSMC requires an intimate knowledge of all the issues involved in monitoring that specific clinical trial and of the detailed deliberations of the DSMC, not just a mathematical calculation that a boundary would have been crossed if the design had been changed.
My conclusion for DSMC members in the “age of second-guessing”? Make sure you have liability insurance.
ACKNOWLEDGMENT
The author is supported by Grant DMS-0539100 from the National Science Foundation.
Notes
Recommended by N. Mukhopadhyay