Discussion on “Second-Guessing Clinical Trial Designs” by Jonathan J. Shuster and Myron N. Chang

Abstract

Professors Shuster and Chang provided some optimal group sequential designs, where the optimization is done by a “grid search” method, before the trial begins. In this discussion, we suggest possible modification of the design where the design to be carried out in a stage-wise fashion, and the design for any group/stage is determined optimally based on the available data before that stage. Thus, one can modify several logistics of the design at the interim looks. This type of truly group sequential design is computationally simpler. We also point out the possible extension in terms of optimal response-adaptive group sequential designs, which can be obtained by such stage-wise optimal designing. Some points of the paper of Shuster and Chang are also mentioned that need more concrete discussions.

Keywords:

• Conditional error function
• Conditional power
• Group sequential design
• Optimal design
• Response-adaptive design
• Type I error spending function

Subject Classifications:

• 62L05
• 62K05

1. INTRODUCTION

The paper bv Professors Shuster and Chang provides some optimal group sequential designs. The optimal design is obtained at the outset, before the trial begins. They provided four-stage reference designs, as they considered this “a pragmatic maximum number of stages one might consider.” But, for theoretical presentation, we can always discuss the designs for k stages in a unified way.

In the present discussion article, the main focus is laid on the designing at the outset for all the groups versus designing in a stage-wise manner. We argue that a stage-wise designing may give more flexibility in adjusting the trial. Moreover, it might be computationally an easier task. We discuss these issues in Section 2. The article is concluded with some other relevant points in Section 3.

2. STAGE-WISE OPTIMAL DESIGNS

2.1. Conditional Designing

Shuster and Chang provided the optimal design at the outset, before the trial starts. The global optimal design for all four stages is obtained by a global optimization. But if the data are obtained instantaneously or within a reasonable time, so that a considerable proportion of data are obtained within the time frame of the trial, it is possibly a better idea to use that data information to suitably adjust the trial. This adaptive nature of the design can be best implemented by obtaining the design in a stage-wise manner.

For example, Shuster and Chang assumed a Brownian motion approximation of the process. This assumption gives the flexibility to easily use the concept of Type I error spending function (Lan and DeMets, 1983) to spend the total Type I error α among the k groups, with α _i being the Type I error at the ith group, such that the total Type I error of the group sequential procedure is α. Lan and DeMets (1983) provided some possible functional forms of α∗(t) the Type I error spending function, which is generally concave in nature, such as

With such a Type I error spending; function, the total Type I error can be made equal to α, even without having preset group lengths or a preset group number. In a similar spirit, one can as well spend the Type II error in different stages, in a group sequential manner. Thus, there is no need to find optimal designs for all the stages at the outset; one can sequentially decide.

Thus, as an alternative, I suggest finding an optimal design for the first stage only at the outset. Following the notation of the paper by Shuster and Chang, we assign some suitably chosen values of α₁, β₁, and E(θ₁) for this first stage, chosen by some optimal way. Let Y(i) be the data of the ith stage. Based on Y(1) and keeping in mind the values of α₁, β₁, and E(θ₁), one can optimally obtain α₂ = α₂ (Y(1)), β₂ = β₂(Y(1)), and E(θ₂) = E(θ₂|Y(1)), which are functions of Y(1). We continue this procedure. Finally, at the kth stage, we obtain α _k  = α _k (Y(1),…, Y(k − 1)), β _k  = β _k (Y(1),…, Y(k − 1)), and E(θ _k ) = E(θ _k |Y(1),…, Y(k − 1)) such that the eventual values α, β, and E(θ) are guaranteed. The idea is closely related to the “conditional error function” (Posch and Bauer, 1999; Proschan and Hunsberger, 1995). Here, for any ith stage, one can obtain α _i  = α _i (Y(1),…, Y(i − 1)), β _i  = β _i (Y(1),…, Y(i − 1)), and E(θ _i ) = E(θ _i |Y(1),…, Y(i − 1)) optimally, maybe in the same way Shuster and Chang suggested to minimize the inner product B∂ in their equation (3.2). But this has to be done in a group sequential way, using all the available data up to that stage. Clearly, b ₀, b ₁, and b ₂ in this stage-wise situation can be a function of the available data up to that stage, and thus can change at every occasion.

The above adaptive nature of designing provides more flexibilities for practical implementation. Several important issues in designing flexible trials are discussed by Posch et al. (2003). For example, one can readjust the Type I and Type II errors, and, most importantly, the sample size, at the interim looks. This has been done in two-stage designs in recent years. For example, there is recent research on two-stage adaptive designs that focus on second-stage sample size determination based on “conditional error function” (see Posch and Bauer, 1999; Proschan and Hunsberger, 1995). “Conditional power function” is also discussed in that context. Also, in the two-stage designs, an interim look allows midcourse sample size modification (see Proschan et al., 2003), change in sample size and inference (see Liu and Chi, 2001), and even change in the test statistic (see Lawrence, 2002). For a design with more than two stages, the above ideas can be extended, and this is certainly a worthy exercise. Some studies are available in the literature (see Bauer and Kohne, 1994; Posch and Bauer, 1999; and the references therein), but are certainly not adequate.

2.2. Optimal Response-Adaptive Group Sequential Designs

Shuster and Chang assumed a set up of approximately 50–50 allocation among the two competing treatments. In contrast, response-adaptive allocation designs are becoming popular in two-stage designs or group sequential designs. The idea is to use the first-stage data to assess which treatment is doing better, and then to modify the second-stage allocation in favor of the treatment doing better at that stage. For more than two groups, adaptation for skewing allocation is possible if the course of action, for the ith stage is suitably determined based on the available data. A group sequential optimal response-adaptive design can be obtained in this way.

Bandyopadhyay et al. (2007) provided an optimal response-adaptive two-stage design. This is an extension of the design of Atkinson and Biswas (2005), and essentially maximizes some utility at the second stage. This utility is a linear combination of the log of the determinant of the information matrix (analogous to D-optimal design) and some entropy function (which provides a “distance” from a predetermined skewed allocation to be determined from the first-stage data). Thus, the utility is a function of the first-stage data. This idea can be extended for more than two stages. For example, for the third stage, the utility will be a function of the data from the first two stages.

At the ith stage, i = 2,…, k, one can maximize the utility

to find p, the allocation probability to Treatment A for any patient in the ith stage, where I _i is the information matrix up to the patients of the ith stage and π _i−1 is any intuitive skewed allocation probability in favor of Treatment A based on the data up to the (i − 1)th stage. This π _i−1 reflects the relative performance of Treatment A over Treatment B up to the (i − 1)th stage.

3. OTHER POINTS AND CONCLUSION

There are quite a few subjective components in the formulation of the design of Shuster and Chang, which are the design parameters. I understand that such subjective issues are inevitable in the description of such designs, but a detailed study is needed to provide a proper guideline to choose these design parameters in practice.

In addition, the practical problem of optimization in a multidimensional space is always there, although not explicitly mentioned by Shuster and Chang. If the optimal design is obtained separately at every stage, the optimization problem, being conducted at different stages, becomes a simpler exercise, as the number of parameters to be decided at each stage is less than the situation where optimization for all the stages is done at the outset. When the number of parameters being less in any stage of the stage-wise design, the optimization is feasible with greater accuracy. For example, the grid search allows a finer search with a much lower number of parameters.

The presence of important covariates is an important practical scenario. But the design in the presence of covariates is a more complicated problem. A stage-wise optimal design can be possible under random covariates, and an optimal response-adaptive group sequential design can be obtained even under nonstochastic covariates.

Thus, we observe that the optimization at the beginning of each stage, based on all the accumulated data up to that point, might be a more flexible and better way of conducting group sequential trials. One can modify the Type I error, Type II error, sample size, proportion of allocation, or even the test statistic suitably at different stages. In a nutshell, interim analyses should not the only goal of group sequential trials; the flexibility obtained through “interim designing” should be a major goal of group sequential trials. After all, sequential methods are all about benefiting from using accumulated data in the course of the experiment.

Notes

Recommended by N. Mukhopadhyay