Time-trend impact on treatment estimation in two-arm clinical trials with a binary outcome and Bayesian response adaptive randomization

ABSTRACT

Clinical trial design and analysis often assume study population homogeneity, although patient baseline profile and standard of care may evolve over time, especially in trials with long recruitment periods. The time-trend phenomenon can affect the treatment estimation and the operating characteristics of trials with Bayesian response adaptive randomization (BRAR). The mechanism of time-trend impact on BRAR is increasingly being studied but some aspects remain unclear. The goal of this research is to quantify the bias in treatment effect estimation due to the use of BRAR in the presence of time-trend. In addition, simulations are conducted to compare the performance of three commonly used BRAR algorithms under different time-trend patterns with and without early stopping rules. The results demonstrate that using these BRAR methods in a two-arm trial with time-trend may cause type I error inflation and treatment effect estimation bias. The magnitude and direction of the bias are affected by the parameters of the BRAR algorithm and the time-trend pattern.

KEYWORDS:

• Bayesian response adaptive randomization
• time-trend
• bias
• adaptive allocation
• clinical trial

Acknowledgments

The authors thank the anonymous reviewers for their careful review and great comments for this manuscript.

Proof 2

Scenario 1. Without time-trend effect, the average response rate does not change over time for either arm:

$\begin{array}{rl}& p_{1,1}=p_{1,2}=p_{1,3}=p_{1,4}=........=p_{1,k-1}=p_{1,k}={\tilde{p}}_1\\ & p_{2,1}=p_{2,2}=p_{2,3}=p_{2,4}=........=p_{2,k-1}=p_{2,k}={\tilde{p}}_2\end{array}$

The response rate difference between two treatment arms up to stage k is

$\begin{array}{rl}& {\mathrm{\Delta }}_k=E(p_{1,k})-E(p_{2,k})\\ & =\frac{\sum _{s=1}^k{p}_{1,s}\times n_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{s=1}^k{p}_{2,s}\times n_{2,s}}{\sum _{s=1}^k{n}_{2,s}}=\frac{{\tilde{p}}_1\sum _{s=1}^k{n}_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{{\tilde{p}}_2\sum _{s=1}^k{n}_{2,s}}{\sum _{s=1}^k{n}_{2,s}}={\tilde{p}}_1-{\tilde{p}}_2\end{array}$

Scenario 2. With time-trend effect for both arms, the average response rate changes across interim stages by ${\tau }_j$. $p_{j,k}={\tilde{p}}_j+{\tau }_{j}k$ $j=1,2.$ In this proof, we assume ${\tau }_1={\tau }_2$.

The expected cumulative response rate difference between two treatment arms up to stage k is

$\begin{array}{rl}& {\mathrm{\Delta }}_k=E(p_{1,k})-E(p_{2,k})=\frac{\sum _{s=1}^k{p}_{1,s}\times n_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{s=1}^k{p}_{2,s}\times n_{2,s}}{\sum _{s=1}^k{n}_{2,s}}=\frac{\sum _{s=1}^k({\tilde{p}}_1+s{\tau }_1)\times n_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{s=1}^k({\tilde{p}}_2+s{\tau }_2)\times n_{2,s}}{\sum _{s=1}^k{n}_{2,s}}\\ & =\frac{\sum _{s=1}^k{\tilde{p}}_1\times n_{1,s}+\sum _{s=1}^k(s{\tau }_1)\times n_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{s=1}^k{\tilde{p}}_2\times n_{2,k}+\sum _{s=1}^k(s{\tau }_2)\times n_{2,k}}{\sum _{s=1}^k{n}_{2,s}}\\ & =({\tilde{p}}_1-{\tilde{p}}_2)+\left[\frac{{\tau }_1\sum _{s=1}^{k}s{n}_{1,k}}{\sum _{s=1}^k{n}_{1,k}}-\frac{{\tau }_2\sum _{s=1}^{k}s{n}_{2,k}}{\sum _{s=1}^k{n}_{2,k}}\right]\end{array}$

$\frac{\sum _{s=1}^{k}s\times n_{1,s}}{\sum _{s=1}^k{n}_{1,s}}=\frac{n_{1,1}+2n_{1,2}+3n_{1,3}+.....+s{n}_{1,s}}{n_{1,1}+n_{1,2}+n_{1,3}+.....+n_{1,s}},\frac{\sum _{s=1}^{k}s\times n_{2,s}}{\sum _{s=1}^k{n}_{2,s}}=\frac{n_{2,1}+2n_{2,2}+3n_{2,3}+.....+s{n}_{2,s}}{n_{2,1}+n_{2,2}+n_{2,3}+.....+n_{2,s}}.$

$k=1,{\mathrm{\Delta }}_k=({\tilde{p}}_1-{\tilde{p}}_2)+\left(\frac{{\tau }_1{n}_{1,1}}{n_{1,1}}-\frac{{\tau }_2{n}_{2,1}}{n_{2,1}}\right)=({\bar{p}}_{10}-{\bar{p}}_{20})+({\tau }_1-{\tau }_2)$

$\begin{array}{rl}& k=2,{\mathrm{\Delta }}_k=({\tilde{p}}_1-{\tilde{p}}_2)+\left(\frac{{\tau }_1(n_{1,1}+2n_{1,2})}{n_{1,1}+n_{1,2}}-\frac{{\tau }_2(n_{2,1}+2n_{2,2})}{n_{2,1}+n_{2,2}}\right)=({\tilde{p}}_1-{\tilde{p}}_2)+({\tau }_1-{\tau }_2)+\left(\frac{{\tau }_1{n}_{1,2}}{n_{1,1}+n_{1,2}}-\frac{{\tau }_2{n}_{2,2}}{n_{2,1}+n_{2,2}}\right)\\ & =({\bar{p}}_{10}-{\bar{p}}_{20})+({\tau }_1-{\tau }_2)+\left(\frac{{\tau }_1}{\frac{n_{1,1}}{n_{1,2}}+1}-\frac{{\tau }_2}{\frac{n_{2,1}}{n_{2,2}}+1}\right)=({\tilde{p}}_1-{\tilde{p}}_2)+({\tau }_1-{\tau }_2)+\left(\frac{{\tau }_1}{\left(\frac{r_1}{r_2}\right)\frac{n_{2,1}}{n_{2,2}}+1}-\frac{{\tau }_2}{\frac{n_{2,1}}{n_{2,2}}+1}\right)\\ & \mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}\mathrm{e}{n}_{1,1}=r_1{n}_{2,1}\mathrm{a}\mathrm{n}\mathrm{d}{n}_{1,2}=r_2{n}_{2,2}.\end{array}$To find the general formula for defining the bias increase from (k – 1)th to kth interim stage:

${\mathrm{\Delta }}_k-{\mathrm{\Delta }}_{k-1}=\tau \left(\frac{\sum _{s=1}^{k}s\times n_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{s=1}^{k}s\times n_{2,s}}{\sum _{s=1}^k{n}_{2,s}}-\left(\frac{\sum _{s=1}^{k-1}s\times n_{1,s}}{\sum _{s=1}^{k-1}{n}_{1,s}}-\frac{\sum _{s=1}^{k-1}s\times n_{2,s}}{\sum _{s=1}^{k-1}{n}_{2,s}}\right)\right)$

$\begin{array}{rl}& \frac{\sum _{s=1}^{k}s{n}_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{k=1}^{k-1}s{n}_{1,s}}{\sum _{s=1}^{k-1}{n}_{1,s}}=\frac{\left(\sum _{s=1}^{k}s{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)-\left(\sum _{s=1}^{k-1}s{n}_{1,s}\right)\left(\sum _{s=1}^k{n}_{1,s}\right)}{\left(\sum _{k=1}^k{n}_{1,k}\right)\left(\sum _{k=1}^{k-1}{n}_{1,k}\right)}\\ & \left(\sum _{s=1}^{k}s{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)=\left(\sum _{s=1}^{k-1}s{n}_{1,s}+k{n}_{1,k}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right),\left(\sum _{s=1}^{k-1}s{n}_{1,s}\right)\left(\sum _{s=1}^k{n}_{1,s}\right)=\left(\sum _{s=1}^{k-1}s{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}+n_{1,k}\right)\\ & \left(\sum _{s=1}^{k}s{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)-\left(\sum _{s=1}^{k-1}s{n}_{1,s}\right)\left(\sum _{s=1}^k{n}_{1,s}\right)=k{n}_{1,k}\sum _{s=1}^{k-1}{n}_{1,s}-n_{1,k}\sum _{s=1}^{k-1}s{n}_{1,s}=n_{1,k}\sum _{s=1}^{k-1}s{n}_{1,s}\end{array}$Similarly, $\left(\sum _{s=1}^{k}s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)-\left(\sum _{s=1}^{k-1}s{n}_{2,s}\right)\left(\sum _{s=1}^k{n}_{2,s}\right)=n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}$. Therefore,

$\frac{\sum _{s=1}^{k}s{n}_{1,s}}{\sum _{s=1}^k{n}_{1,s}}-\frac{\sum _{k=1}^{k-1}s{n}_{1,s}}{\sum _{s=1}^{k-1}{n}_{1,s}}-\left(\frac{\sum _{s=1}^{k}s{n}_{2,s}}{\sum _{s=1}^k{n}_{2,s}}-\frac{\sum _{s=1}^{k-1}s{n}_{2,s}}{\sum _{s=1}^{k-1}{n}_{2,s}}\right)=\frac{n_{1,k}\sum _{s=1}^{k-1}s{n}_{1,s}}{\left(\sum _{s=1}^k{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)}-\frac{n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}}{\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)}$

We define the allocation ratio $r_k=\frac{n_{1,k}}{n_{2,k}},\hspace{0.17em}{n}_{1,k}=r_k{n}_{2,k}$

$\begin{array}{rl}& \hspace{0.17em}\frac{n_{1,k}\sum _{s=1}^{k-1}s{n}_{1,s}}{\left(\sum _{s=1}^k{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)}-\frac{n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}}{\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)}=\frac{r_k{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}}{\left(\sum _{s=1}^k{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)}-\frac{n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}}{\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)}\\ & =\frac{r_k{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)-n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^k{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)}{\left(\sum _{s=1}^k{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)}\end{array}$

$\begin{array}{rl}& \hspace{0.17em}{r}_k{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^{k-1}{n}_{2,s}+n_{2,k}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)-n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}+r_k{n}_{2,k}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\\ & =r_k{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)+r_k{n}_{2,k}{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)\\ & \hspace{0.17em}-n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)-r_k{n}_{2,k}{n}_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\end{array}$

$\begin{array}{rl}Q& \sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)=\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}\sum _{c=1}^{k-1}a{r}_a{n}_{2,a}{n}_{2,b}{n}_{2,c},\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\\ & =\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}\sum _{c=1}^{k-1}a{n}_{2,a}{r}_b{n}_{2,b}{r}_c{n}_{2,c}\\ & \text{there}4r_k{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)-n_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\\ & =n_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}\sum _{c=1}^{k-1}a{n}_{2,a}{n}_{2,b}{n}_{2,b}(r_k{r}_a-r_b{r}_c)>0\\ & as{r}_1<r_2<r_3<.....<r_{k-1}<r_k.\end{array}$

Similarly, we can find that

$\begin{array}{rl}& r_k{n}_{2,k}{n}_{2,k}\sum _{s=1}^{k-1}s{r}_s{n}_{2,s}\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)-r_k{n}_{2,k}{n}_{2,k}\sum _{s=1}^{k-1}s{n}_{2,s}\left(\sum _{s=1}^{k-1}{r}_s{n}_{2,s}\right)\\ & =r_k{n}_{2,k}{n}_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}a{r}_a{n}_{2,a}{n}_{2,b}-r_k{n}_{2,k}{n}_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}a{n}_{2,a}{r}_b{n}_{2,b}\\ & =r_k{n}_{2,k}{n}_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}a{n}_{2,a}{n}_{2,b}(r_a-r_b)=0\end{array}$

Therefore, the bias increase from (k – 1)th to kth interim stage is

${\mathrm{\Delta }}_k-{\mathrm{\Delta }}_{k-1}=\tau \left(\frac{n_{1,k}\sum _{s=1}^{k-1}k{n}_{1,s}}{\left(\sum _{s=1}^k{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)}-\frac{n_{2,k}\sum _{s=1}^{k-1}k{n}_{2,s}}{\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{k=1}^{k-1}{n}_{2,s}\right)}\right)=\frac{\tau n_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}\sum _{c=1}^{k-1}k{n}_{2,a}{n}_{2,b}{n}_{2,c}(r_a{r}_k-r_b{r}_c)}{\left(\sum _{s=1}^k{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)}$

which is positive. Therefore, with equal trend, the bias increases with increase in the randomization allocation ratio across stages.

Scenario 3. If the trend effects are different between two arms:${\tau }_1\ne {\tau }_2$,

$\begin{array}{rl}& {\mathrm{\Delta }}_k-{\mathrm{\Delta }}_{k-1}=\frac{{\tau }_1{n}_{1,k}\sum _{s=1}^{k-1}k{n}_{1,s}}{\left(\sum _{s=1}^k{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)}-\frac{{\tau }_2{n}_{2,k}\sum _{s=1}^{k-1}k{n}_{2,s}}{\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{k=1}^{k-1}{n}_{2,s}\right)}\\ & =\frac{n_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}\sum _{c=1}^{k-1}a{n}_{2,a}{n}_{2,b}{n}_{2,b}({\tau }_1{r}_k{r}_a-{\tau }_2{r}_b{r}_c)+r_k{n}_{2,k}{n}_{2,k}\sum _{a=1}^{k-1}\sum _{b=1}^{k-1}a{n}_{2,a}{n}_{2,b}({\tau }_1{r}_a-{\tau }_2{r}_b)}{\left(\sum _{s=1}^k{n}_{1,s}\right)\left(\sum _{s=1}^{k-1}{n}_{1,s}\right)\left(\sum _{s=1}^k{n}_{2,s}\right)\left(\sum _{s=1}^{k-1}{n}_{2,s}\right)}\end{array}$

From this equation, we observe that the treatment effect estimation in adaptive design can be biased when there is time-trend effect. The bias comes from two sources: (1) change in randomization allocation across stages $r_1,r_2,r_3\dots .,r_K,r_j\ne r_k,j\ne k|j,kϵ\{1,2,3,\dots ,\mathrm{K}\};$ and (2) the difference in the magnitude of time-trend effect (${\alpha }_1\ne {\alpha }_2$). If either condition is not met, the treatment effect estimation will be biased.

WinBUGS Code

model{

for (i in 1:N){

out[i]~dbern(p[i])

ypred[i]~dbern(p[i])

logit(p[i])<-b0+btrt*arm[i]+bt*group[i]

}

b0~dnorm(0.0,tau.b0)

tau.b0~dgamma(0.2,0.4)

btrt~dnorm(0.0,tau.btrt)

tau.btrt~dgamma(0.2,0.4)

bt~dnorm(0.0,tau.bt)

tau.bt<-pow(sd.bt,-2)

sd.bt~dunif(0,10)

OR<-exp(btrt)

ppbest<-1-step(1-OR)

}

Notation:

i: ith subject.

out: binary response outcome.

p: response rate.

arm: treatment.

group: the interim stage for subject i.

btrt(β_trt): treatment effect estimate.

bt (β): beta coefficient for time-trend effect.

tau.btrt (ψ_βtrt): precision of the parameter estimate for treatment effect.

tau.bt (ψ_β): precision of the parameter estimate for time-trend effect.

sd.bt (σ_β): standard deviation of the parameter estimate for time-trend effect.

ppbest: probability that the odds ratio of one treatment versus the other is greater than 1.

Additional information

Funding

This research is partly supported by the NIH/NINDS grants [U01NS0059041] (NETT) and [U01NS087748] (StrokeNet).