A causal modelling framework for reference-based imputation and tipping point analysis in clinical trials with quantitative outcome

Figures & data

Figure 1. Notation illustrated. Lines indicate mean potential outcomes under three potential treatment scenarios. Circles indicate observable outcomes for a participant who discontinues treatment at visit $s$.

Table 1. Imputation distribution of ${\mathit{Y}}_{>t}(t)$ for $t<t_{max}$ given randomisation $Z=a$, past ${\mathit{Y}}_{\le t}$ and treatment discontinuation visit $D=t$, under various reference-based imputation methods with control arm as reference (Carpenter et al. 2013) and under the causal model. $C_t$ is a ‘carry-forward’ $(t_{max}-t)\times (t+1)$ matrix containing $t$ columns of zeroes and a final column of ones, so that ${\mathit{C}}_t{\mathit{\mu }}_{\le t}(t)$ is a column vector containing $t_{max}-t$ copies of ${\mu }_t(t)$.

	Imputation distribution
Method	Mean	Variance
Reference-based imputation methods
MAR	${\beta }_t(t_{max})\left\{{\mathit{Y}}_{\le t}-{\mathit{\mu }}_{\le t}(t)\right\}+{\mathit{\mu }}_{>t}(t_{max})$	${\mathbf{\Omega }}_t(t_{max})$
LMCF	${\beta }_t(t_{max})\left\{{\mathit{Y}}_{\le t}-{\mathit{\mu }}_{\le t}(t)\right\}+C_t{\mathit{\mu }}_{\le t}(t)$	${\mathbf{\Omega }}_t(t_{max})$
J2R	${\beta }_t(0)\left\{{\mathit{Y}}_{\le t}-{\mathit{\mu }}_{\le t}(t)\right\}+{\mathit{\mu }}_{>t}(0)$	${\mathbf{\Omega }}_t(0)$
CIR	${\beta }_t(0)\left\{{\mathit{Y}}_{\le t}-{\mathit{\mu }}_{\le t}(t)\right\}+C_t\left\{{\mathit{\mu }}_{\le t}(t)-{\mathit{\mu }}_{\le t}(0)\right\}+{\mathit{\mu }}_{>t}(0)$	${\mathbf{\Omega }}_t(0)$
CR*	${\beta }_t(0)\left\{{\mathit{Y}}_{\le t}-{\mathit{\mu }}_{\le t}(t)\right\}+{\beta }_t(0)\left\{{\mathit{\mu }}_{\le t}(t)-{\mathit{\mu }}_{\le t}(0)\right\}+{\mathit{\mu }}_{>t}(0)$	${\mathbf{\Omega }}_t(0)$
Causal model
	${\beta }_t(t)\left\{{\mathit{Y}}_{\le t}-{\mathit{\mu }}_{\le t}(t)\right\}+{\mathit{K}}_t\left\{{\mathit{\mu }}_{\le t}(t)-{\mathit{\mu }}_{\le t}(0)\right\}+{\mathit{\mu }}_{>t}(0)$	${\mathbf{\Omega }}_t(t)$

* The CR mean is more simply written ${\beta }_t(0)\left\{Y_{\le t}-{\mu }_{\le t}(0)\right\}+{\mu }_{>t}(0)$, but the expression given here facilitates comparison with the other methods.

Table 2. Simulation study with $D=1$ or $2$: estimates of treatment effect at visit 2 using complete data, causal model imputation and RBI imputation. ${\beta }_1(0)=(0,0.5{)}^{\prime }$ in all cases. ${\beta }_1(2)\ne {\beta }_1(0)$ means ${\beta }_1(2)=(-0.12,0.74{)}^{\prime }$.

	Data generating mechanisms for observed data
	MCAR		MAR
	${\beta }_1(2)={\beta }_1(0)$	${\beta }_1(2)\ne {\beta }_1(0)$	${\beta }_1(2)={\beta }_1(0)$	${\beta }_1(2)\ne {\beta }_1(0)$
A. Complete data generated with:
${\beta }_1(1)={\beta }_1(0)$
$k=0.00$	0.99	0.99	1.00	0.70
$k=0.50$	1.24	1.24	1.25	0.95
$k=0.74$	1.36	1.36	1.37	1.07
$k=1.00$	1.49	1.49	1.50	1.20
${\beta }_1(1)={\beta }_1(2)$
$k=0.00$	0.99	1.00	1.00	1.00
$k=0.50$	1.24	1.25	1.25	1.25
$k=0.74$	1.36	1.37	1.37	1.37
$k=1.00$	1.49	1.50	1.50	1.50
B. Causal model imputation with assumed ${\tilde{\beta }}_1(1)$ and $\tilde{k}$
${\tilde{\beta }}_1(1)={\beta }_1(0)$
$\tilde{k}=0.00$	1.00	1.00	1.00	0.71
$\tilde{k}=0.50$	1.24	1.25	1.25	0.96
$\tilde{k}=0.74$	1.36	1.37	1.37	1.08
$\tilde{k}=1.00$	1.49	1.50	1.50	1.21
${\tilde{\beta }}_1(1)={\beta }_1(2)$
$\tilde{k}=0.00$	1.00	1.00	1.00	1.00
$\tilde{k}=0.50$	1.24	1.25	1.25	1.25
$\tilde{k}=0.74$	1.36	1.37	1.37	1.37
$\tilde{k}=1.00$	1.49	1.50	1.50	1.50
C. RBI imputation with assumed variance-covariance and method
Variance-covariance matrix from control arm
J2R	1.00	1.00	1.00	0.71
CR	1.24	1.25	1.25	0.96
CIR	1.49	1.50	1.50	1.21
Variance-covariance matrix from active arm
J2R	1.00	1.00	1.00	1.00
CR	1.24	1.37	1.25	1.38
CIR	1.49	1.50	1.50	1.50

Note: Maximum Monte Carlo standard error $<0.01$

Table 3. Simulation study: average standard error (empirical standard error) for the treatment difference at the final visit using complete data, causal model imputation and RBI imputation. ${\beta }_1(0)$ and ${\beta }_1(2)$ as in Table 2.

	Data generating mechanisms for observed data
	MCAR		MAR
	${\beta }_1(2)={\beta }_1(0)$	${\beta }_1(2)\ne {\beta }_1(0)$	${\beta }_1(2)={\beta }_1(0)$	${\beta }_1(2)\ne {\beta }_1(0)$
A. Complete data generated with:
${\beta }_1(1)={\beta }_1(0)$
$k=0.00$	0.276 (0.273)	0.318 (0.311)	0.260 (0.255)	0.295 (0.288)
$k=0.50$	0.272 (0.269)	0.315 (0.308)	0.261 (0.256)	0.298 (0.291)
$k=0.74$	0.302 (0.206)	0.342 (0.247)	0.311 (0.219)	0.351 (0.258)
$k=1.00$	0.270 (0.266)	0.313 (0.306)	0.262 (0.257)	0.301 (0.295)
${\beta }_1(1)={\beta }_1(2)$
$k=0.00$	0.276 (0.273)	0.318 (0.316)	0.260 (0.255)	0.292 (0.288)
$k=0.50$	0.272 (0.269)	0.315 (0.312)	0.261 (0.256)	0.296 (0.291)
$k=0.74$	0.271 (0.268)	0.314 (0.311)	0.262 (0.256)	0.298 (0.294)
$k=1.00$	0.270 (0.266)	0.313 (0.310)	0.262 (0.257)	0.300 (0.296)
B. Causal model imputation with assumed ${\tilde{\beta }}_1(1)$ and $\tilde{k}$
${\tilde{\beta }}_1(1)={\beta }_1(0)$
$\tilde{k}=0.00$	0.310 (0.168)	0.337 (0.189)	0.305 (0.171)	0.328 (0.181)
$\tilde{k}=0.50$	0.301 (0.190)	0.327 (0.226)	0.299 (0.189)	0.322 (0.214)
$\tilde{k}=0.74$	0.302 (0.206)	0.327 (0.249)	0.301 (0.206)	0.325 (0.237)
$\tilde{k}=1.00$	0.305 (0.226)	0.332 (0.277)	0.306 (0.227)	0.332 (0.267)
${\tilde{\beta }}_1(1)={\beta }_1(2)$
$\tilde{k}=0.00$	0.310 (0.168)	0.359 (0.187)	0.315 (0.188)	0.359 (0.206)
$\tilde{k}=0.50$	0.301 (0.190)	0.344 (0.224)	0.310 (0.204)	0.350 (0.236)
$\tilde{k}=0.74$	0.302 (0.206)	0.342 (0.247)	0.311 (0.219)	0.351 (0.258)
$\tilde{k}=1.00$	0.305 (0.226)	0.345 (0.275)	0.316 (0.239)	0.356 (0.285)
C. RBI imputation with assumed variance-covariance matrix and method
Variance-covariance matrix from control arm
J2R	0.310 (0.168)	0.337 (0.189)	0.305 (0.171)	0.328 (0.181)
CR	0.303 (0.192)	0.328 (0.229)	0.306 (0.200)	0.331 (0.226)
CIR	0.305 (0.226)	0.332 (0.277)	0.306 (0.227)	0.332 (0.267)
Variance-covariance matrix from active arm
J2R	0.310 (0.168)	0.359 (0.187)	0.315 (0.188)	0.359 (0.206)
CR	0.305 (0.192)	0.344 (0.249)	0.332 (0.236)	0.370 (0.283)
CIR	0.305 (0.226)	0.345 (0.275)	0.316 (0.239)	0.356 (0.285)

Note: Maximum Monte Carlo standard error $<0.0005$

Figure 2. HAMD17 and pain score data sets: observed mean profile according to the visit at which treatment was discontinued in the active and placebo arms.

Note: In the pain score data, four subjects in the active arm and two subjects in the placebo arm did not complete any post-baseline visit and were excluded from analysis.

Table 4. HAMD17 and pain score data: estimated treatment effect at the final visit using standard multiple imputation with 100 imputations, mixed model for repeated measures (MMRM) and RBI methods.

Estimand &	HAMD17			Pain score
Method	Estimate${ }^1$	Std. error	p-value	Estimate${ }^2$	Std. error	p-value
De jure
Standard MI	−2.62	0.99	0.01	−0.88	0.39	0.03
MMRM	−2.58	1.03	0.01	−0.88	0.39	0.03
De facto
RBI: variance-covariance matrix from the placebo arm
J2R	−2.01	1.01	0.05	−0.64	0.40	0.11
CR	−2.22	0.99	0.03	−0.75	0.39	0.06
CIR	−2.30	0.99	0.02	−0.77	0.39	0.05
RBI alternative: variance-covariance matrix from the drug arm
J2R	−1.99	1.01	0.05	−0.60	0.39	0.13
CR	−2.20	0.99	0.03	−0.71	0.39	0.07
CIR	−2.28	0.99	0.02	−0.73	0.39	0.06

${ }^1$ Monte Carlo standard error for MI methods is $\le 0.04$.

${ }^2$ Monte Carlo standard error for MI methods is $\le 0.02$.

Figure 3. HAMD17 and pain score data sets: tipping point analysis for the estimated treatment effect at the final visit using causal model (5). The model has a constant treatment effect after treatment discontinuation, equal to fraction $k_0$ of the treatment effect at treatment discontinuation. The horizontal solid and dotted lines represent the treatment estimates and their pointwise 95% CI, respectively. The vertical solid line corresponds to $k_0$ such that p-value $>0.05$ in the left-hand side of the line (tipping point).

Figure 4. HAMD17 and pain score data sets: tipping point analysis for the estimated treatment effect at the final visit using causal model (6). The model has the treatment effect decaying exponentially after treatment discontinuation, by a ratio $k_1$ for each visit. The horizontal solid and dashed lines represent the treatment estimates and their pointwise 95% CI, respectively. The tipping point is not attained in the range $0\le k_1\le 1$.

Carpenter, J. R., J. H. Roger, and M. G. Kenward. 2013. Analysis of longitudinal trials with protocol deviation: A framework for relevant, accessible assumptions, and inference via multiple imputation. Journal of Biopharmaceutical Statistics 23 (3):1352–1371. doi:10.1080/10543406.2013.834911.

 PubMedGoogle Scholar