The Use of a Variable Representing Compliance Improves Accuracy of Estimation of the Effect of Treatment Allocation Regardless of Discontinuation in Trials with Incomplete Follow-up

Figures & data

Fig. 1 The data-generating mechanism if treatment can affect the outcome, represented according to Mohan, Pearl, and Tian (2013). Each directed line represents a causal relationship from a variable to another. Z = random assignment; X_t = whether the patient complies with the treatment at time t; RY_t = the response indicator at time t; Y_t= the real value of HAM17 in a patient at time t; Y_t* = the observed HAM17 value at time t. See text for details.

Fig. 2 The data-generating mechanism if treatment never affects outcome, represented according to Mohan, Pearl, and Tian (2013). Each directed line represents a causal relationship from a variable to another. Z = random assignment; X_t = whether the patient complies with the treatment at time t; RY_t = the response indicator at time t; Y_t= the real value of HAM17 in a patient at time t; Y_t* = the observed HAM17 value at time t. See text for details.

Table 1 Some of the parameters allowed to vary at each simulated trial and their allowed range.

Parameter	Allowed range (uniform distribution)
Sample size (per arm)	150–250
Mean drug effect between t0 and t1	0.01–0.3
Mean drug effect between t1 and t2	Mean drug effect between t0 and t1 to 0.8
Mean drug effect between t2 and t3	Mean drug effect between t1 and t2 to 1.5
% of delayed effect	0.01–0.15
Treatment discontinuation probability (per visit) for patients performing better than average	0.15–0.45
Treatment discontinuation probability (per visit) for patients performing worse than average	Rate for the ones performing better to 0.6
Post-discontinuation retention probability for patients performing worse than average	0.25–0.5
Post-discontinuation retention probability for patients performing better than average	Rate for the ones performing worse to 0.6

Table 2 Bias, coverage, length of the 95%CI, and mean squared error (MSE) for the 10,000 simulations performed under the existence of a treatment effect.

Method	Bias (SE)	Coverage (%)	CI length	MSE
Complete case analysis	–1.026 (0.005)	63.91	2.356	1.315
Mixed effects model	–0.606 (0.002)	75.22	1.589	0.451
				NORM
Baseline imputation model	–0.701 (0.003)	86.85	2.245	0.629
+ other HAM17 values	–0.398 (0.002)	99.37	2.112	0.216
+ compliance	–0.004 (0.01)	99.8	2.33	0.094
				PMM
Baseline imputation model	–0.697 (0.004)	86.87	2.255	0.629
+ other HAM17 values	–0.401 (0.002)	99.29	2.1	0.219
+ compliance	–0.013 (0.003)	99.89	2.314	0.096
				CART
Baseline imputation model	–0.676 (0.004)	83.38	2.1	0.602
+ other HAM17 values	–0.343 (0.002)	99.44	2.048	0.175
+ compliance	–0.198 (0.002)	99.877	2.043	0.096
				FIML
Baseline as auxiliary	–0.699 (0.003)	87.92	2.245	0.613
+ other HAM17 values	–0.399 (0.002)	99.59	2.119	0.21
+ compliance	0.029 (0.003)	99.94	2.258	0.08
				IPW
Baseline only	–0.715 (0.003)	91.62	2.425	0.64
+ other HAM17 values	–0.808 (0.004)	84.85	2.433	0.841
+ compliance	–0.298 (0.006)	95.47	2.94	0.466

NOTE: A negative bias implies a bigger estimated effect (reduction of the HAM17 score is an improvement). The median true effect of the estimand in the sample across simulations was -0.89 (IQ -1.28 to -0.5).

Table 3 Bias, coverage, length of the 95%CI, and mean squared error (MSE) for the 10,000 simulations performed under the absence of a treatment effect.

Method	Bias (SE)	Coverage (%)	CI length	MSE
Complete case analysis	–0.6 (0.005)	88.12	2.3	0.566
Mixed effects model	–0.189 (0.002)	99.18	1.57	0.087
				NORM
Baseline imputation model	–0.274 (0.003)	98.94	2.215	0.174
+ other HAM17 values	–0.015 (0.009)	100	2.08	0.0367
+ compliance	–0.008 (0.003)	99.86	2.286	0.0973
				PMM
Baseline imputation model	–0.279 (0.003)	98.94	2.216	0.183
+ other HAM17 values	–0.025 (0.002)	100	2.07	0.0382
+ compliance	–0.017 (0.003)	99.93	2.268	0.0939
				CART
Baseline imputation model	–0.277 (0.003)	98.08	2.049	0.185
+ other HAM17 values	–0.016 (0.002)	99.99	2.003	0.041
+ compliance	–0.014 (0.002)	99.99	2.002	0.045
				FIML
Baseline as auxiliary	–0.274 (0.003)	99.35	2.187	0.162
+ other HAM17 values	–0.012 (0.002)	100	2.069	0.031
+ compliance	0.012 (0.003)	99.94	2.217	0.076
				IPW
Baseline only	–0.294 (0.003)	99.61	2.367	0.177
+ other HAM17 values	–0.387 (0.004)	97.63	2.372	0.294
+ compliance	–0.305 (0.006)	96.14	2.866	0.459

NOTE: A negative bias implies a bigger estimated effect (reduction of the HAM17 score is an improvement). The median true effect of the estimand in the sample across simulations was 0 (IQ -0.32 to 0.33).

Fig. 3 Relationship between proportion of missing outcomes and mean squared error (MSE) with each of the methods in presence of a treatment effect. (a) The performance of static methods for context, (b–f) how—regardless of the method, the inclusion of post-baseline values and of a compliance indicator increases the accuracy of the estimates, and that such gain in accuracy increases with the proportion of missingness. Lines are smoothed with span = 0.7 and areas in gray represent the 95% CI.

Mohan, K., Pearl, J., and Tian, J. (2013), “Graphical Models for Inference With Missing Data,” in Advances in Neural Information Processing Systems, pp. 1277–1285.

 Google Scholar