How accurate are the longer-term projections of overall survival for cancer immunotherapy for standard versus more flexible parametric extrapolation methods?

Figures & data

Figure 1. Estimation of dPE, dRAUC and dIAUC. Abbreviations. dIAUC, difference between the interval area under the curve; dPE, difference in the point estimates; dRAUC, difference between the restricted area under the curve; FoU, follow-up; KM, Kaplan-Meier.

Table 1. Model fit summary.

Summary		TA319	TA357	TA366	TA384	TA417	TA428	TA447	TA483	TA484	TA490
n		250	180	277	210	410	344	154	135	292	240
Initial follow-up time (months)		47.4	16.6	17.2	16.5	30.5	22.2	18.7	22.7	25.3	16.6
Later follow-up time (months)		72.0	35.1	26.1	27.7	53.8	35.0	33.0	51.8	51.9	24.4
Model fit summary
Survival point estimates (dPE)
Proportion of models that underestimate survival point estimate at midpoint (%)	Std. PSM	100	100	100	57	86	86	50	43	71	29
	Spline	100	100	67	44	100	89	11	0	78	0
	MCM (general population)	100	71	43	0	71	33	0	0	71	0
Proportion of models that underestimate survival point estimate at the end of follow-up (%)	Std. PSM	100	100	100	71	86	71	33	86	100	29
	Spline	100	100	100	67	100	67	0	33	89	0
	MCM (general population)	100	57	43	14	71	17	0	0	100	0
Proportion of models within two standard errors of the Kaplan-Meier estimate at the midpoint (%)	Std. PSM	0	57	71	100	43	71	100	71	57	71
	Spline	67	78	89	100	22	56	100	100	67	78
	MCM (general population)	100	86	100	86	57	57	71	29	71	0
Proportion of models within two standard errors of the Kaplan-Meier estimate at the end of follow-up (%)	Std. PSM	0	71	17	71	43	71	83	71	57	71
	Spline	0	89	56	67	0	89	78	100	56	89
	MCM (general population)	57	71	57	86	43	50	14	29	71	0
Restricted mean survival (dRAUC)
Proportion of models that underestimate restricted mean survival (AUC) at midpoint (%)	Std. PSM	100	100	100	43	100	86	33	0	71	0
	Spline	100	100	100	33	100	89	0	0	78	0
	MCM (general population)	100	100	71	0	100	33	0	0	71	0
Proportion of models that underestimate restricted mean survival (AUC) at the end of follow-up (%)	Std. PSM	100	100	100	57	86	86	33	29	71	0
	Spline	100	100	100	33	100	89	0	0	89	0
	MCM (general population)	100	71	57	0	86	17	0	0	71	0

Abbreviations. AUC, area under the curve; CI, confidence interval; KM, Kaplan–Meier; MCM, mixture cure model; n, number; TA, technology appraisal; Std. PSM, standard parametric survival model.

Notes: Follow-up, survival, and numbers at risk based on digitized data.

Figure 2. Six-month interval restricted mean survival (dIAUC) for each TA. Abbreviations. dIAUC, difference between the interval area under the curve; Exp, exponential; GF, Generalized F; GG, generalized gamma; Gom, Gompertz; LL, log-logistic; LN, log-normal; MCM, mixture cure model; PH, proportional hazards; PO, proportional odds; Pr, probit; Std., standard; PSM, parametric survival model; TA, technology appraisal; Weib, Weibull.

Figure 3. KM and best and worst fitting models overlaid for each TA. Abbreviations. Exp, exponential; GF, generalized F; GG, generalized gamma; Gom, Gompertz; KM, Kaplan-Meier; LL, log-logistic; LN, log-normal; MCM, mixture cure model; PH, proportional hazards; PO, proportional odds; Pr, probit; Std., standard; PSM, parametric survival model; TA, technology appraisal; Weib, Weibull.

Table 2. Comparison of MCM dPEs at the midpoint when adjusted for general population mortality or with the general population subject to additional excess mortality.

	TA319		TA357		TA366		TA384		TA417		TA428		TA447		TA483		TA484		TA490
N	250		180		277		210		410		344		154		135		292		240
Initial follow-up time (months)	47.4		16.6		17.2		16.5		30.5		22.2		18.7		22.7		25.3		16.6
	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard	No additional hazard	Additional hazard
Exponential	−3.3	−6.0	−8.2	−8.2	−4.2	−4.2	0.5	−3.1	−2.0	−2.1	−2.4	−4.8	3.9	−7.1	5.8	0.9	−5.8	−6.5	6.5	4.9
Generalized F	−3.9	−5.8	−4.3	−4.3	3.2	−0.7	3.6	3.6	−7.4	−9.8	NA	NA	7.3	−3.9	8.1	5.2	1.2	1.2	7.0	7.0
Generalized Gamma	−3.9	−5.8	−4.2	−4.3	4.9	0.7	3.6	3.6	−7.0	−13.6	10.3	1.2	9.4	−4.3	12.0	4.9	1.2	1.2	7.0	6.9
Gompertz	−2.0	−5.1	5.5	2.5	5.2	1.0	7.9	3.1	0.2	−14.5	10.0	1.0	10.7	−3.3	13.8	6.2	−5.1	−5.1	9.4	7.1
Log-logistic	−3.4	−5.3	−1.7	−2.6	−2.3	−4.7	3.5	−0.4	−2.5	−2.9	−0.2	−1.1	4.7	−4.4	8.5	3.6	−0.6	−0.8	8.9	7.5
Log-normal	−3.9	−5.8	−2.5	−2.5	−3.1	−3.1	2.8	−0.8	1.2	1.2	1.2	1.2	2.9	2.8	5.0	2.0	−0.8	−0.8	7.1	6.2
Weibull	−1.3	−4.6	2.3	−0.3	0.5	−3.3	5.7	1.1	−8.1	−11.1	5.2	−2.3	7.5	−5.4	13.1	5.7	−3.5	−5.9	10.6	8.2
Proportion of models that underestimate survival point estimate at midpoint	100%	100%	71%	86%	43%	71%	0%	43%	71%	86%	33%	50%	0%	86%	0%	0%	71%	71%	0%	0%

Abbreviations. dPE, difference in point estimate; MCM, mixture cure model.

Notes. Red cells indicate a larger absolute dPE with additional hazard (reduced accuracy); green cells indicate a reduced dPE with additional hazard (improved accuracy).

Table 3. dPE and dRAUC values at both the midpoint and end of follow-up for the top three most accurate fitting models.

Top three most accurate models	dPE midpoint: model (value)	dPE end of follow-up: model (value)	dRAUC midpoint: model (value)	dRAUC end of follow-up: model (value)
TA319
1	MCM Weib	MCM Weib	MCM Weib	MCM Weib
	(0.01)	(0.02)	(−0.29)	(−0.47)
2	MCM Gom	MCM Gom	MCM Gom	MCM Gom
	(0.02)	(0.02)	(−0.39)	(−0.66)
3	MCM Exp	MCM Exp	Std. PSM Gom	MCM Exp
	(0.03)	(0.04)	(−0.47)	(−0.97)
TA357
1	MCM LL	MCM LN	MCM Gom	MCM Weib
	(0.02)	(0.01)	(−0.29)	(0.13)
2	MCM Weib	Std. PSM LN	MCM Weib	MCM Gom
	(−0.02)	(0.01)	(−0.37)	(0.58)
3	MCM LN	Spline PO k = 1	MCM LN	MCM LL
	(0.02)	(0.01)	(−0.49)	(−0.60)
TA366
1	Spline PH k = 3	Spline PO k = 3	MCM GF	Spline PO k = 3
	(0.00)	(0.01)	(−0.01)	(−0.01)
2	Spline Pr k = 3	Spline Pr k = 3	MCM GG	Spline Pr k = 3
	(0.00)	(0.01)	(0.02)	(−0.02)
3	MCM Weib	MCM Weib	MCM Gom	Spline PH k = 3
	(0.00)	(−0.01)	(0.03)	(−0.03)
TA384
1	Spline PH k = 1	MCM Exp	Std. PSM LN	Std. PSM LN
	(0.00)	(0.00)	(0.05)	(−0.07)
2	MCM Exp	Std. PSM Gom	MCM Exp	Spline PH k = 1
	(0.00)	(0.01)	(0.07)	(0.10)
3	Spline PO k = 1	Spline Pr k = 1	Std. PSM LL	MCM Exp
	(−0.01)	(0.01)	(−0.08)	(0.11)
TA417
1	MCM Gom	Std. PSM LN	Std. PSM LN	MCM Gom
	(0.00)	(0.00)	(−0.09)	(−0.14)
2	MCM LN	MCM LN	MCM LN	Std. PSM LN
	(−0.01)	(0.00)	(−0.09)	(0.14)
3	Std. PSM LN	MCM LL	Std. PSM LL	MCM LN
	(−0.01)	(0.04)	(−0.43)	(0.15)
TA428
1	MCM LL	Spline PO k = 1	MCM LL	Spline PO k = 3
	(0.00)	(0.00)	(−0.01)	(−0.02)
2	Spline PH k = 3	Std. PSM LL	Spline PH k = 3	MCM LL
	(0.00)	(0.00)	(0.02)	(0.03)
3	Spline PO k = 3	Spline Pr k = 3	Spline PO k = 3	Spline PH k = 3
	(0.00)	(−0.01)	(−0.06)	(0.07)
TA447
1	Spline PH k = 1	Std. PSM Weib	Std. PSM Weib	Std. PSM LL
	(0.01)	(0.01)	(−0.06)	(0.16)
2	Std. PSM GG	Std. PSM Exp	Std. PSM LL	Spline PH k = 1
	(−0.01)	(0.04)	(0.10)	(0.22)
3	Std. PSM LL	Spline PH k = 1	Spline PH k = 1	Std. PSM Weib
	(0.01)	(−0.04)	(0.17)	(−0.31)
TA483
1	Spline Pr k = 1	Spline PO k = 1	Std. PSM Weib	Std. PSM LL
	(0.00)	(0.00)	(0.11)	(0.18)
2	Spline PH k = 1	Std. PSM Gom	Std. PSM Exp	Spline Pr k = 1
	(0.00)	(0.01)	(0.17)	(0.55)
3	Std. dist. LL	Spline PH k = 2	Std. PSM LL	Spline PH k = 1
	(0.00)	(−0.01)	(0.46)	(0.59)
TA484
1	Spline Pr k = 1	Std. PSM GG	Spline Pr k = 1	Spline Pr k = 1
	(−0.01)	(0.00)	(0.07)	(0.00)
2	MCM LL	Std. PSM GF	MCM LL	MCM GG
	(0.01)	(0.00)	(−0.07)	(0.17)
3	MCM LN	MCM GF	MCM LN	MCM GF
	(0.01)	(0.00)	(−0.08)	(0.17)
TA490
1	Std. PSM Weib	Spline PH k = 3	Std. PSM Exp	Std. PSM Exp
	(0.02)	(0.00)	(0.12)	(0.02)
2	Spline PH k = 3	Spline PH k = 2	Std. PSM Weib	Std. PSM Weib
	(−0.02)	(−0.01)	(0.13)	(0.03)
3	Std. PSM Exp	Spline Pr k = 3	Std. PSM LL	Std. PSM LL
	(0.02)	(−0.02)	(0.28)	(0.43)

Abbreviations. dPE, difference in the point estimates; dRAUC, difference between the restricted area under the curve; Exp, exponential; GF, generalized F; GG, generalized gamma; Gom, Gompertz; LL, log-logistic; LN, log-normal; MCM, mixture cure model; PH, proportional hazards; PO, proportional odds; Pr, probit; Std., standard; PSM, parametric survival model; Weib, Weibull.

Table 4. dPE and dRAUC values at both the midpoint and end of follow-up for the top three least accurate fitting models.

Top three least accurate models	dPE midpoint: model (value)	dPE end of follow-up: model (value)	dRAUC midpoint: model (value)	dRAUC end of follow-up: model (value)
TA319
1	Std. PSM Exp	Std. PSM Exp	Std. PSM LL	Std. PSM Exp
	(0.12)	(0.15)	(−1.28)	(−2.60)
2	Std. PSM Weib	Std. PSM Weib	Std. PSM Exp	Std. PSM Weib
	(0.12)	(0.14)	(−0.92)	(−2.49)
3	Std. PSM LL	Std. PSM LN	Std. PSM Weib	Std. PSM LL
	(0.08)	(0.10)	(−0.88)	(−2.40)
TA357
1	Std. PSM Gom	Std. PSM Gom	Std. PSM Weib	Std. PSM Gom
	(0.15)	(0.16)	(−1.49)	(−2.99)
2	Std. PSM Weib	Std. PSM Weib	Std. PSM Gom	Std. PSM Weib
	(0.14)	(0.15)	(−1.46)	(−2.92)
3	Spline PH k = 2	MCM Gom	Spline PH k = 2	Spline PH k = 2
	(0.11)	(−0.14)	(−1.15)	(−2.25)
TA366
1	Std. PSM Gom	Std. PSM Gom	Std. PSM Gom	Std. PSM Gom
	(0.09)	(0.16)	(−0.37)	(−0.89)
2	Std. PSM Weib	Std. PSM Weib	Std. PSM Weib	Std. PSM Weib
	(0.09)	(0.15)	(−0.32)	(−0.81)
3	Spline PH k = 1	Spline PH k = 1	Std. PSM Exp	Spline PH k = 1
	(0.06)	(0.11)	(−0.31)	(−0.63)
TA384
1	MCM Gom	Std. PSM Exp	MCM Gom	MCM Gom
	(−0.08)	(0.12)	(0.4)	(0.95)
2	Std. PSM Exp	Std. PSM Weib	Spline PH k = 3	MCM Weib
	(0.07)	(0.11)	(0.37)	(0.71)
3	Std. PSM Weib	MCM Gom	Spline PO k = 3	Spline PH k = 3
	(0.06)	(−0.10)	(0.37)	(0.66)
TA417
1	Std. PSM Gom	Std. PSM Gom	Std.PSM Gom	Std. PSM Gom
	(0.16)	(0.22)	(−1.61)	(−3.68)
2	Std. PSM Weib	Std. PSM Weib	Std. PSM Weib	Std. PSM Weib
	(0.10)	(0.13)	(−1.07)	(−2.29)
3	Spline PH k = 1	Spline PH k = 1	Spline PH k = 1	Spline PH k = 1
	(0.09)	(0.13)	(−1.06)	(−2.25)
TA428
1	MCM GG	MCM GG	Std. PSM Weib	MCM GG
	(−0.10)	(−0.16)	(−0.54)	(1.27)
2	MCM Gom	MCM Gom	Spline PO k = 2	MCM Gom
	(−0.10)	(−0.16)	(−0.5)	(1.23)
3	Std. PSM Weib	MCM Weib	Spline Pr k = 2	Std. PSM Weib
	(0.08)	(−0.10)	(−0.45)	(−1.07)
TA447
1	MCM Gom	MCM Gom	MCM Gom	MCM Gom
	(−0.11)	(−0.22)	(0.7)	(1.81)
2	MCM GG	MCM GG	MCM GG	MCM GG
	(−0.09)	(−0.20)	(0.63)	(1.62)
3	MCM Weib	MCM Weib	MCM GF	MCM Weib
	(−0.08)	(−0.18)	(0.56)	(1.36)
TA483
1	MCM Gom	MCM Gom	MCM Gom	MCM Gom
	(−0.14)	(−0.16)	(1.92)	(4.07)
2	MCM Weib	MCM Weib	MCM Weib	MCM Weib
	(−0.13)	(−0.15)	(1.89)	(3.93)
3	MCM GG	MCM GG	MCM GG	MCM GG
	(−0.12)	(−0.14)	(1.77)	(3.64)
TA484
1	Spline PH k = 3	Spline PH k = 3	Spline PH k = 3	Spline PH k = 3
	(0.08)	(0.10)	(−0.77)	(−2.04)
2	Spline PH k = 2	Spline PH k = 2	Spline PH k = 2	Spline PH k = 2
	(0.08)	(0.10)	(−0.76)	(−2.03)
3	Std. PSM Weib	Std. PSM Weib	Std. PSM Weib	Std. PSM Weib
	(0.08)	(0.10)	(−0.65)	(−1.86)
TA490
1	MCM Weib	MCM Weib	MCM Weib	MCM Weib
	(−0.11)	(−0.12)	(0.64)	(1.12)
2	MCM Gom	MCM Gom	Spline PO k = 1	MCM Gom
	(−0.09)	(−0.11)	(0.61)	(1.00)
3	MCM LL	MCM LL	MCM LL	MCM LL
	(−0.09)	(−0.09)	(0.59)	(0.97)

Abbreviations. dPE, difference in the point estimates; dRAUC, difference between the restricted area under the curve; Exp, exponential; GF, generalized F; GG, generalized gamma; Gom, Gompertz; LL, log-logistic; LN, log-normal; MCM, mixture cure model; PH, proportional hazards; PO, proportional odds; Pr, probit; Std., standard; PSM, parametric survival model; Weib, Weibull.

Table 5. Additional mortality for mixture cure models.

TA	N	Excess mortality source data and further detail	Excess mortality source data timepoint	Estimated hazard
TA319	250	AJCC, Melanoma of the skin – data digitized and combined from 3 separate metastatic populations:	3 years	7.67 (4.79–12.28)
TA357	180			5.41 (3.38–8.66)
TA366	277			5.13 (3.53–7.47)
TA384	210	M1a: metastases to skin, subcutaneous, or distant lymph nodes		4.88 (3.21–7.41)
		M1b: metastases to lung
		M1c: metastases to all other visceral sites or distant metastases to any site combined with elevated LDH
TA417	410	AJCC, data digitized for Stage IV kidney cancer	3 years	16.98 (9.31–30.98)
TA428	344	AJCC, data digitized for Stage IV NSCLC	4 years	14.16 (9.98–20.07)
TA447	154			12.06 (7.7–18.9)
TA483	135	AJCC, data digitized and combined from Stage IIIb and IV NSCLC		10.84 (6.67–17.64)
TA484	292			12.88 (8.90–18.64)
TA490	240	AJCC, data digitized and combined from three separate head and neck cancer populations:	3 years	8.71 (3.91–19.41)
		Larynx:
		data combined from Stage III and IV
		Oral:
		data combined from Stage II and IV
		Pharynx:
		data combined from Stage II and IV

Abbreviations. AJCC, American Joint Committee on Cancer; LDH, lactate dehydrogenase; NSCLC, non-small cell lung cancer; TA, technology appraisal.

Note: Estimated hazards presented in Table 4 were generated using Cox proportional hazard modeling. The estimated hazard is the hazard associated with the AJCC population compared to US general mortality based on US life tables from the Centers for Disease Control and Prevention. Excess mortality data were used from the timepoint when survival curves began to level off, which was 3 years for the populations with melanoma, kidney, and head and neck cancers and 4 years for the population with NSCLC.