Differences in evidentiary requirements for oncology drug effectiveness assessments among six European health technology assessment bodies — can alignment be improved?

Figures & data

Figure 1. Outline of the methods of the secondary analysis of this research project.

Table 1. Evidentiary requirements for oncology drug assessments of six HTA bodies, labeled for acceptability (symbols).

DOMAIN	SUBD.	HTA BODIES						ALIGN. PER SUBD.*	ACCEPT. PER SUBD.
		Italy (AIFA)	Netherlands (ZINL)	Poland (AOTMiT)	Portugal (INFARMED)	England and Wales (NICE)	Sweden (TLV)
Population	Target population as authorized by EMA	Restricted to the subgroup that benefits most^§	Can be restricted to the subgroup that benefits most^‡	Restricted to the subgroup that benefits most^§	Can be restricted to the subgroup that benefits most^‡	^†	Can be restricted to the subgroup that benefits most^‡	33%	17%
	Use of biomarkers	When validated^∥	When validated^∥	When validated^∥	When validated^∥	When validated^∥	When validated^∥	100%	100%
	Extrapolation to other populations	Except for age groups^§	Depending on justification (e.g. children) ^‡	^†	Except for rare diseases^§	^∥	Depending on justification (e.g. children) ^‡	33%	33%
Comparator	Selected comparator	Drug used in the clinical trial and available in the country^∥	Standard of care^∥	Drug used in the clinical trial and available in the country^∥	Standard of care^∥	Standard of care^∥	Standard of care^∥	100%**	100%
	Class effects	^¶	Should be measured separately^‡	^†	^¶	Creates uncertainty^‡	Accepted for ‘me-too’ drugs^‡	33%	17%
	Indirect comparisons	When needed^†	When needed, creates more uncertainty^†	When needed and done in accordance with guidelines^∥	When needed^†	When needed and done in accordance with guidelines^∥	When needed and done in accordance with guidelines^∥	50%	100%
Endpoints	PFS as endpoint	^§	^∥	^∥	^§	^†	^∥	50%	67%
	Other surrogate endpoints	^‡	^¶	^†	^§	^‡	^∥	0%	33%
	Absence of QoL data	QoL is supportive^∥	QoL is important^§	QoL is supportive^∥	QoL is important^§	QoL is very important^£¶	QoL is important^§	50%	33%
Trial design and data sources	Real-world evidence	But not enough (supportive) ^†	But not enough (supportive) ^†	But not enough (supportive) ^†	But not enough (supportive) ^†	But not enough (supportive) ^†	But not enough (supportive) ^†	100%	100%
	Network Meta- Analysis	Could be accepted^‡	When needed and preferably published^∥	Could be accepted^‡	When needed^†	When needed, creates uncertainty^†	When needed^†	50%	67%
	Single-arm trials	If evidence is satisfactory^∥	Can be accepted, creates uncertainty^∥	But treated as less strong evidence^†	^§	Creates uncertainty^†	Creates uncertainty^†	50%	83%
	Novel trial designs	If controlled^∥	^‡	If methodology is well-described^∥	If plausible biological mechanism^∥	Creates uncertainty^†	If accepted by EMA^†	50%**	83%
	Cross-over in trials	Based on the influence on the interpretability of the results^‡	Can be accepted, creates uncertainty^∥	But creates an interpretation challenge^∥	Based on the influence on the interpretability of the results^‡	Creates uncertainty^†	^‡	33%	50%
	Evidence from small populations	If evidence is satisfactory^∥	If requirements of GRADE methodology are met^∥	Creates uncertainty^†	If evidence is satisfactory^∥	Creates uncertainty^†	If it is the best available evidence^∥	67%**	100%
	Short time period	^‡	Hard endpoint should be identified^§	The longer the better^∥	Clinically relevant in context of natural history^‡	The longer the better, short period creates uncertainty^∥	The longer the better, short period creates uncertainty, a convincing mean OS should be demonstrated^∥	50%**	50%
Statistical analysis	Absence of statistical significance	^¶	Not very decisive (focus on CI)^‡	^¶	^¶	No hard cutoff point^‡	^¶	67%	0%
	Post-hoc subgroup analyses	Unless drug does harm^§	^¶	Unless request HTA body^§	^§	^§	Unless request HTA body^§	83%	0%
	Clinical relevance of effect size as assessed by EMA	Own assessment, no threshold^‡	Own assessment (threshold 3 months OS) ^§	Follows EMA^†	Own assessment, no threshold^‡	Own assessment, no threshold^‡	Own assessment, no threshold^‡	0%	17%
ACCEPT. PER HTA BODY^††‡‡		42%	47%	79%	37%	68%	58%

ACCEPT.: acceptability; ALIGN.: alignment; AIFA indicates Agenzia italiana del farmaco; AOTMiT, Agencja Oceny Technologii Medycznych i Taryfikacji; CI; confidence interval; EMA, European Medicines Agency; GRADE, Grading of Recommendations, Assessment, Development, and Evaluation; HTA, health technology assessments; INFARMED, National Authority for Medicines and Health Products; NICE, National Institute for Health and Care Excellence; OS, overall survival; PFS, progression-free survival; QoL, quality of life; SUBD.: subdomain; TLV, Dental and Pharmaceutical Benefits Agency; UK, United Kingdom; ZINL, Zorginstituut Nederland

*Alignment per subdomain was calculated by summing up the subdomains belonging to the alignment label, dividing that by the total number of subdomains, and expressing it as a percent. The acceptability label ‘case dependent’ cannot be classified as aligned, as implications can differ per HTA body.

^†Accepted.

^‡Case dependent.

^§Often not accepted.

^∥Often accepted.

^¶Not accepted.

^£Quality of Life (QoL) refers in England and Wales to the health state utility, rather than QoL.

**Same acceptability label (‘often accepted’), but acceptability is based on different conditions.

^‡‡††Degree of evidence acceptability was calculated by summing up the subdomains which were labeled as ‘accepted’ or ‘often accepted,’ dividing that by the total number of subdomains, and expressing it as a percent.

Figure 2. Relative alignment on group level: alignment on subdomains for each HTA body relative to the other HTA bodies from complete to no alignment*.

Table

Country	Location	Access organization	Time between marketing authorization and patient access*	HTA orientation
Italy	South	Regional	Modest	Clinical effectiveness
The Netherlands	West	National and local	Short	Cost-effectiveness
Poland	East	National	Long	Budget impact
Portugal	South	National	Long	Cost-effectiveness
England and Wales	West	National	Short	Cost-effectiveness
Sweden	North	National	Short	Cost-effectiveness

EFPIA indicates European Federation of Pharmaceutical Industries and Associations; HTA, health technology assessment; W.A.I.T., Waiting to Access Innovative Therapies.

Table Footnotes.

*Based on the 34 countries included in the EFPIA W.A.I.T. indicator. Short: the 11 countries included in the EFPIA W.A.I.T. indicator with the least time to patient access. Modest: the 12 countries included in the EFPIA W.A.I.T. indicator with median time to patient access. Long: the 11 countries included in the EFPIA W.A.I.T. indicator with the longest time to patient access. The same HTA bodies were included as in the previous publication.*.

*Obtained from: Wolters S, Jansman F, Postma M. Differences in Evidentiary Requirements Between European Medicines Agency and European Health Technology Assessment of Oncology Drugs – Can Alignment Be Enhanced? Value in Health 2022; S1098–3015: 01987–8.

Table

Domains	Subdomains
Population	Target population Use of biomarkers Extrapolation
Comparator	Selected comparator Class effects Indirect comparison
Endpoints	PFS as endpoint QoL/HRQoL and other PROs Other surrogate endpoints
Trial design and data sources	Real-world evidence Network meta-analysis Single-arm trials Novel trial designs Cross-over designs Evidence from small populations Acceptability of short time period
Statistical analyses	Statistical significance Post-hoc subgroup analyses Clinical relevance of the effect size

Note: based on Shaping European early Dialogues (SEED)** and Tafuri et al.***.

HRQoL indicates health-related quality of life; PFS, progression-free survival; PRO, patient-reported outcome; QoL, quality of life; SEED, Shaping European Early Dialogues

*Obtained from:Wolters S, Jansman F, Postma M. Differences in Evidentiary Requirements Between European Medicines Agency and European Health Technology Assessment of Oncology Drugs – Can Alignment Be Enhanced? Value in Health 2022; S1098-3015: 01987–8.

**Facey K, Granados A, Meyer F, et al. Multi-stakeholder collaboration in generating the best possible knowledge – the SEED experience. Report of HTAi 2015 Panel Session. Health Technology Assessment International – Canada. 2015., https://htai.org/wp-content/uploads/2018/02/HTAi_SEED_Panel_Report_final.pdf (accessed 20 March 2020).

***Tafuri G, Pagnini M, Moseley J, et al. How aligned are the perspectives of EU regulators and HTA bodies? A comparative analysis of regulatory-HTA parallel scientific advice. Br J Clin Pharmacol. 2016;82:965–973.

Table

Organization	Nationality	Function
AIFA CTS (Commissione Tecnico Scientifica), Università del Piemonte Orientale	Italy	Member
−	Italy	HTA-expert from Italy
National Health Care Institute (ZIN)	The Netherlands	Advisor Zorginstituut Nederland; secretaris WAR-CG
National Health Care Institute (ZIN)	The Netherlands	Project leader CBG-ZiN
Warsaw Institute of Mother and Child	Poland	Professor and Head of the Department of Pharmacoeconomics
INFARMED	Portugal	Executive Board Advisor
INFARMED	Portugal	Head of Information and Strategic Planning Office
NICE	England and Wales	Senior Adviser
NT counsil	Sweden	Professor, program manager

*Obtained from: Wolters S, Jansman F, Postma M. Differences in Evidentiary Requirements Between European Medicines Agency and European Health Technology Assessment of Oncology Drugs – Can Alignment Be Enhanced? Value in Health 2022; S1098–3015: 01987–8.

Table

		HTA BODY
DOMAIN	SUBDOMAIN	Italy	Netherlands	Poland	ALIGNMENT AMONG HTA BODIES PER SUBDOMAIN
Population	Target population as authorized by EMA	Restricted to the subgroup that benefits most^§	Can be restricted to the subgroup that benefits most^‡	Restricted to the subgroup that benefits most^§	50%
	Use of biomarkers	When validated^∥	When validated^∥	When validated^∥	100%
	Extrapolation to other populations	Except for age groups^§	Depending on justification (e.g. children) ^‡	^†	0%
ACCEPTABILITY PER HTA BODY		33%	33%	67%

^†Accepted.

^‡Case dependent.

^§Often not accepted.

^∥Often accepted

Table

Legend	<25%	25%–49%	50%–74%	≥75%
	Italy	The Netherlands	Poland	Portugal	England and Wales	Sweden
Italy		26%	42%	58%	21%	32%
The Netherlands	26%		26%	32%	21%	32%
Poland	42%	26%		32%	42%	47%
Portugal	58%	32%	32%		26%	42%
England and Wales	21%	21%	42%	26%		47%
Sweden	32%	32%	47%	42%	47%