Real world evidence for public health decision-making on vaccination policies: perspectives from an expert roundtable

Figures & data

Figure 1. Factors influencing vaccine effectiveness [48–50].

Table 1. Strengths and limitations of RCTs and RWE.

	RCTs	RWE
Strengths	Unbiased if excellent performance and reporting Good control of confounding Excellent for estimations of efficacy and safety in study population based on prespecified outcomes Performance under highly controlled conditions^a	Real-world conditions Permits analysis of groups not represented or under-represented in RCTs Permits post-marketing evaluation of effectiveness and safety
Limitations	Difficult to include large numbers of patients from at-risk subgroups Due to ethical considerations, conducting placebo-controlled trials for influenza vaccines, which are recommended for specific high-risk groups, is not appropriate Limited follow-up Limited population exposure due to stringent criteria for inclusion (older age, pregnancy, immunodeficiencies, underlying conditions often excluded) Performance under highly controlled conditions^a Performance under not usual health care provision conditions: more intensive and continuous contact during the study period	Prone to information bias Prone to selection bias Prone to confounding
Barriers to data generation	Recruitment, low influenza attack rates, other aspects of good clinical practice (GCP) [94–96]	Potential lack or incompleteness of necessary data Risk of bias Access to reliable and representative data/all aspects of GCP (i.e. ROBINS-I) [97]
Monitoring of vaccine efficacy/effectiveness and safety over multiple, consecutive influenza seasons and in broad cohorts/populations	Difficult and expensive, even economically unviable [94]	Feasible and relatively cheap [94]
Assessment of incidence of rare safety outcomes	Potentially not possible even in large RCTs (N > 10,000)	Large study populations (N > 100,000) enable assessment of the incidence of rare events
Endpoints relevant to public health decision making	Efficacy in preventing laboratory-confirmed infection, transmission, mild and severe illness (hospital admission, ICU admission, death, length of stay, antibiotic use, other relevant outcomes)	Effectiveness in preventing medical encounters at different points of care (outpatient, inpatient, ICU) or impact on several infection complications (e.g. pneumonia, cardiovascular outcomes, worsening frailty, death) Incidence of rare safety outcomes

Abbreviations: GCP, good clinical practice; ICU, intensive care unit.

^aPotentially a strength and a weakness.

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919.

 PubMed Web of Science ®Google Scholar

Collins R, Bowman L, Landray M, et al. The magic of randomization versus the myth of real-world evidence. N Engl J Med. 2020 Feb 13;382(7):674–678. doi: 10.1056/NEJMsb1901642

 PubMed Web of Science ®Google Scholar

Data availability statement

Data are derived from public domain resources and are referenced in the manuscript.