Real world data: an opportunity to supplement existing evidence for the use of long-established medicines in health care decision making

Figures & data

Figure 1 General hierarchy of study types.^11–15

Table 1 Overview of the strengths and weaknesses of randomized controlled trials^9,10

Strengths	• Adequately powered
	• Internal validity due to unbiased methodology, for example, narrowly defined study population, randomization, blinding, and inclusion of control groups
	• Scientifically robust
	• Provide substantial information regarding the efficacy and safety of interventional products
	• Prospective design
	• Prespecified, well-defined end points
Weaknesses	• Lack external validity and generalizability to different settings
	• Lack statistical power if sample size is not large enough to answer research question
	• Can be hampered by ethical and practical considerations
	• Do not provide all evidence required for medical decision making or guide patient-centered care
	• Require a lot of investment in terms of finances, resources, and time
	• Restrictive enrollment criteria; involve homogeneous patient populations, which are not reflective of those seen in real-life clinical settings
	• Often conducted over a shorter period of time than is required to fully assess the clinical and economic impact of a medical intervention
	• Volunteer bias
	• “Placebo” response

Table 2 Overview of the strengths and limitations of different types of real world data sources^2,23–25

Source	Key characteristics	Strengths	Limitations
Supplements to registration RCTs	• Additional data such as patient-reported outcomes, medical resource use, and costs gathered alongside standard, clinically focused registration RCTs • May provide evidence on treatment patterns for common events	• Randomized design	• Restricted patient population • Carefully controlled clinical setting for data collection • Protocol-driven resource use • Lack of statistical power to detect events other than specified key end points • Relatively short time frame
Practical/pragmatic clinical trials	• Simple trials involving prospective, randomized study designs but with larger and more diverse patient populations than conventional RCTs • Often focusing on obtaining policy-relevant outcomes data	• Broad patient population • Randomized design • Sufficient statistical power to establish significant differences in key outcomes • Resource use less likely to be protocol driven	• Increased cost of data collection due to larger number of patients and clinical settings involved • Potential for reduced data quality (missing data, data entry/coding errors) • Lack of standardization across settings
Patient/disease registries	• Observational, prospective, cohort studies assessing real world safety and effectiveness, quality of care/provider performance, and cost-effectiveness • Often conducted to collect postauthorization marketing safety data (to address specific safety concerns or to satisfy regulatory requirements)	• Larger and more diverse patient groups than RCTs • Reflect real world outcomes, as well as treatment patterns and clinical decision making • Longer time frame than RCTs	• Nonrandomized design • Visit schedules not required/data not collected at fixed intervals • Potential for reduced data quality (missing data, data entry/coding errors) • Lack of standardization across settings • Risk factors/outcomes may change during follow-up • Statistical adjustments may be required to address confounding/imbalance • Causality cannot be confirmed
Administrative data (claims databases)	• Retrospective, longitudinal, and cross-sectional analyses of clinical and economic outcomes at patient level • Claims data are collected primarily for reimbursement, but databases may also contain some clinical diagnosis/procedure information and details on related resource use and costs	• Large size of databases allows for identification of outcomes of patients with rare events • Analyses can be performed at low cost and over a short time frame	• Nonrandomized design • Potential for reduced data quality (missing data, data entry/coding errors) • Limited comprehensive clinical data across health care settings • Lack of distinction between costs and charges
Population health surveys	• Designed to collect descriptions of health status and well-being, health care utilization, treatment patterns, and health care expenditures from patients, providers, or individuals in the general population	• Provide unique contributions about generalizability of treatments and their impacts, and about use of and expenditures for health services • Methodologically rigorous, relying on complex sample survey designs	• Lacking relevant data on specific products • Data subject to issues of subjectivity and recall bias
EHRs/other technologies capturing real-time clinical treatment and outcomes	• Used for medical chart reviews to produce specific information on the real world use of specific tests or medications for particular conditions	• Important sources for RWD from a wide range of clinical settings throughout the world • Expansion of electronic data capture is lowering the cost of the medical chart reviews • May contain detailed, longitudinal information, including patient-level, disease-specific symptoms	• High-end statistical analysis tools required to transform the information for research purposes

Abbreviations: EHR, electronic health record; RCT, randomized controlled trial; RWD, real world data.

Figure 2 Schematic illustration of the utilization of randomized controlled trial data and real world data through the lifecycle of a medical intervention.

Figure 3 Interconnected platform for maximizing the use of real world evidence.

GuyattGHSackettDLSinclairJCUsers’ guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working GroupJAMA199527422180018047500513

 PubMed Web of Science ®Google Scholar

EvansDHierarchy of evidence: a framework for ranking evidence evaluating healthcare interventionsJ Clin Nurs2003121778412519253

 PubMed Web of Science ®Google Scholar

BoothCMTannockIFRandomised controlled trials and population-based observational research: partners in the evolution of medical evidenceBr J Cancer2014110355155524495873

 PubMed Web of Science ®Google Scholar

NallamothuBKHaywardRABatesERBeyond the randomized clinical trial: the role of effectiveness studies in evaluating cardiovascular therapiesCirculation2008118121294130318794402

 PubMed Web of Science ®Google Scholar

GarrisonLPJrNeumannPJEricksonPUsing real-world data for coverage and payment decisions: the ISPOR Real-world Data Task Force reportValue Health200710532633517888097

 PubMed Web of Science ®Google Scholar

European Medicines AgencyUpdate on real world evidence data collection Available from: http://ec.europa.eu/health/files/committee/stamp/2016-03_stamp4/4_real_world_evidence_ema_presentation.pdfAccessed April 5, 2017

 Google Scholar

SedgwickPProspective cohort studies: advantages and disadvantagesBMJ2013347f6726

 Web of Science ®Google Scholar