Targeted Learning: Toward a Future Informed by Real-World Evidence

Abstract

The 21st Century Cures Act of 2016 includes a provision for the U.S. Food and Drug Administration10.13039/100000038 (FDA) to evaluate the potential use of Real-World Evidence (RWE) to support new indications for use for previously approved drugs, and to satisfy post-approval study requirements. Extracting reliable evidence from Real-World Data (RWD) is often complicated by a lack of treatment randomization, potential intercurrent events, and informative loss to follow-up. Targeted Learning (TL) is a sub-field of statistics that provides a rigorous framework to help address these challenges. The TL Roadmap offers a step-by-step guide to generating valid evidence and assessing its reliability. Following these steps produces an extensive amount of information for assessing whether the study provides reliable scientific evidence, including in support of regulatory decision-making. This article presents two case studies that illustrate the utility of following the roadmap. We used targeted minimum loss-based estimation combined with super learning to estimate causal effects. We also compared these findings with those obtained from an unadjusted analysis, propensity score matching, and inverse probability weighting. Nonparametric sensitivity analyses illuminate how departures from (untestable) causal assumptions affect point estimates and confidence interval bounds that would impact the substantive conclusion drawn from the study. TL’s thorough approach to learning from data provides transparency, allowing trust in RWE to be earned whenever it is warranted.

KEYWORDS:

• Real-world data
• Real-world evidence
• RWD
• RWE
• Super learner
• Targeted Learning
• TMLE

Acknowledgments

The content is the view of the author(s), and does not necessarily represent the official views of, nor an endorsement by, the FDA/HHS or the U.S. Government.

Disclosure Statement

The authors report there are no competing interests to declare.

Supplementary Materials

Appendix Tables A1–A4 and sample data analysis code are provided as supplementary materials.

Additional information

Funding

The authors gratefully acknowledge funding from the United States Food and Drug Administration (US FDA) pursuant to Contract 75F40119C10155.