Combining Real-World and Randomized Control Trial Data Using Data-Adaptive Weighting via the On-Trial Score

Abstract

Clinical trials with a hybrid control arm (a control arm constructed from a combination of randomized patients and real-world data on patients receiving usual care in standard clinical practice) have the potential to decrease the cost of randomized trials while increasing the proportion of trial patients given access to novel therapeutics. However, due to stringent trial inclusion criteria and differences in care and data quality between trials and community practice, trial patients may have systematically different outcomes compared to their real-world counterparts. We propose a new method for analyses of trials with a hybrid control arm that efficiently controls bias and Type I error. Under our proposed approach, selected real-world patients are weighted by a function of the “on-trial score,” which reflects their similarity to trial patients. In contrast to previously developed hybrid control designs that assign the same weight to all real-world patients, our approach up-weights real-world patients who more closely resemble randomized control patients while dissimilar patients are discounted. Estimates of the treatment effect are obtained via Cox proportional hazards models. We compare our approach to existing approaches via simulations and apply these methods to a study using pseudo-electronic health record data. Our proposed method is able to control Type I error, minimize bias, and decrease variance when compared to using trial data only in nearly all scenarios examined. Therefore, our new approach can be used when conducting clinical trials by augmenting the standard-of-care arm with weighted patients from the EHR to increase power without inducing bias.

Keywords:

• Clinical trials
• Electronic health record
• Hybrid control
• Survival analysis

Supplementary Materials

Additional figures describing the simulation and case studies may be found in the supplementary materials.

Acknowledgments

This study, carried out under YODA Project # 2020-4453, used data obtained from the Yale University Open Data Access Project, which has an agreement with Janssen Research & Development, L.L.C. The interpretation and reporting of research using this data are solely the responsibility of the authors and does not necessarily represent the official views of the Yale University Open Data Access Project or Janssen Research & Development, L.L.C.

Data Availability Statement

The data that support the findings of this study are available from Janssen Research & Development, L.L.C. via the Yale University Open Data Access Project. Restrictions apply to the availability of these data, which were used under license for this study. Data are available at https://yoda.yale.edu/ with the permission of the Yale University Open Data Access Project.

Disclosure Statement

The author(s) declared no other potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Additional information

Funding

Research reported in this publication was supported in part by NIH grant R21CA227613. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.