The Current Landscape in Biostatistics of Real-World Data and Evidence: Clinical Study Design and Analysis

References

• Ades, A. E., Caldwell, D. M., Reken, S., Welton, N. J., Sutton, A. J., and Dias, S. (2013), “Evidence Synthesis for Decision Making 7: A Reviewer’s Checklist,” Medical Decision Making, 33, 679–691. DOI: 10.1177/0272989X13485156.
   PubMed Web of Science ®Google Scholar
• Baker, S. G., and Lindeman, K. S. (2001), “Rethinking Historical Controls,” Biostatistics, 2, 383–396. DOI: 10.1093/biostatistics/2.4.383.
   PubMedGoogle Scholar
• Bell, H., Wailoo, A. J., Hernandez, M., Grieve, R., Faria, R., Gibson, L., and Grimm, S. (2016), “NICE DSU Technical Support Document: The Use of Real World Data for the Estimation of Treatment Effects in NICE Decision Making,” 60, available at http://nicedsu.org.uk/wp-content/uploads/2018/05/RWD-DSU-REPORT-Updated-DECEMBER-2016.pdf.
   Google Scholar
• Bennett, M. S. (2018), “Improving the Efficiency of Clinical Trial Designs by Using Historical Control Data or Adding a Treatment Arm to an Ongoing Trial,” Thesis, University of Cambridge.
   Google Scholar
• Berger, M. L., Dreyer, N., Anderson, F., Towse, A., Sedrakyan, A., and Normand, S.-L. (2012), “Prospective Observational Studies to Assess Comparative Effectiveness: The ISPOR Good Research Practices Task Force Report,” Value in Health, 15, 217–230. DOI: 10.1016/j.jval.2011.12.010.
   PubMed Web of Science ®Google Scholar
• Berger, M. L., Mamdani, M., Atkins, D., and Johnson, M. L. (2009), “Good Research Practices for Comparative Effectiveness Research: Defining, Reporting and Interpreting Nonrandomized Studies of Treatment Effects Using Secondary Data Sources: The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part I,” Value in Health, 12, 1044–1052.
   PubMed Web of Science ®Google Scholar
• Berger, M. L., Sox, H., Willke, R. J., Brixner, D. L., Eichler, H.-G., Goettsch, W., Madigan, D., Makady, A., Schneeweiss, S., Tarricone, R., Wang, S. V., Watkins, J., and Mullins, C. D. (2017), “Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations From the Joint ISPOR-ISPE Special Task Force on Real-World Evidence in Health Care Decision Making,” Pharmacoepidemiology and Drug Safety, 26, 1033–1039. DOI: 10.1002/pds.4297.
   PubMed Web of Science ®Google Scholar
• Broglio, K. R., Quintana, M., Foster, M., Olinger, M., McGlothlin, A., Berry, S. M., Boileau, J.-F., Brezden-Masley, C., Chia, S., and Dent, S. (2016), “Association of Pathologic Complete Response to Neoadjuvant Therapy in HER2-Positive Breast Cancer With Long-Term Outcomes: A Meta-Analysis,” JAMA Oncology, 2, 751–760. DOI: 10.1001/jamaoncol.2015.6113.
   PubMed Web of Science ®Google Scholar
• Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., and Stürmer, T. (2006), “Variable Selection for Propensity Score Models,” American Journal of Epidemiology, 163, 1149–1156. DOI: 10.1093/aje/kwj149.
   PubMed Web of Science ®Google Scholar
• Burke, D. L., Billingham, L. J., Girling, A. J., and Riley, R. D. (2014), “Meta-Analysis of Randomized Phase II Trials to Inform Subsequent Phase III Decisions,” Trials, 15, 346. DOI: 10.1186/1745-6215-15-346.
   PubMed Web of Science ®Google Scholar
• Chen, L., Li, Y., Parekh, R., and Wang, J. (2012), “The Outlook for China’s Medical Products Industry,” Unlocking Pharma Growth, pp. 72–79.
   Google Scholar
• China National Medical Product Administration (2019), “Real-World Evidence to Support Drug Development and Regulatory Decision-Making,” PRC, available at https://www.nmpa.gov.cn/yaopin/ypggtg/ypqtgg/20200107151901190.html.
   Google Scholar
• Cook, J. A., Julious, S. A., Sones, W., Hampson, L. V., Hewitt, C., Berlin, J. A., Ashby, D., Emsley, R., Fergusson, D. A., Walters, S. J., Wilson, E. C. F., Maclennan, G., Stallard, N., Rothwell, J. C., Bland, M., Brown, L., Ramsay, C. R., Cook, A., Armstrong, D., Altman, D., and Vale, L. D. (2018), “DELTA2 Guidance on Choosing the Target Difference and Undertaking and Reporting the Sample Size Calculation for a Randomised Controlled Trial,” Trials, 19, 606. DOI: 10.1186/s13063-018-2884-0.
   PubMed Web of Science ®Google Scholar
• Cox, E., Martin, B. C., Staa, T. V., Garbe, E., Siebert, U., and Johnson, M. L. (2009), “Good Research Practices for Comparative Effectiveness Research: Approaches to Mitigate Bias and Confounding in the Design of Nonrandomized Studies of Treatment Effects Using Secondary Data Sources: The International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force Report—Part II,” Value in Health, 12, 1053–1061.
   PubMed Web of Science ®Google Scholar
• Crump, R. K., Hotz, V. J., Imbens, G. W., and Mitnik, O. A. (2009), “Dealing With Limited Overlap in Estimation of Average Treatment Effects,” Biometrika, 96, 187–199. DOI: 10.1093/biomet/asn055.
   Web of Science ®Google Scholar
• Debray, T. P. A., Moons, K. G. M., Valkenhoef, G. van, Efthimiou, O., Hummel, N., Groenwold, R. H. H., and Reitsma, J. B. (2015), “Get Real in Individual Participant Data (IPD) Meta-Analysis: A Review of the Methodology,” Research Synthesis Methods, 6, 293–309. DOI: 10.1002/jrsm.1160.
   PubMed Web of Science ®Google Scholar
• Debray, T. P., Schuit, E., Efthimiou, O., Reitsma, J. B., Ioannidis, J. P., Salanti, G., and Moons, K. G. (2018), “An Overview of Methods for Network Meta-Analysis Using Individual Participant Data: When Do Benefits Arise?,” Statistical Methods in Medical Research, 27, 1351–1364. DOI: 10.1177/0962280216660741.
   PubMed Web of Science ®Google Scholar
• Deeks, J. J., Higgins, J. P., and Altman, D. G. (2019), “Analysing Data and Undertaking Meta-Analyses,” in Cochrane Handbook for Systematic Reviews of Interventions, eds. J. P. T. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, V. A. Welch, Chichester: Wiley, pp. 241–284.
   Google Scholar
• Dejardin, D., Delmar, P., Warne, C., Patel, K., van Rosmalen, J., and Lesaffre, E. (2018), “Use of a Historical Control Group in a Noninferiority Trial Assessing a New Antibacterial Treatment: A Case Study and Discussion of Practical Implementation Aspects,” Pharmaceutical Statistics, 17, 169–181. DOI: 10.1002/pst.1843.
   PubMed Web of Science ®Google Scholar
• DerSimonian, R., and Laird, N. (1986), “Meta-Analysis in Clinical Trials,” Controlled Clinical Trials, 7, 177–188. DOI: 10.1016/0197-2456(86)90046-2.
   PubMed Web of Science ®Google Scholar
• DerSimonian, R., and Laird, N. (2015), “Meta-Analysis in Clinical Trials Revisited,” Contemporary Clinical Trials, 45, 139–145.
   PubMed Web of Science ®Google Scholar
• Dias, S., Sutton, A. J., Ades, A. E., and Welton, N. J. (2013a), “Evidence Synthesis for Decision Making 2: A Generalized Linear Modeling Framework for Pairwise and Network Meta-analysis of Randomized Controlled Trials,” Medical Decision Making, 33, 607–617. DOI: 10.1177/0272989X12458724.
   PubMed Web of Science ®Google Scholar
• Dias, S., Sutton, A. J., Welton, N. J., and Ades, A. E. (2013b), “Evidence Synthesis for Decision Making 3: Heterogeneity—Subgroups, Meta-Regression, Bias, and Bias-Adjustment,” Medical Decision Making, 33, 618–640. DOI: 10.1177/0272989X13485157.
   PubMed Web of Science ®Google Scholar
• Dias, S., Sutton, A. J., Welton, N. J., and Ades, A. E. (2013c), “Evidence Synthesis for Decision Making 6: Embedding Evidence Synthesis in Probabilistic Cost-Effectiveness Analysis,” Medical Decision Making, 33, 671–678.
   PubMed Web of Science ®Google Scholar
• Dias, S., Welton, N. J., Sutton, A. J., and Ades, A. E. (2013d), “Evidence Synthesis for Decision Making 1: Introduction,” Medical Decision Making, 33, 597–606. DOI: 10.1177/0272989X13487604.
   PubMed Web of Science ®Google Scholar
• Dias, S., Welton, N. J., Sutton, A. J., Caldwell, D. M., Lu, G., and Ades, A. E. (2013e), “Evidence Synthesis for Decision Making 4: Inconsistency in Networks of Evidence Based on Randomized Controlled Trials,” Medical Decision Making, 33, 641–656. DOI: 10.1177/0272989X12455847.
   PubMed Web of Science ®Google Scholar
• Donegan, S., Williamson, P., D’Alessandro, U., Garner, P., and Smith, C. T. (2013), “Combining Individual Patient Data and Aggregate Data in Mixed Treatment Comparison Meta - Analysis: Individual Patient Data May Be Beneficial If Only for a Subset of Trials,” Statistics in Medicine, 32, 914–930. DOI: 10.1002/sim.5584.
   PubMed Web of Science ®Google Scholar
• Duan, Y., Smith, E. P., and Ye, K. (2006), “Using Power Priors to Improve the Binomial Test of Water Quality,” Journal of Agricultural, Biological, and Environmental Statistics, 11, 151–168. DOI: 10.1198/108571106X110919.
   Web of Science ®Google Scholar
• Efthimiou, O., Debray, T. P. A., Valkenhoef, G. van, Trelle, S., Panayidou, K., Moons, K. G. M., Reitsma, J. B., Shang, A., and Salanti, G. (2016), “GetReal in Network Meta-Analysis: A Review of the Methodology,” Research Synthesis Methods, 7, 236–263. DOI: 10.1002/jrsm.1195.
   PubMed Web of Science ®Google Scholar
• Efthimiou, O., Mavridis, D., Debray, T. P. A., Samara, M., Belger, M., Siontis, G. C. M., Leucht, S., Salanti, G., and on behalf of GetReal Work Package 4 (2017), “Combining Randomized and Non-Randomized Evidence in Network Meta-Analysis: Combining Randomized and Non-Randomized Evidence in NMA,” Statistics in Medicine, 36, 1210–1226. DOI: 10.1002/sim.7223.
   PubMed Web of Science ®Google Scholar
• Efthimiou, O., and White, I. R. (2020), “The Dark Side of the Force: Multiplicity Issues in Network Meta-Analysis and How to Address Them,” Research Synthesis Methods, 11, 105–122. DOI: 10.1002/jrsm.1377.
   PubMed Web of Science ®Google Scholar
• Eichler, H. G., Bloechl-Daum, B., Bauer, P., Bretz, F., Brown, J., Hampson, L. V., Honig, P., Krams, M., Leufkens, H., Lim, R., Lumpkin, M. M., Murphy, M. J., Pignatti, F., Posch, M., Schneeweiss, S., Trusheim, M., and Koenig, F. (2016), “‘Threshold-Crossing’: A Useful Way to Establish the Counterfactual in Clinical Trials?,” Clinical Pharmacology & Therapeutics, 100, 699–712.
   PubMed Web of Science ®Google Scholar
• EMA (2017), “Qualification of Novel Methodologies for Medicine Development,” European Medicines Agency, available at https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-advice-protocol-assistance/qualification-novel-methodologies-medicine-development.
   Google Scholar
• EMA (2018), “Regulatory Science to 2025: Strategic Reflection,” available at https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/ema-regulatory-science-2025-strategic-reflection_en.pdf.
   Google Scholar
• FDA (2010), “Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials,” available at https://www.fda.gov/media/71512/download.
   Google Scholar
• FDA (2013a), “Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data Sets,” available at https://www.fda.gov/media/79922/download.
   Google Scholar
• FDA (2013b), “Electronic Source Data in Clinical Investigations,” available at https://www.fda.gov/media/85183/download.
   Google Scholar
• FDA (2017), “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices—Guidance for Industry and Food and Drug Administration Staff,” FDA, available at https://www.fda.gov/media/99447/download.
   Google Scholar
• FDA (2018a), “Use of Electronic Health Record Data in Clinical Investigations Guidance for Industry,” available at https://www.fda.gov/media/97567/download.
   Google Scholar
• FDA (2018b), “Framework for FDA’s Real-World Evidence Program,” FDA, available at https://www.fda.gov/media/120060/download.
   Google Scholar
• FDA (2019a), “Rare Diseases: Natural History Studies for Drug Development Guidance for Industry,” available at https://www.fda.gov/media/122425/download.
   Google Scholar
• FDA (2019b), “Rare Diseases: Common Issues in Drug Development Guidance for Industry (Revision 1),” available at https://www.fda.gov/media/119757/download.
   Google Scholar
• FDA (2019c), “Submitting Documents: Using Real-World Data and Real-World Evidence to FDA for Drugs and Biologics: Guidance for Industry,” available at https://www.fda.gov/media/124795/download.
   Google Scholar
• Ferreira-González, I., Marsal, J. R., Mitjavila, F., Parada, A., Ribera, A., Cascant, P., Soriano, N., Sánchez, P. L., Arós, F., Heras, M., and Bueno, H. (2009), “Patient Registries of Acute Coronary Syndrome: Assessing or Biasing the Clinical Real World Data?,” Circulation: Cardiovascular Quality and Outcomes, 2, 540–547.
   PubMed Web of Science ®Google Scholar
• Fralick, M., Kesselheim, A. S., Avorn, J., and Schneeweiss, S. (2018), “Use of Health Care Databases to Support Supplemental Indications of Approved Medications,” JAMA Internal Medicine, 178, 55–63. DOI: 10.1001/jamainternmed.2017.3919.
   PubMed Web of Science ®Google Scholar
• Franklin, J. M., and Schneeweiss, S. (2017), “When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials?,” Clinical Pharmacology & Therapeutics, 102, 924–933.
   PubMed Web of Science ®Google Scholar
• Friedman, L. M., Furberg, C., DeMets, D. L., Reboussin, D. M., and Granger, C. B. (2010), Fundamentals of Clinical Trials, New York: Springer.
   Google Scholar
• Fu, R., Gartlehner, G., Grant, M., Shamliyan, T., Sedrakyan, A., Wilt, T. J., Griffith, L., Oremus, M., Raina, P., Ismaila, A., Santaguida, P., Lau, J., and Trikalinos, T. A. (2011), “Conducting Quantitative Synthesis When Comparing Medical Interventions: AHRQ and the Effective Health Care Program,” Journal of Clinical Epidemiology, 64, 1187–1197. DOI: 10.1016/j.jclinepi.2010.08.010.
   PubMed Web of Science ®Google Scholar
• Germain, D. P., Oliveira, J. P. Bichet, D. G. Yoo, H.-W. Hopkin, R. J. Lemay, R. Politei, J. Wanner, C. Wilcox, W. R. and Warnock D. G. (2020), “Use of a Rare Disease Registry for Establishing Phenotypic Classification of Previously Unassigned GLA Variants: A Consensus Classification System by a Multispecialty Fabry Disease Genotype–Phenotype Workgroup,” Journal of Medical Genetics, 57, 542–551. DOI: 10.1136/jmedgenet-2019-106467.
   PubMed Web of Science ®Google Scholar
• Ghadessi, M., Tang, R., Zhou, J., Liu, R., Wang, C., Toyoizumi, K., Mei, C., Zhang, L., Deng, C. Q., and Beckman, R. A. (2020), “A Roadmap to Using Historical Controls in Clinical Trials—By Drug Information Association Adaptive Design Scientific Working Group (DIA-ADSWG),” Orphanet Journal of Rare Diseases, 15, 69.
   PubMed Web of Science ®Google Scholar
• Gliklich, R. E., Dreyer, N. A., and Leavy M. B. (2014), Registries for Evaluating Patient Outcomes: A Users Guide (3rd ed.), Rockville, MD: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, available at www.ahrq.gov.
   Google Scholar
• Goring, S., Taylor, A., Müller, K., Li, T. J. J., Korol, E. E., Levy, A. R., and Freemantle, N. (2019), “Characteristics of Non-Randomised Studies Using Comparisons With External Controls Submitted for Regulatory Approval in the USA and Europe: A Systematic Review,” BMJ Open, 9, e024895. DOI: 10.1136/bmjopen-2018-024895.
   PubMed Web of Science ®Google Scholar
• Gould, A. L. (1991), “Another View of Active-Controlled Trials,” Controlled Clinical Trials, 12, 474–485. DOI: 10.1016/0197-2456(91)90008-A.
   PubMedGoogle Scholar
• Gould, A. L. (2002), “Substantial Evidence of Effect,” Journal of Biopharmaceutical Statistics, 12, 53–77.
   PubMedGoogle Scholar
• Greenland, S., Pearl, J., and Robins, J. M. (1999), “Causal Diagrams for Epidemiologic Research,” Epidemiology, 10, 37–48.
   PubMed Web of Science ®Google Scholar
• Hansen, B. B. (2004), “Full Matching in an Observational Study of Coaching for the SAT,” Journal of the American Statistical Association, 99, 609–618. DOI: 10.1198/016214504000000647.
   Web of Science ®Google Scholar
• Hansen, B. B. (2008), “The Prognostic Analogue of the Propensity Score,” Biometrika, 95, 481–488.
   Web of Science ®Google Scholar
• Hatswell, A. J., Baio, G., Berlin, J. A., Irs, A., and Freemantle, N. (2016), “Regulatory Approval of Pharmaceuticals Without a Randomised Controlled Study: Analysis of EMA and FDA Approvals 1999–2014,” BMJ Open, 6, e011666. DOI: 10.1136/bmjopen-2016-011666.
   PubMed Web of Science ®Google Scholar
• Hernán, M. A., Brumback, B., and Robins, J. M. (2001), “Marginal Structural Models to Estimate the Joint Causal Effect of Nonrandomized Treatments,” Journal of the American Statistical Association, 96, 440–448. DOI: 10.1198/016214501753168154.
   Web of Science ®Google Scholar
• Hernán, M. A., and Robins, J. M. (2016), “Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available,” American Journal of Epidemiology, 183, 758–764. DOI: 10.1093/aje/kwv254.
   PubMed Web of Science ®Google Scholar
• Ho, M., van der Laan, M., Lee, H., Chen, J., Lee, K., Fang, Y., He, W., Irony, T., Jiang, Q., Lin, X., Meng, Z., Mishra-Kalyani, P., Rockhold, F., Song, Y., Wang, H., and White, R. (2020), “The Current Landscape in Biostatistics of Real-World Data and Evidence: Causal Inference Frameworks for Study Design and Analysis,” Statistics in Biopharmaceutical Research (to appear).
   Web of Science ®Google Scholar
• Hoaglin, D. C., Hawkins, N., Jansen, J. P., Scott, D. A., Itzler, R., Cappelleri, J. C., Boersma, C., Thompson, D., Larholt, K. M., Diaz, M., and Barrett, A. (2011), “Conducting Indirect-Treatment-Comparison and Network-Meta-Analysis Studies: Report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 2,” Value in Health, 14, 429–437.
   PubMed Web of Science ®Google Scholar
• Hobbs, B. P., Carlin, B. P., Mandrekar, S. J., and Sargent, D. J. (2011), “Hierarchical Commensurate and Power Prior Models for Adaptive Incorporation of Historical Information in Clinical Trials,” Biometrics, 67, 1047–1056. DOI: 10.1111/j.1541-0420.2011.01564.x.
   PubMed Web of Science ®Google Scholar
• Hobbs, B. P., Carlin, B. P., and Sargent, D. J. (2013), “Adaptive Adjustment of the Randomization Ratio Using Historical Control Data,” Clinical Trials, 10, 430–440. DOI: 10.1177/1740774513483934.
   PubMed Web of Science ®Google Scholar
• Hong, H., Fu, H., and Carlin, B. P. (2018), “Power and Commensurate Priors for Synthesizing Aggregate and Individual Patient Level Data in Network Meta-Analysis,” Journal of the Royal Statistical Society, Series C, 67, 1047–1069. DOI: 10.1111/rssc.12275.
   Web of Science ®Google Scholar
• Hummel, N., Debray, T. P. A., Didden, E.-M., Egger, M., Fletcher, C., Moons, K. G. M., Reitsma, B., Ruffieux, Y., Salanti, G., and van Valkenhoef, G. (2016), “Work Package 4 Methodological Guidance, Recommendations and Illustrative Case Studies for (Network) Meta-Analysis and Modelling to Predict Real-World Effectiveness Using Individual Participant and/or Aggregate Data,” 78.
   Google Scholar
• Hutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., Mulrow, C., Catalá-López, F., Gøtzsche, P. C., Dickersin, K., Boutron, I., Altman, D. G., and Moher, D. (2015), “The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-Analyses of Health Care Interventions: Checklist and Explanations,” Annals of Internal Medicine, 162, 777. DOI: 10.7326/M14-2385.
   PubMed Web of Science ®Google Scholar
• ICH (2001), “E10 Choice of Control Group and Related Issues in Clinical Trials,” available at https://www.fda.gov/media/71349/download.
   Google Scholar
• ICH (2019a), “ICH E9 (R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials,” available at https://www.fda.gov/media/108698/download.
   Google Scholar
• ICH (2019b), “Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials E9(R1),” International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use.
   Google Scholar
• Imai, K., King, G., and Stuart, E. A. (2008), “Misunderstandings Between Experimentalists and Observationalists About Causal Inference,” Journal of the Royal Statistical Society, Series A, 171, 481–502. DOI: 10.1111/j.1467-985X.2007.00527.x.
   Web of Science ®Google Scholar
• Imbens, G. W., and Rubin, D. B. (2015), Causal Inference in Statistics, Social, and Biomedical Sciences, New York: Cambridge University Press.
   Google Scholar
• International Committee of Medical Journal Editors (2019), “Recommendations on Clinical Trials Regisration and Data Sharing,” available at http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html.
   Google Scholar
• Jansen, J. P. (2012), “Network Meta-Analysis of Individual and Aggregate Level Data,” Research Synthesis Methods, 3, 177–190. DOI: 10.1002/jrsm.1048.
   PubMed Web of Science ®Google Scholar
• Jansen, J. P., Fleurence, R., Devine, B., Itzler, R., Barrett, A., Hawkins, N., Lee, K., Boersma, C., Annemans, L., and Cappelleri, J. C. (2011), “Interpreting Indirect Treatment Comparisons and Network Meta-Analysis for Health-Care Decision Making: Report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1,” Value in Health, 14, 417–428. DOI: 10.1016/j.jval.2011.04.002.
   PubMed Web of Science ®Google Scholar
• Jansen-van der Weide, M. C., Gaasterland, C. M. Roes, K. C. Pontes, C. Vives, R. Sancho, A. Nikolakopoulos, S. Vermeulen, E. and van der Lee J. H. (2018), “Rare Disease Registries: Potential Applications Towards Impact on Development of New Drug Treatments,” Orphanet Journal of Rare Diseases 13, 1–11. DOI: 10.1186/s13023-018-0836-0.
   PubMed Web of Science ®Google Scholar
• Jenkins, D., Bujkiewicz, S., Martina, R., Dequen, P., and Abrams, K. R. (2018), “Methods for the Inclusion of Real World Evidence in Network Meta-Analysis,” arXiv no. 1805.06839.
   Google Scholar
• Johnson, M. L., Crown, W., Martin, B. C., Dormuth, C. R., and Siebert, U. (2009), “Good Research Practices for Comparative Effectiveness Research: Analytic Methods to Improve Causal Inference From Nonrandomized Studies of Treatment Effects Using Secondary Data Sources: The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part III,” Value in Health, 12, 1062–1073.
   PubMed Web of Science ®Google Scholar
• Kanaya, A. M., Grady, D., and Barrett-Connor, E. (2002), “Explaining the Sex Difference in Coronary Heart Disease Mortality Among Patients With Type 2 Diabetes Mellitus: A Meta-Analysis,” Archives of Internal Medicine, 162, 1737–1745. DOI: 10.1001/archinte.162.15.1737.
   PubMed Web of Science ®Google Scholar
• Koch, B., Vock, D. M., and Wolfson, J. (2018), “Covariate Selection With Group Lasso and Doubly Robust Estimation of Causal Effects,” Biometrics, 74, 8–17. DOI: 10.1111/biom.12736.
   PubMed Web of Science ®Google Scholar
• Kopp-Schneider, A., Calderazzo, S., and Wiesenfarth, M. (2020), “Power Gains by Using External Information in Clinical Trials Are Typically Not Possible When Requiring Strict Type I Error Control,” Biometrical Journal, 62, 361–374. DOI: 10.1002/bimj.201800395.
   PubMed Web of Science ®Google Scholar
• Laws, A., Kendall, R., and Hawkins, N. (2014), “A Comparison of National Guidelines for Network Meta-Analysis,” Value in Health, 17, 642–654. DOI: 10.1016/j.jval.2014.06.001.
   PubMed Web of Science ®Google Scholar
• Leahy, J., Thom, H., Jansen, J. P., Gray, E., O’Leary, A., White, A., and Walsh, C. (2019), “Incorporating SingleArm Evidence Into a Network MetaAnalysis Using Aggregate Level Matching: Assessing the Impact,” Statistics in Medicine, sim.8139.
   PubMedGoogle Scholar
• Levenson, M., He, W., Chen, J., Fang, Y., Faries, D., Goldstein, B. A., Ho, M., Lee, K., Mishra-Kalyani, P., Rockhold, F., Wang, H., and Zink, R. (2020), “The Current Landscape in Biostatistics of the Use of Real-World Data and Evidence for Medical Product Development: General Considerations.”
   Google Scholar
• Lewis, C. J., Sarkar, S., Zhu, J., and Carlin, B. P. (2019), “Borrowing From Historical Control Data in Cancer Drug Development: A Cautionary Tale and Practical Guidelines,” Statistics in Biopharmaceutical Research, 11, 67–78. DOI: 10.1080/19466315.2018.1497533.
   PubMed Web of Science ®Google Scholar
• Li, F., Morgan, K. L., and Zaslavsky, A. M. (2018), “Balancing Covariates via Propensity Score Weighting,” Journal of the American Statistical Association, 113, 390–400. DOI: 10.1080/01621459.2016.1260466.
   Web of Science ®Google Scholar
• Liebeskind, D. S. (2015), “Innovative Interventional and Imaging Registries: Precision Medicine in Cerebrovascular Disorders,” Interventional Neurology, 4, 5–17. DOI: 10.1159/000438773.
   PubMed Web of Science ®Google Scholar
• Lim, J., Walley, R., Yuan, J., Liu, J., Dabral, A., Best, N., Grieve, A., Hampson, L., Wolfram, J., Woodward, P., Yong, F., Zhang, X., and Bowen, E. (2018), “Minimizing Patient Burden Through the Use of Historical Subject-Level Data in Innovative Confirmatory Clinical Trials: Review of Methods and Opportunities,” Therapeutic Innovation & Regulatory Science, 52, 546–559.
   PubMed Web of Science ®Google Scholar
• Lu, G., and Ades, A. (2004), “Combination of Direct and Indirect Evidence in Mixed Treatment Comparisons,” Statistics in Medicine, 23, 3105–3124. DOI: 10.1002/sim.1875.
   PubMed Web of Science ®Google Scholar
• Madigan, D., Ryan, P. B., Schuemie, M., Stang, P. E., Overhage, J. M., Hartzema, A. G., Suchard, M. A., DuMouchel, W., and Berlin, J. A. (2013), “Evaluating the Impact of Database Heterogeneity on Observational Study Results,” American Journal of Epidemiology, 178, 645–651. DOI: 10.1093/aje/kwt010.
   PubMed Web of Science ®Google Scholar
• Makuch, R. W., and Simon, R. M. (1980), “Sample Size Considerations for Non-Randomized Comparative Studies,” Journal of Chronic Diseases, 33, 175–181. DOI: 10.1016/0021-9681(80)90017-X.
   PubMedGoogle Scholar
• Mao, H., Li, L., Yang, W., and Shen, Y. (2018), “On the Propensity Score Weighting Analysis With Survival Outcome: Estimands, Estimation, and Inference,” Statistics in Medicine, 37, 3745–3763. DOI: 10.1002/sim.7839.
   PubMed Web of Science ®Google Scholar
• Martina, R., Jenkins, D., Bujkiewicz, S., Dequen, P., Abrams, K., and GetReal Workpackage 1 (2018), “The Inclusion of Real World Evidence in Clinical Development Planning,” Trials, 19, 468. DOI: 10.1186/s13063-018-2769-2.
   PubMed Web of Science ®Google Scholar
• Morton, S., Murad, M., and O’Connor, E. (2018), “AHRQ Methods for Effective Health Care,” in Quantitative Synthesis—An Update. Methods Guide for Effectiveness and Comparative Effectiveness Reviews, Rockville, MD: Agency for Healthcare Research and Quality (US).
   Google Scholar
• Neelon, B. H., O’Malley, A. J., and Normand, S.-L. T. (2010), “A Bayesian Model for Repeated Measures Zero-Inflated Count Data With Application to Outpatient Psychiatric Service Use,” Statistical Modelling, 10, 421–439. DOI: 10.1177/1471082X0901000404.
   PubMed Web of Science ®Google Scholar
• Neuenschwander, B., Branson, M., and Spiegelhalter, D. J. (2009), “A Note on the Power Prior,” Statistics in Medicine, 28, 3562–3566. DOI: 10.1002/sim.3722.
   PubMed Web of Science ®Google Scholar
• Neuenschwander, B., Capkun-Niggli, G., Branson, M., and Spiegelhalter, D. J. (2010), “Summarizing Historical Information on Controls in Clinical Trials,” Clinical Trials, 7, 5–18. DOI: 10.1177/1740774509356002.
   PubMed Web of Science ®Google Scholar
• Nikolakopoulou, A., Trelle, S., Sutton, A. J., Egger, M., and Salanti, G. (2019), “Synthesizing Existing Evidence to Design Future Trials: Survey of Methodologists from European Institutions,” Trials, 20, 334. DOI: 10.1186/s13063-019-3449-6.
   PubMed Web of Science ®Google Scholar
• PCORI (2019), “PCORI Methodology Standards,” available at https://www.pcori.org/research-results/about-our-research/research-methodology/methodology-standards-academic-curriculum-5.
   Google Scholar
• Pearl, J. (2009), “Causal Inference in Statistics: An Overview,” Statistics Surveys, 3, 96–146. DOI: 10.1214/09-SS057.
   Google Scholar
• Phelan, M., Bhavsar, N. A., and Goldstein, B. A. (2017), “Illustrating Informed Presence Bias in Electronic Health Records Data: How Patient Interactions With a Health System Can Impact Inference,” EGEMS, 5, 22. DOI: 10.5334/egems.243.
   PubMedGoogle Scholar
• Pocock, S. J. (1976), “The Combination of Randomized and Historical Controls in Clinical Trials,” Journal of Chronic Diseases, 29, 175–188. DOI: 10.1016/0021-9681(76)90044-8.
   PubMedGoogle Scholar
• Riley, R. D., Jackson, D., Salanti, G., Burke, D. L., Price, M., Kirkham, J., and White, I. R. (2017), “Multivariate and Network Meta-Analysis of Multiple Outcomes and Multiple Treatments: Rationale, Concepts, and Examples,” BMJ, 358, j3932.
   Web of Science ®Google Scholar
• Robins, J. M., and Finkelstein, D. M. (2000), “Correcting for Noncompliance and Dependent Censoring in an AIDS Clinical Trial With Inverse Probability of Censoring Weighted (IPCW) Log-Rank Tests,” Biometrics, 56, 779–788. DOI: 10.1111/j.0006-341x.2000.00779.x.
   PubMed Web of Science ®Google Scholar
• Rosenbaum, P. R. (1991), “A Characterization of Optimal Designs for Observational Studies,” Journal of the Royal Statistical Society, Series B, 53, 597–610. DOI: 10.1111/j.2517-6161.1991.tb01848.x.
   Google Scholar
• Rosenbaum, P. R., and Rubin, D. B. (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects,” Biometrika, 70, 41–55. DOI: 10.1093/biomet/70.1.41.
   Web of Science ®Google Scholar
• Rosenbaum, P. R., and Rubin, D. B. (1985), “Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score,” The American Statistician, 39, 33–38.
   Web of Science ®Google Scholar
• Rubin, D. B. (1973), “Matching to Remove Bias in Observational Studies,” Biometrics, 29, 159–183. DOI: 10.2307/2529684.
   Web of Science ®Google Scholar
• Rubin, D. B. (1976), “Inference and Missing Data,” Biometrika, 63, 581–592.
   Web of Science ®Google Scholar
• Rubin, D. B. (2001a), “Estimating the Causal Effects of Smoking,” Statistics in Medicine, 20, 1395–1414.
   PubMed Web of Science ®Google Scholar
• Rubin, D. B. (2001b), “Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation,” Health Services and Outcomes Research Methodology, 2, 169–188.
   Google Scholar
• Rubin, D. B. (2007), “The Design Versus the Analysis of Observational Studies for Causal Effects: Parallels With the Design of Randomized Trials,” Statistics in Medicine, 26, 20–36.
   PubMed Web of Science ®Google Scholar
• Rubin, D. B. (2008), “For Objective Causal Inference, Design Trumps Analysis,” The Annals of Applied Statistics, 2, 808–840.
   Web of Science ®Google Scholar
• Rubin, D. B., and Thomas, N. (1996), “Matching Using Estimated Propensity Scores: Relating Theory to Practice,” Biometrics, 52, 249–264. DOI: 10.2307/2533160.
   PubMed Web of Science ®Google Scholar
• Rubin, D. B. (2000), “Combining Propensity Score Matching With Additional Adjustments for Prognostic Covariates,” Journal of the American Statistical Association, 95, 573–585.
   Web of Science ®Google Scholar
• Salanti, G., Giovane, C. D., Chaimani, A., Caldwell, D. M., and Higgins, J. P. T. (2014), “Evaluating the Quality of Evidence From a Network Meta-Analysis,” PLOS ONE, 9, e99682. DOI: 10.1371/journal.pone.0099682.
   PubMed Web of Science ®Google Scholar
• Schmidli, H., Gsteiger, S., Roychoudhury, S., O’Hagan, A., Spiegelhalter, D., and Neuenschwander, B. (2014), “Robust Meta-Analytic-Predictive Priors in Clinical Trials With Historical Control Information,” Biometrics, 70, 1023–1032. DOI: 10.1111/biom.12242.
   PubMed Web of Science ®Google Scholar
• Schmidli, H., Häring, D. A., Thomas, M., Cassidy, A., Weber, S., and Bretz, F. (2020), “Beyond Randomized Clinical Trials: Use of External Controls,” Clinical Pharmacology & Therapeutics, 107, 806–816.
   PubMed Web of Science ®Google Scholar
• Shortreed, S. M., and Ertefaie, A. (2017), “Outcome-Adaptive Lasso: Variable Selection for Causal Inference,” Biometrics, 73, 1111–1122. DOI: 10.1111/biom.12679.
   PubMed Web of Science ®Google Scholar
• Spiegelhalter, D. J. (2004), “Incorporating Bayesian Ideas Into Health-Care Evaluation,” Statistical Science, 19, 156–174. DOI: 10.1214/088342304000000080.
   Web of Science ®Google Scholar
• Steiner, P. M., and Cook, D. (2013), “Matching and Propensity Scores,” The Oxford Handbook of Quantitative Methods, 1, 237–259.
   Google Scholar
• Stuart, E. A. (2010), “Matching Methods for Causal Inference: A Review and a Look Forward,” Statistical Science, 25, 1–21. DOI: 10.1214/09-STS313.
   PubMed Web of Science ®Google Scholar
• Stuart, E. A., Lee, B. K., and Leacy, F. P. (2013), “Prognostic Score-Based Balance Measures Can Be a Useful Diagnostic for Propensity Score Methods in Comparative Effectiveness Research,” Journal of Clinical Epidemiology, 66, S84–S90.e1. DOI: 10.1016/j.jclinepi.2013.01.013.
   PubMed Web of Science ®Google Scholar
• Stuart, E. A., and Rubin, D. B. (2008), “Matching With Multiple Control Groups With Adjustment for Group Differences,” Journal of Educational and Behavioral Statistics, 33, 279–306. DOI: 10.3102/1076998607306078.
   Web of Science ®Google Scholar
• Sutton, A. J., Cooper, N. J., Jones, D. R., Lambert, P. C., Thompson, J. R., and Abrams, K. R. (2007), “Evidence-Based Sample Size Calculations Based Upon Updated Meta-Analysis,” Statistics in Medicine, 26, 2479– 2500. DOI: 10.1002/sim.2704.
   PubMed Web of Science ®Google Scholar
• Thorlund, K., Dron, L., Park, J. J., and Mills, E. J. (2020), “Synthetic and External Controls in Clinical Trials—A Primer for Researchers,” Clinical Epidemiology, 12, 457–467. DOI: 10.2147/CLEP.S242097.
   PubMed Web of Science ®Google Scholar
• VanderWeele, T. J. (2019), “Principles of Confounder Selection,” European Journal of Epidemiology, 34, 211–219. DOI: 10.1007/s10654-019-00494-6.
   PubMed Web of Science ®Google Scholar
• Viele, K., Berry, S., Neuenschwander, B., Amzal, B., Chen, F., Enas, N., Hobbs, B., Ibrahim, J. G., Kinnersley, N., Lindborg, S., Micallef, S., Roychoudhury, S., and Thompson, L. (2014), “Use of Historical Control Data for Assessing Treatment Effects in Clinical Trials,” Pharmaceutical Statistics, 13, 41–54. DOI: 10.1002/pst.1589.
   PubMed Web of Science ®Google Scholar
• Weinger, M. B., Slagle, J., Jain, S., and Ordonez, N. (2003), “Retrospective Data Collection and Analytical Techniques for Patient Safety Studies,” Journal of Biomedical Informatics, 36, 106–119. DOI: 10.1016/j.jbi.2003.08.002.
   PubMed Web of Science ®Google Scholar
• Weng, C., Li, Y., Ryan, P., Zhang, Y., Liu, F., Gao, J., Bigger, J. T., and Hripcsak, G. (2014), “A Distribution-Based Method for Assessing the Differences Between Clinical Trial Target Populations and Patient Populations in Electronic Health Records,” Applied Clinical Informatics, 5, 463–479. DOI: 10.4338/ACI-2013-12-RA-0105.
   PubMed Web of Science ®Google Scholar
• Wittes, J. (2002), “Sample Size Calculations for Randomized Controlled Trials,” Epidemiologic Reviews, 24, 39–53. DOI: 10.1093/epirev/24.1.39.
   PubMed Web of Science ®Google Scholar
• Yuan, J., Liu, J., Zhu, R., Lu, Y., and Palm, U. (2019), “Design of Randomized Controlled Confirmatory Trials Using Historical Control Data to Augment Sample Size for Concurrent Controls,” Journal of Biopharmaceutical Statistics, 29, 558–573. DOI: 10.1080/10543406.2018.1559853.
   PubMed Web of Science ®Google Scholar
• Yue, L. Q., Lu, N., and Xu, Y. (2014), “Designing Premarket Observational Comparative Studies Using Existing Data as Controls: Challenges and Opportunities,” Journal of Biopharmaceutical Statistics, 24, 994–1010. DOI: 10.1080/10543406.2014.926367.
   PubMed Web of Science ®Google Scholar
• Zhang, S., Cao, J., and Ahn, C. (2010), “Calculating Sample Size in Trials Using Historical Controls,” Clinical Trials, 7, 343–353. DOI: 10.1177/1740774510373629.
   PubMed Web of Science ®Google Scholar
• Zhao, P., Su, X., Ge, T., and Fan, J. (2016), “Propensity Score and Proximity Matching Using Random Forest,” Contemporary Clinical Trials, 47, 85–92. DOI: 10.1016/j.cct.2015.12.012.
   PubMed Web of Science ®Google Scholar
• Zhu, H., Zhang, S., and Ahn, C. (2016), “Sample Size Considerations for Historical Control Studies With Survival Outcomes,” Journal of Biopharmaceutical Statistics, 26, 657–671. DOI: 10.1080/10543406.2015.1052495.
   PubMed Web of Science ®Google Scholar