Statistical Consideration for Fit-for-Use Real-World Data to Support Regulatory Decision Making in Drug Development

References

• Agarwal, V., Podchiyska, T., Banda, J. M., Goel, V., Leung, T. I., Minty, E. P., Sweeney, T. E., Gyang, E., and Shah, N. H. (2016), “Learning Statistical Models of Phenotypes using Noisy Labeled Training Data,” Journal of the American Medical Informatics Association, 23, 1166–1173. DOI: 10.1093/jamia/ocw028.
   PubMed Web of Science ®Google Scholar
• Arondekar, B., Duh, M. S., Bhak, R. H., DerSarkissian, M., Huynh, L., Wang, K., Wojciehowski, J., Wu, M., Wornson, B., Niyazov, A., and Demetri, G. D. (2021), “Real-World Evidence in Support of Oncology Product Registration: A Systematic Review of New Drug Application and Biologics License Application Approvals from 2015–2020,” Clinical Cancer Research, 28, 27–35, available at https://clincancerres.aacrjournals.org/content/28/1/27.full.
   PubMed Web of Science ®Google Scholar
• Banda, J. M., Seneviratne, M., Hernandez-Boussard, T., and Shah, N. H. (2018), “Advances in Electronic Phenotyping: from Rule-based Definitions to Machine Learning Models,” Annual Review of Biomedical Data Science, 1, 53–68. DOI: 10.1146/annurev-biodatasci-080917-013315.
   PubMedGoogle Scholar
• Berkson, J. (1946), “Limitations of the Application of Fourfold Table Analysis to Hospital Data,” Biometrics Bulletin, 2, 47–53. DOI: 10.2307/3002000.
   Google Scholar
• BIO (2019), “Incorporating Real-world Evidence Within the Label of an FDA-approved Drug: Perspectives from BIO Membership,” available at https://www.bio.org/sites/default/files/legacy/bioorg/docs/BIO%20White%20paper%20%20-%20Incorporating%20RWE%20Within%20the%20Label_FINAL%202019.pdf.
   Google Scholar
• Carroll, R. J., Eyler, A. E., and Denny, J. C. (2011), “Naïve Electronic Health Record Phenotype Identification for Rheumatoid Arthritis,” in AMIA Annual Symposium Proceedings, American Medical Informatics Association, p. 189.
   Google Scholar
• Cutrona, S. L., Toh, S., Iyer, A., Foy, S., Daniel, G. W., Nair, V. P., Ng, D., Butler, M. G., Boudreau, D., Forrow, S., Goldberg, R., Gore, J., McManus, D., Racoosin, J. A., and Gurwitz, J. H. (2013), “Validation of Acute Myocardial Infarction in the Food and Drug Administration’s Mini-Sentinel Program,” Pharmacoepidemiology and Drug Safety, 22, 40–54. DOI: 10.1002/pds.3310.
   PubMed Web of Science ®Google Scholar
• Desai, R. J., Levin, R., Lin, K. J., and Patorno, E. (2020), “Bias Implications of Outcome Misclassification in Observational Studies Evaluating Association Between Treatments and All-Cause or Cardiovascular Mortality Using Administrative Claims,” Journal of the American Heart Association, 9, e016906. DOI: 10.1161/JAHA.120.016906.
   PubMed Web of Science ®Google Scholar
• Dreyer, N. A., Velentgas, P., Westrich, K., and Dubois, R. (2014), “The GRACE Checklist for Rating the Quality of Observational Studies of Comparative Effectiveness: A Tale of Hope and Caution,” Journal of Managed Care & Specialty Pharmacy, 20, 301–308.
   PubMed Web of Science ®Google Scholar
• Duke Margolis (2018), “Characterizing RWD Quality and Relevancy for Regulatory Purposes,” Duke-Margolis Center for Health Policy.
   Google Scholar
• Duke Margolis - (2019a), “Determining Real-world Data’s Fitness for Use and the Role of Reliability. Duke-Margolis Center for Health Policy,” Duke-Margolis Center for Health Policy.
   Google Scholar
• Duke Margolis - (2019b), “Unpacking Real-World Data Curation: Principles and Best Practices to Support Transparency and Quality,” available at https://healthpolicy.duke.edu/sites/default/files/2020-03/data_curation_master_slide_deck_2019_02_01_1000_hours.pdf.
   Google Scholar
• FARXIGA (2020), “Farxiga Granted Fast Track Designation in the US for Heart Failure following Acute Myocardial Infarction Leveraging an Innovative Registry-based Trial Design,” available at https://www.astrazeneca.com/media-centre/press-releases/2020/farxiga-granted-fast-track-designation-in-the-us-for-heart-failure-following-acute-myocardial-infarction-leveraging-an-innovative-registry-based-trial-design.html.
   Google Scholar
• FDA (2018), “Rare Diseases: Early Drug Development and the Role of Pre-IND Meetings,” available at https://www.fda.gov/media/117322/download.
   Google Scholar
• FDA - (2021a), “Real-World Data: Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products,” available at https://www.fda.gov/media/152503/download.
   Google Scholar
• FDA - (2021b), “Real-World Data: Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products,” available at https://www.fda.gov/media/154449/download.
   Google Scholar
• FDA - (2021c), “Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products,” available at https://www.fda.gov/media/154714/download.
   Google Scholar
• FDA - (2021d), “Data Standards for Drug and Biological Product Submissions Containing Real-World Data,” available at https://www.fda.gov/media/153341/download.
   Google Scholar
• Gehrmann, S., Dernoncourt, F., Li, Y., Carlson, E. T., Wu, J. T., Welt, J., Foote, J., Moseley, E. T., Grant, D. W., Tyler, P. D., and Celi, L. A. (2018), “Comparing Deep Learning and Concept Extraction based Methods for Patient Phenotyping from Clinical Narratives,” PLOS One, 13, e0192360. DOI: 10.1371/journal.pone.0192360.
   PubMed Web of Science ®Google Scholar
• Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018), “Potential Biases in Machine Learning Algorithms using Electronic Health Record Data,” JAMA Internal Medicine, 178, 1544–1547.
   PubMed Web of Science ®Google Scholar
• Gibson, T. B., Nguyen, M. D., Burrell, T., Yoon, F., Wong, J., Dharmarajan, S., Ouellet-Hellstrom, R., Hua, W., Ma, Y., Baro, E., Bloemers, S., Pack, C., Kennedy, A., Toh, S., and Ball, R. (2021), “Electronic Phenotyping of Health Outcomes of Interest using a Linked Claims-electronic Health Record Database: Findings from a Machine Learning Pilot Project,” Journal of the American Medical Informatics Association, 28, 1507–1517. DOI: 10.1093/jamia/ocab036.
   PubMed Web of Science ®Google Scholar
• Goldstein, B. A., Phelan, M., Pagidipati, N. J., and Peskoe, S. B. (2019), “How and When Informative Visit Processes Can Bias Inference When Using Electronic Health Records Data for Clinical Research,” Journal of the American Medical Informatics Association, 26, 1609–1617. DOI: 10.1093/jamia/ocz148.
   PubMed Web of Science ®Google Scholar
• Greenland, S. (1996), “Basic Methods for Sensitivity Analysis of Biases,” International Journal of Epidemiology, 25, 1107–1116. DOI: 10.1093/ije/25.6.1107.
   PubMed Web of Science ®Google Scholar
• He, W., Fang, Y., Wang, H., and Chan, I. (2021), “Applying Quantitative Approaches in the Use of RWE in Clinical Development and Life-Cycle Management,” Statistics in Biopharmaceutical Research, available at DOI: 10.1080/19466315.2021.1927827..
   Web of Science ®Google Scholar
• Ho, J. C., Ghosh, J., and Sun, J. (2014), “Marble: High-throughput Phenotyping from Electronic Health Records via Sparse Nonnegative Tensor Factorization,” in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 115–124.
   Google Scholar
• HRSA (2021), “U.S. Scientific Registry of Transplant Recipients (SRTR),” Health Resource and Services Administration, available at https://srtr.transplant.hrsa.gov/.
   Google Scholar
• Kalincik, T., Kuhle, J., Pucci, E., Rojas, J. I., Tsolaki, M., Sirbu, C.-A., Slee, M., Butzkueven, H., Group, Msb. S. L., and Group, Msb. S. (2017), “Data Quality Evaluation for Observational Multiple Sclerosis Registries,” Multiple Sclerosis, 23, 647–655. DOI: 10.1177/1352458516662728.
   PubMed Web of Science ®Google Scholar
• Keogh, R. H., Shaw, P. A., Gustafson, P., Carroll, R. J., Deffner, V., Dodd, K. W., Küchenhoff, H., Tooze, J. A., Wallace, M. P., Kipnis, V., and Freedman, L. S. (2020), “STRATOS Guidance Document on Measurement Error and Misclassification of Variables in Observational Epidemiology: Part 1—Basic Theory and Simple Methods of Adjustment,” Statistics in Medicine, 39, 2197–2231. DOI: 10.1002/sim.8532.
   PubMed Web of Science ®Google Scholar
• Kern, E. F., Maney, M., Miller, D. R., Tseng, C.-L., Tiwari, A., Rajan, M., Aron, D., and Pogach, L. (2006), “Failure of ICD-9-CM Codes to Identify Patients with Comorbid Chronic Kidney Disease in Diabetes,” Health Services Research, 41, 564–580. DOI: 10.1111/j.1475-6773.2005.00482.x.
   PubMed Web of Science ®Google Scholar
• Kho, A. N., Hayes, M. G., Rasmussen-Torvik, L., Pacheco, J. A., Thompson, W. K., Armstrong, L. L., Denny, J. C., Peissig, P. L., Miller, A. W., Wei, W.-Q., Bielinski, S. J., Chute, C. G., Leibson, C. L., Jarvik, G. P., Crosslin, D. R., Carlson, C. S., Newton, K. M., Wolf, W. A., Chisholm, R. L., and Lowe, W. L. (2012), “Use of Diverse Electronic Medical Record Systems to Identify Genetic Risk for Type 2 Diabetes within a Genome-wide Association Study,” Journal of the American Medical Informatics Association, 19, 212–218. DOI: 10.1136/amiajnl-2011-000439.
   PubMed Web of Science ®Google Scholar
• Lash, T. L., Fox, M. P., Cooney, D., Lu, Y., and Forshee, R. A. (2016), “Quantitative Bias Analysis in Regulatory Settings,” American Journal of Public Health, 106, 1227–1230. DOI: 10.2105/AJPH.2016.303199.
   PubMed Web of Science ®Google Scholar
• Levenson, M., He, W., Chen, J., Fang, Y., Faries, D., Goldstein, B. A., Ho, M., Lee, K., Mishra-Kalyani, P., Rockhold, F., Wang, H., and Zink, R. (2021), “Biostatistical Considerations When Using RWD and RWE in Clinical Studies for Regulatory Purposes: A Landscape Assessment,” Statistics in Biopharmaceutical Research, available at https://doi.org/10.1080/19466315.2021.1883473.
   Google Scholar
• Lian, Q., Hodges, J. S., MacLehose, R., and Chu, H. (2019), “A Bayesian Approach for Correcting Exposure Misclassification in Meta-Analysis,” Statistics in Medicine, 38, 115–130. DOI: 10.1002/sim.7969.
   PubMed Web of Science ®Google Scholar
• Lim, J., Walley, R., Yuan, J., Liu, J., Dabral, A., Best, N., Grieve, A., Hampson, L., Wolfram, J., Woodward, P., Yong, F., Zhang, X., and Bowen, E. (2018), “Minimizing Patient Burden Through the Use of Historical Subject-level Data in Innovative Confirmatory Clinical Trials: Review of Methods and Opportunities,” Therapeutic Innovation & Regulatory Science, 52, 546–559.
   PubMed Web of Science ®Google Scholar
• Mello, M. M., Lieou, V., and Goodman, S. N. (2018), “Clinical Trial Participants’ Views of the Risks and Benefits of Data Sharing,” The New England Journal of Medicine, 378, 2202–2211. DOI: 10.1056/NEJMsa1713258.
   PubMed Web of Science ®Google Scholar
• Menikoff, J., Kaneshiro, J., and Pritchard, I. (2017), “The Common Rule,” The New England Journal of Medicine, 376, 613–615. DOI: 10.1056/NEJMp1700736.
   PubMed Web of Science ®Google Scholar
• MONJUVI (2020), “Drug Approval Package: MONJUVI,” available at https://www.accessdata.fda.gov/drugsatfda_docs/nda/2020/761163Orig1s000TOC.cfm.
   Google Scholar
• NASEM. (2019), Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series, Washington DC: The National Academies Press, available at DOI: 10.17226/25352..
   Google Scholar
• Newton, K. M., Peissig, P. L., Kho, A. N., Bielinski, S. J., Berg, R. L., Choudhary, V., Basford, M., Chute, C. G., Kullo, I. J., Li, R., Pacheco, J. A., Rasmussen, L. V., Spangler, L., and Denny, J. C. (2013), “Validation of Electronic Medical Record-based Phenotyping Algorithms: Results and Lessons Learned from the eMERGE Network,” Journal of the American Medical Informatics Association, 20, e147–e154. DOI: 10.1136/amiajnl-2012-000896.
   PubMed Web of Science ®Google Scholar
• Pacheco, J. A., Rasmussen, L. V., Kiefer, R. C., Campion, T. R., Speltz, P., Carroll, R. J., Stallings, S. C., Mo, H., Ahuja, M., Jiang, G., LaRose, E. R., Peissig, P. L., Shang, N., Benoit, B., Gainer, V. S., Borthwick, K., Jackson, K. L., Sharma, A., Wu, A. Y., Kho, A. N., Roden, D. M., Pathak, J., Denny, J. C., and Thompson, W. K. (2018), “A Case Study Evaluating the Portability of an Executable Computable Phenotype Algorithm across Multiple Institutions and Electronic Health Record Environments,” Journal of the American Medical Informatics Association, 25, 1540–1546. DOI: 10.1093/jamia/ocy101.
   PubMed Web of Science ®Google Scholar
• Petersen, J. M., Ranker, L. R., Barnard-Mayers, R., MacLehose, R. F., and Fox, M. P. (2021), “A Systematic Review of Quantitative Bias Analysis Applied to Epidemiological Research,” International Journal of Epidemiology, 50, 1708–1730. DOI: 10.1093/ije/dyab061.
   PubMed Web of Science ®Google Scholar
• Pratt, N. L., Mack, C. D., Meyer, A. M., Davis, K. J., Hammill, B. G., Hampp, C., Setoguchi, S., Raman, S. R., Chun, D. S., Stürmer, T., and Lund, J. L. (2020), “Data Linkage in Pharmacoepidemiology: A Call for Figorous Evaluation and Reporting,” Pharmacoepidemiology and Drug Safety, 29, 9–17. DOI: 10.1002/pds.4924.
   PubMed Web of Science ®Google Scholar
• Rivera, D. R., Gokhale, M. N., Reynolds, M. W., Andrews, E. B., Chun, D., Haynes, K., Jonsson-Funk, M. L., Lynch, K. E., Lund, J. L., Strongman, H., Bhullar, H., and Raman, S. R. (2020), “Linking Electronic Health Data in Pharmacoepidemiology: Appropriateness and Feasibility,” Pharmacoepidemiology and Drug Safety, 29, 18–29. DOI: 10.1002/pds.4918.
   PubMed Web of Science ®Google Scholar
• Rockhold, F., Bromley, C., Wagner, E. K., and Buyse, M. (2019), “Open Science: The Open Clinical Trials Data Journey,” Clinical Trials, 16, 539–546. DOI: 10.1177/1740774519865512.
   PubMed Web of Science ®Google Scholar
• Rockhold, F. W., and Goldstein, B. A. (2020), “Pragmatic raNdomized Trials Using Claims or Electronic Health Record Data,” in Principles and Practice of Clinical Trials, eds. S. Piantadosi and C. L. Meinert, pp. 1–11, Cham: Springer.
   Google Scholar
• Rockhold, F. W., Tenenbaum, J. D., Richesson, R., Marsolo, K. A., and O’Brien, E. C. (2020), “Design and Analytic Considerations for Using Patient-reported Health Data in Pragmatic Clinical Trials: Report from an NIH Collaboratory Roundtable,” Journal of the American Medical Informatics Association, 27, 634–638. DOI: 10.1093/jamia/ocz226.
   PubMed Web of Science ®Google Scholar
• Schmidli, H., Häring, D. A., Thomas, M., Cassidy, A., Weber, S., and Bretz, F. (2020), “Beyond Randomized Clinical Trials: Use of External Controls,” Clinical Pharmacology & Therapeutics, 107, 806–816.
   PubMed Web of Science ®Google Scholar
• Schulz, A., Ajayi, T., Specchio, N., de Los Reyes, E., Gissen, P., Ballon, D., Dyke, J. P., Cahan, H., Slasor, P., Jacoby, D., and Kohlschütter, A. (2018), “Study of Intraventricular cerliponase alfa for CLN2 Disease,” New England Journal of Medicine, 378, 1898–1907. DOI: 10.1056/NEJMoa1712649.
   PubMed Web of Science ®Google Scholar
• Shaw, P. A., Gustafson, P., Carroll, R. J., Deffner, V., Dodd, K. W., Keogh, R. H., Kipnis, V., Tooze, J. A., Wallace, M. P., Küchenhoff, H., and Freedman, L. S. (2020), “STRATOS Guidance Document on Measurement Error and Misclassification of Variables in Observational Epidemiology: Part 2—More Complex Methods of Adjustment and Advanced Topics,” Statistics in Medicine, 39, 2232–2263. DOI: 10.1002/sim.8531.
   PubMed Web of Science ®Google Scholar
• Suissa, S. (2008), “Immortal Time Bias in Pharmacoepidemiology,” American Journal of Epidemiology, 167, 492–499. DOI: 10.1093/aje/kwm324.
   PubMed Web of Science ®Google Scholar
• TACROLIMUS (2021), “FDA Approves New Use of Transplant Drug based on Real-World Evidence,” available at https://www.fda.gov/drugs/news-events-human-drugs/fda-approves-new-use-transplant-drug-based-real-world-evidence.
   Google Scholar
• Tripepi, G., Jager, K. J., Dekker, F. W., and Zoccali, C. (2010), “Selection Bias and Information Bias in Clinical Research,” Nephron. Clinical Practice, 115, c94–c99.
   PubMedGoogle Scholar
• Wang, S. V., Schneeweiss, S., Berger, M. L., Brown, J., de Vries, F., Douglas, I., Gagne, J. J., Gini, R., Klungel, O., Mullins, C. D., Nguyen, M. D., Rassen, J. A., Smeeth, L., and Sturkenboom, M. (2017), “Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1. 0,” Value in Health, 20, 1009–1022. DOI: 10.1016/j.jval.2017.08.3018.
   PubMed Web of Science ®Google Scholar
• Zhang, Q., Gossai, A., Monroe, S., Nussbaum, N. C., and Parrinello, C. M. (2021), “Validation Analysis of a Composite Real-world Mortality Endpoint for Patients with Cancer in the United States,” Health Services Research, 56, 1281–1287. DOI: 10.1111/1475-6773.13669.
   PubMed Web of Science ®Google Scholar
• Zhu, Y., Matsuyama, Y., Ohashi, Y., and Setoguchi, S. (2015), “When to Conduct Probabilistic Linkage vs. Deterministic Linkage? A Simulation Study,” Journal of Biomedical Informatics, 56, 80–86. DOI: 10.1016/j.jbi.2015.05.012.
   PubMed Web of Science ®Google Scholar