Targeted Learning: Toward a Future Informed by Real-World Evidence

References

• Balzer, L. B., Zheng, W., van der Laan, M. J., and Petersen, M. L. (2019), “A New Approach to Hierarchical Data Analysis: Targeted Maximum Likelihood Estimation for the Causal Effect of a Cluster-Level Exposure,” Statistical Methods in Medical Research, 28, 1761–1780. DOI: 10.1177/0962280218774936.
   PubMed Web of Science ®Google Scholar
• Bang, H., and Robins, J. M. (2005), “Doubly Robust Estimation in Missing Data and Causal Inference Models,” Biometrics, 61, 962–973. DOI: 10.1111/j.1541-0420.2005.00377.x.
   PubMed Web of Science ®Google Scholar
• Carrell, D., Gruber, S., Floyd, J. S., Bann, M. A., Cushing-Haugen, K. L., Johnson, R. L., Graham, V., Cronkite, D. J., Hazlehurst, B. L., Felcher, A. H., Bejin, C. A., Kennedy, A., Shinde, M., Karami, S., Ma, Y., Stojanovic, D., Zhao, Y., Ball, R., and Nelson, J. (in press), “Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning,” American Journal of Epidemiology.
   Google Scholar
• Chipman, H., George, E. I., and McCulloch, R. E. (2010), “BART: Bayesian Additive Regression Trees,” The Annals of Applied Statistics, 4, 266–298. DOI: 10.1214/09-AOAS285.
   Web of Science ®Google Scholar
• Cole, S. R., and Hernán, M. A. (2008), “Constructing Inverse Probability Weights for Marginal Structural Models,” American Journal of Epidemiology, 168, 656–664. DOI: 10.1093/aje/kwn164.
   PubMed Web of Science ®Google Scholar
• Concato, J., Stein, P., Dal Pan, G. J., Ball, R., and Corrigan-Curay, J. (2020), “Randomized, Observational, Interventional, and Real-World—What’s in a Name?” Pharmacoepidemiology and Drug Safety, 29, 1514–1517. DOI: 10.1002/pds.5123.
   PubMed Web of Science ®Google Scholar
• Corrigan-Curay, J., Sacks, L., and Woodcock, J. (2018), “Real-World Evidence and Real-World Data for Evaluating Drug Safety and Effectiveness,” JAMA, 320, 867–868. DOI: 10.1001/jama.2018.10136.
   PubMed Web of Science ®Google Scholar
• Díaz, I., and van der Laan, M. J. (2013), “Sensitivity Analysis for Causal Inference under Unmeasured Confounding and Measurement Error Problems,” The International Journal of Biostatistics, 9, 149–160. DOI: 10.1515/ijb-2013-0004.
   PubMedGoogle Scholar
• FDA (2018), “Framework for FDA’s Real-world Evidence Program,” Available at https://www.fda.gov/media/120060/download.
   Google Scholar
• FDA (2020), “21st Century Cures Act,” Available at https://www.fda.gov/regulatory-information/selected-amendments-fdc-act/21st-century-cures-act
   Google Scholar
• FDA (2021), “Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products,” Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory.
   Google Scholar
• Franklin, J. M., Schneeweiss, S., Polinski, J. M., and Rassen, J. A. (2014), “Plasmode Simulation for the Evaluation of Pharmacoepidemiologic Methods in Complex Healthcare Databases,” Computational Statistics & Data Analysis, 72, 219–226. DOI: 10.1016/j.csda.2013.10.018.
   PubMed Web of Science ®Google Scholar
• Friedman, J., Hastie, T., and Tibshirani, R. (2010), “Regularization Paths for Generalized Linear Models via Coordinate Descent,” Journal of Statistical Software, 33, 1–22.
   PubMed Web of Science ®Google Scholar
• Friedman, J., Hastie, T., and Tibshirani, R. (2001), The Elements of Statistical Learning, Springer Series in Statistics, New York: Springer.
   Google Scholar
• Gruber, S., Krakower, D., Menchaca, J. T., Hsu, K., Hawrusik, R., Maro, J. C., Cocoros, N. M., Kruskal, B. A., Wilson, I. B., Mayer, K. H., and Klompas, M. (2020), “Using Electronic Health Records to Identify Candidates for Human Immunodeficiency Virus Pre-exposure Prophylaxis: An Application of Super Learning to Risk Prediction When the Outcome is Rare,” Statistics in Medicine, 39, 3059–3073. DOI: 10.1002/sim.8591.
   PubMed Web of Science ®Google Scholar
• Gruber, S., Lee H., Phillips, R., Ho, M., and van der Laan, M. (2022a), “Developing a Targeted Learning-Based Statistical Analysis Plan,” Statistics in Biopharmaceutical Research. DOI: 10.1080/19466315.2022.2116104.
   Web of Science ®Google Scholar
• Gruber, S., Phillips, R., Lee, H., and van der Laan, M. (2022b), “Data-Adaptive Selection of the Propensity Score Truncation Level for Inverse Probability Weighted and Targeted Maximum Likelihood Estimators of Marginal Point Treatment Effects,” American Journal of Epidemiology, 191, 1640–1651. DOI: 10.1093/aje/kwac087.
   PubMed Web of Science ®Google Scholar
• Gruber, S., and van der Laan, M. (2012), “tmle: An R Package for Targeted Maximum Likelihood Estimation,” Journal of Statistical Software, 51, 1–35. DOI: 10.18637/jss.v051.i13.
   PubMed Web of Science ®Google Scholar
• Hansen, B. B. (2004), “Full Matching in an Observational Study of Coaching for the SAT,” Journal of the American Statistical Association, 99, 609–618. DOI: 10.1198/016214504000000647.
   Web of Science ®Google Scholar
• Hastie, T. (2019), gam: Generalized Additive Models. R package version 1.16.1. Available at https://CRAN.R-project.org/package=gam.
   Google Scholar
• Hernán, M. A., Brumback, B., and Robins, J. M. (2000), “Marginal Structural Models to Estimate the Causal Effect of Zidovudine on the Survival of HIV-Positive Men,” Epidemiology, 11, 561–570. DOI: 10.1097/00001648-200009000-00012.
   PubMed Web of Science ®Google Scholar
• Hernán, M. A., and Robins, J. M. (2020), Causal Inference: What If, Boca Raton: Chapman & Hall/CRC.
   Google Scholar
• Ho, D. E., Imai, K., King, G., and Stuart, E. A. (2011), “MatchIt: Nonparametric Preprocessing for Parametric Causal Inference,” Journal of Statistical Software, 42, 1–28. DOI: 10.18637/jss.
   Web of Science ®Google Scholar
• Ho, M., van der Laan, M., Lee, H., Chen, J., Lee, K., Fang, Y., He, W., Irony, T., Jiang, Q., Lin, X., Meng, Z., Mishra-Kalyani, P., Rockhold, F., Song, Y., Wang, H., and White, R. (2021), “The Current Landscape in Biostatistics of Real-World Data and Evidence: Causal Inference Frameworks for Study Design and Analysis,” Statistics in Biopharmaceutical Research, 15, 43–56. DOI: 10.1080/19466315.2021.1883475.
   Web of Science ®Google Scholar
• ICH (2020), “ICH E9(R1) Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials,” Available at https://www.ich.org/page/efficacy-guidelines.
   Google Scholar
• Imai, K., and Ratkovic, M. (2013), “Covariate Balancing Propensity Score. Covariate Balancing Propensity Score,” Journal of the Royal Statistical Society, Series B, 76, 243–263. DOI: 10.1111/rssb.12027.
   Google Scholar
• International Stroke Trial Collaborative Group. (1997), “The International Stroke Trial (IST): A Randomised Trial of Aspirin, Subcutaneous Heparin, both, or Neither among 19435 Patients with Acute Ischaemic Stroke,” The Lancet, 349, 1569–1581.
   PubMed Web of Science ®Google Scholar
• Ju, C., Wyss, R., Franklin, J. M., Schneeweiss, S., Häggström, J., and van der Laan, M. J. (2017), “Collaborative-Controlled LASSO for Constructing Propensity Score-based Estimators in High-Dimensional Data,” Statistical Methods in Medical Research, 28, 1044–1063. DOI: 10.1177/0962280217744588.
   PubMed Web of Science ®Google Scholar
• Kahale, L. A., Diab, B., Khamis, A. M., Chang, Y., Cruz Lopes, L., Agarwal, A., Li, L., Mustafa, R. A., Koujanian, S., Waziry, R., Busse, J. W., Dakik, A., Guyatt, G., and Aki, E. A. (2019), “Potentially Missing Data are Considerably More Frequent than Definitely Missing Data: A Methodological Survey of 638 Randomized Controlled Trials,” Journal of Clinical Epidemiology, 106, 18–31. DOI: 10.1016/j.jclinepi.2018.10.001.
   PubMed Web of Science ®Google Scholar
• Kempker, R. R., Mikiashvili, L., Zhao, Y., Benkeser, D., Barbakadze, K., Bablishvili, N., Avaliani, Z., Peloquin, C. A., Blumberg, H. M., and Kipiani, M. (2020), “Clinical Outcomes Among Patients With Drug-resistant Tuberculosis Receiving Bedaquiline- or Delamanid-Containing Regimens,” Clinical Infectious Diseases: An Official Publication of the Infectious Diseases Society of America, 71, 2336–2344. DOI: 10.1093/cid/ciz1107.
   PubMed Web of Science ®Google Scholar
• Kreif, N., Tran, L., Grieve, R., De Stavola, B., Tasker, R. C., and Petersen, M. (2017), “Estimating the Comparative Effectiveness of Feeding Interventions in the Pediatric Intensive Care Unit: A Demonstration of Longitudinal Targeted Maximum Likelihood Estimation,” American Journal of Epidemiology, 186, 1370–1379. DOI: 10.1093/aje/kwx213.
   PubMed Web of Science ®Google Scholar
• Lendle, S. D., Schwab, J., Petersen, M. L., and van der Laan, M. J. (2017), “ltmle: An R Package Implementing Targeted Minimum Loss-Based Estimation for Longitudinal Data,” Journal of Statistical Software, 81, 1–21. DOI: 10.18637/jss.v081.i01.
   Web of Science ®Google Scholar
• Lendle, S. D., Fireman, B., and van der Laan, M. J. (2013), “Targeted Maximum Likelihood Estimation in Safety Analysis,” Journal of Clinical Epidemiology, 66, S91–S98. DOI: 10.1016/j.jclinepi.2013.02.017.
   PubMed Web of Science ®Google Scholar
• Ling, A. Y., Montez-Rath, M. E., Mathur, M. B., Kapphahn, K., and Desai, M. (2019), “How to Apply Multiple Imputation in Propensity Score Matching with Partially Observed Confounders: A Simulation Study and Practical Recommendations,” arXiv Preprint arXiv:1904.07408.
   Google Scholar
• Pearl, J. (2009), Causality, Cambridge: Cambridge University Press.
   Google Scholar
• Pearl, J. (2010), “An Introduction to Causal Inference,” The International Journal of Biostatistics, 6, Article 7. DOI: 10.2202/1557-4679.1203.
   PubMedGoogle Scholar
• Petersen, M., Schwab, J., Gruber, S., Blaser, N., Schomaker, M., and van der Laan, M. (2014), “Targeted Maximum Likelihood Estimation for Dynamic and Static Longitudinal Marginal Structural Working Models,” Journal of Causal Inference, 2, 147–185. DOI: 10.1515/jci-2013-0007.
   PubMedGoogle Scholar
• Petersen, M. L., and van der Laan, M. J. (2014), “Causal Models and Learning from Data: Integrating Causal Modeling and Statistical Estimation,” Epidemiology (Cambridge, Mass.), 25, 418–426. DOI: 10.1097/EDE.0000000000000078.
   PubMed Web of Science ®Google Scholar
• Pirracchio, R., Petersen, M. L., Carone, M., Rigon, M. R., Chevret, S., and van der Laan, M. J. (2015), “Mortality Prediction in Intensive Care Units with the Super ICU Learner Algorithm (SICULA): A Population-based Study,” The Lancet Respiratory Medicine, 3, 42–52. DOI: 10.1016/S2213-2600(14)70239-5.
   PubMed Web of Science ®Google Scholar
• Polley, E., LeDell, E., Kennedy, C., and van der Laan, M. (2019), SuperLearner: Super Learner Prediction. R package version 2.0-26. Available at https://CRAN.R-project.org/package=SuperLearner.
   Google Scholar
• Porter, K. E., Gruber, S., van der Laan, M. J., and Sekhon, J. S. (2011), “The Relative Performance of Targeted Maximum Likelihood Estimators,” The International Journal of Biostatistics, 7, 1–34. DOI: 10.2202/1557-4679.1308.
   Google Scholar
• R Core Team (2020), R: A Language and Environment for Statistical Computing, Vienna, Austria: R Foundation for Statistical Computing. Available at http://www.R-project.org/.
   Google Scholar
• Ramsahai, R. R., Grieve, R., and Sekhon, J. S. (2011), “Extending Iterative Matching Methods: An Approach to Improving Covariate Balance That Allows Prioritisation,” Health Services and Outcomes Research Methodology, 11, 95–114. DOI: 10.1007/s10742-011-0075-5.
   Google Scholar
• Rosenbaum, P. R., and Rubin, D. B. (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects,” Biometrika, 70, 41–55. DOI: 10.1093/biomet/70.1.41.
   Web of Science ®Google Scholar
• Rosenblum, M., and van der Laan, M. J. (2010), “Simple, Efficient Estimators of Treatment Effects in Randomized Trials using Generalized Linear Models to Leverage Baseline Variables,” The International Journal of Biostatistics, 6, 1–41. DOI: 10.2202/1557-4679.1138.
   Google Scholar
• Sandercock, P., Niewada, M., and Czlonkowska, A. (2011), International Stroke Trial database (version 2), University of Edinburgh. Department of Clinical Neurosciences. DOI: 10.7488/ds/104.
   Google Scholar
• Simon, G. E., Bindman, A. B., Dreyer, N. A., Platt, R., Watanabe, J. H., Horberg, M., Hernandez, A., and Califf, R. M. (2022), “When Can We Trust Real-World Data To Evaluate New Medical Treatments?,” Clinical Pharmacology and Therapeutics, 111, 24–29. DOI: 10.1002/cpt.2252.
   PubMed Web of Science ®Google Scholar
• Stuart, E. A. (2010), “Matching Methods for Causal Inference: A Review and a Look Forward,” Statistical Science, 25, 1–21. DOI: 10.1214/09-STS313.
   PubMed Web of Science ®Google Scholar
• Tackney, M. S., Morris, T., White, I., Leyrat, C., Diaz-Ordaz, K., and Williamson, E. (2023), “A Comparison of Covariate Adjustment Approaches Under Model Misspecification In Individually Randomized Trials,” Trials, 24, 1–8. DOI: 10.21203/rs.3.rs-1053600/v1.
   PubMed Web of Science ®Google Scholar
• van der Laan, M. J., and Robins, J. (2003), Unified Methods for Censored Longitudinal Data and Causality, New York: Springer.
   Google Scholar
• van der Laan, M. J., and Rose, S. (2011), Targeted Learning: Causal Inference for Observational and Experimental Data, New York: Springer.
   Google Scholar
• VanderWeele, T., and Ding, P. (2017), “Sensitivity Analysis in Observational Research: Introducing the E-value,” Annals of Internal Medicine, 167, 268–274. DOI: 10.7326/M16-2607.
   PubMed Web of Science ®Google Scholar
• Vickers, A. J., and McCarney, R. (2003), “Use of a Single Global Assessment to Reduce Missing Data in a Clinical Trial with Follow-Up at One Year,” Controlled Clinical Trials, 24, 731–735. DOI: 10.10.16/j.cct.2003.10.001.
   PubMedGoogle Scholar
• Vickers, A. J., Rees, R. W., Zollman, C. E., McCarney, R., Smith, C., Ellis, N., Fisher, P., and Van Haselen, R. (2004), “Acupuncture for Chronic Headache in Primary Care: Large, Pragmatic, Randomised Trial,” BMJ, 328, 744. DOI: 10.1136/bmj.38029.421863.EB.
   PubMed Web of Science ®Google Scholar
• Vickers, A. J. (2006), “Whose Data Set Is It Anyway? Sharing Raw Data from Randomized Trials,” Trials, 7, 1–6. DOI: 10.1186/1745-6215-7-15.
   PubMed Web of Science ®Google Scholar
• Zeileis, A. (2004), “Econometric Computing with HC and HAC Covariance Matrix Estimators,” Journal of Statistical Software, 11, 1–17. DOI: 10.18637/jss.v011.i10.
   Google Scholar
• Zeileis, A., Köll, S., and Graham, N. (2020), “Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R,” Journal of Statistical Software, 95, 1–36. DOI: 10.18637/jss.v095.i01.
   Web of Science ®Google Scholar