An overview of methodological flaws of real-world studies investigating drug safety in the post-marketing setting

References

• Monti S, Grosso V, Todoerti M, et al. Randomized controlled trials and real-world data: differences and similarities to untangle literature data. Rheumatology (Oxford). 2018;57:vii54–8.
   PubMed Web of Science ®Google Scholar
• Bérard A. Pharmacoepidemiology research-real-world evidence for decision making. Front Pharmacol. 2021;12:723427.
   PubMed Web of Science ®Google Scholar
• Spelsberg A, Prugger C, Doshi P, et al. Contribution of industry funded post-marketing studies to drug safety: survey of notifications submitted to regulatory agencies. BMJ. 2017;356:j337.
   PubMedGoogle Scholar
• Crisafulli S, Sultana J, Ingrasciotta Y, et al. Role of healthcare databases and registries for surveillance of orphan drugs in the real-world setting: the Italian case study. Expert Opin Drug Saf. 2019;18(6):497–509. DOI:10.1080/14740338.2019.1614165
   PubMed Web of Science ®Google Scholar
• Hemkens LG. How routinely collected data for randomized trials provide long-term randomized real-world evidence. JAMA Netw Open. 2018;1(8):e186014.
   PubMed Web of Science ®Google Scholar
• Crisafulli S, Trifirò G. [The role of real-world evidence for the study of rare diseases epidemiology and the post-marketing evaluation of orphan drugs.]. Recenti Prog Med. 2022;113(7):425–429.
   PubMedGoogle Scholar
• Gaasterland CMW, van der Weide MCJ-, du Prie-Olthof MJ, et al. The patient’s view on rare disease trial design – a qualitative study. Orphanet J Rare Dis. 2019;14(1):31. DOI:10.1186/s13023-019-1002-z
   PubMedGoogle Scholar
• Mofid S, Bolislis WR, Kühler TC. Real-world data in the postapproval setting as applied by the EMA and the US FDA. Clin Ther. 2022;44(2):306–322.
   PubMed Web of Science ®Google Scholar
• Trifirò G, Crisafulli S. A new era of pharmacovigilance: future challenges and opportunities. Frontiers In Drug Safety And Regulation. 2022;2. DOI:10.3389/fdsfr.2022.866898
   Google Scholar
• Dhanda S, Osborne V, Lynn E, et al. Postmarketing studies: can they provide a safety net for COVID-19 vaccines in the UK? BMJ EBM. 2022;27(1):1–6.
   Google Scholar
• Food and Drug Administration. Real world evidence; Accessed 2023 Mar 3. Available form https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence.
   Google Scholar
• Liu F, Panagiotakos D. Real-world data: a brief review of the methods, applications, challenges and opportunities. BMC Med Res Methodol. 2022;22(1):287.
   PubMed Web of Science ®Google Scholar
• Raj N, Fernandes S, Charyulu NR, et al. Postmarket surveillance: a review on key aspects and measures on the effective functioning in the context of the United Kingdom and Canada. Ther Adv Drug Saf. 2019;10:2042098619865413.
   Web of Science ®Google Scholar
• Lavertu A, Vora B, Giacomini KM, et al. A new era in pharmacovigilance: toward real-world data and digital monitoring. Clin Pharmacol Ther. 2021;109(5):1197–1202.
   PubMed Web of Science ®Google Scholar
• Tan YY, Papez V, Chang WH, et al. Comparing clinical trial population representativeness to real-world populations: an external validity analysis encompassing 43 895 trials and 5 685 738 individuals across 989 unique drugs and 286 conditions in England. Lancet Healthy Longev. 2022;3(10):e674–89.
   PubMedGoogle Scholar
• Pinnow E, Amr S, Bentzen SM, et al. Postmarket safety outcomes for new molecular entity (NME) drugs approved by the food and drug administration between 2002 and 2014. Clin Pharmacol Ther. 2018;104(2):390–400. DOI:10.1002/cpt.944
   PubMed Web of Science ®Google Scholar
• Zhang X, Zhang Y, Ye X, et al. Overview of phase IV clinical trials for postmarket drug safety surveillance: a status report from the ClinicalTrials.Gov registry. BMJ Open. 2016;6(11):e010643.
   PubMed Web of Science ®Google Scholar
• V WS, Sreedhara SK, Schneeweiss S. Repeat initiative. reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions. Nat Commun. 2022;13(1):5126.
   PubMed Web of Science ®Google Scholar
• Hiramatsu K, Barrett A, Miyata Y, et al. Current status, challenges, and future perspectives of real-world data and real-world evidence in Japan. Drugs Real World Outcomes. 2021;8(4):459–480. DOI:10.1007/s40801-021-00266-3
   PubMedGoogle Scholar
• Burns L, Le RN, Kalesnik-Orszulak R, et al. Real-world evidence for regulatory decision-making: guidance from around the world. Clin Ther. 2022;44(3):420–437. DOI:10.1016/j.clinthera.2022.01.012
   PubMed Web of Science ®Google Scholar
• Flynn R, Plueschke K, Quinten C, et al. Marketing authorization applications made to the European medicines agency in 2018–2019: what was the contribution of real-world evidence? Clin Pharmacol Ther. 2022;111(1):90–97. DOI:10.1002/cpt.2461
   PubMed Web of Science ®Google Scholar
• Purpura CA, Garry EM, Honig N, et al. The role of real‐world evidence in FDA‐approved new drug and biologics license applications. Clin Pharmacol Ther. 2022;111(1):135–144.
   PubMed Web of Science ®Google Scholar
• Concato J, Corrigan-Curay J. Real-world evidence — where are we now? N Engl J Med. 2022;386(18):1680–1682.
   PubMed Web of Science ®Google Scholar
• Hall GC, Sauer B, Bourke A, et al. Guidelines for good database selection and use in pharmacoepidemiology research. Pharmacoepidemiol Drug Saf. 2012;21(1):1–10.
   PubMed Web of Science ®Google Scholar
• Food and Drug Administration, Sentinel Operations Center. Standardization and querying of data quality metrics and characteristics for electronic health data: data quality metrics system final report; 2023. https://www.sentinelinitiative.org/sites/default/files/Methods/Standardization_and_Querying_of_Data_Quality_Metrics.pdf
   Google Scholar
• European Network of Centres for Pharmacoepidemiology and Pharmacovigilance(ENCePP). Guide on methodological standards in pharmacoepidemiology; 2022 [Accessed 2023 Mar 3], ( Revision 10). t: http://www.encepp.eu/standards_and_guidance
   Google Scholar
• Ingrasciotta Y, Sultana J, Giorgianni F, et al. Association of individual non-steroidal anti-inflammatory drugs and chronic kidney disease: a population-based case control study. PLoS ONE. 2015;10(4):e0122899. DOI:10.1371/journal.pone.0122899
   PubMed Web of Science ®Google Scholar
• Crisafulli S, Ientile V, L’Abbate L, et al. COVID-19 patient management in outpatient setting: a population-based study from southern Italy. JCM. 2021;11(1):11. DOI:10.3390/jcm11010051
   Google Scholar
• Belleudi V, Trotta F, Pinnarelli L, et al. Neonatal outcomes following new reimbursement limitations on palivizumab in Italy. Arch Dis Child. 2018;103(12):1163–1167.
   PubMed Web of Science ®Google Scholar
• Sultana J, Giorgianni F, Rea F, et al. All-cause mortality and antipsychotic use among elderly persons with high baseline cardiovascular and cerebrovascular risk: a multi-center retrospective cohort study in Italy. Expert Opin Drug Metab Toxicol. 2019;15(2):179–188. DOI:10.1080/17425255.2019.1561860
   PubMed Web of Science ®Google Scholar
• Ingrasciotta Y, Bertuccio MP, Crisafulli S, et al. Real world use of antidiabetic drugs in the years 2011–2017: a population-based study from Southern Italy. IJERPH. 2020;17(24):17. DOI:10.3390/ijerph17249514
   Web of Science ®Google Scholar
• Rea F, Calusi G, Franchi M, et al. Adherence of elderly patients with cardiovascular disease to statins and the risk of exacerbation of chronic obstructive pulmonary disease: evidence from an Italian real-world investigation. Drugs Aging. 2018;35(12):1099–1108. DOI:10.1007/s40266-018-0600-0
   PubMed Web of Science ®Google Scholar
• Trifirò G, Gini R, Barone-Adesi F, et al. The role of European healthcare databases for post-marketing drug effectiveness, safety and value evaluation: where does Italy stand? Drug Saf. 2019;42(3):347–363. DOI:10.1007/s40264-018-0732-5
   PubMed Web of Science ®Google Scholar
• Sultana J, Fontana A, Giorgianni F, et al. Measuring the effectiveness of safety warnings on the risk of stroke in older antipsychotic users: a nationwide cohort study in two large electronic medical records databases in the United Kingdom and Italy. Drug Saf. 2019;42(12):1471–1485. DOI:10.1007/s40264-019-00860-z
   PubMed Web of Science ®Google Scholar
• Lapi F, Capogrosso Sansone A, Mantarro S, et al. Hepatitis C virus infection: opportunities for an earlier detection in primary care. Eur J Gastroenterol Hepatol. 2017;29(3):271–276. DOI:10.1097/MEG.0000000000000785
   PubMed Web of Science ®Google Scholar
• Ferrajolo C, Verhamme KMC, Trifirò G, ’T Jong GW, Picelli G, Giaquinto C, et al. Antibiotic-induced liver injury in paediatric outpatients: a case-control study in primary care databases. Drug Saf. 2017;40(4):305–315. DOI:10.1007/s40264-016-0493-y
   PubMed Web of Science ®Google Scholar
• Traversa G, Sagliocca L, Magrini N, et al. A perspective for the role of drug registries in the post-marketing surveillance. Recenti Prog Med. 2013;104(6):280–287.
   PubMedGoogle Scholar
• Khan MAA, Hamid S, Z-U-D B. Pharmacovigilance in high-income countries: current developments and a review of literature. Pharmacy (Basel). 2023;11(1):11.
   PubMedGoogle Scholar
• Coloma PM, Trifirò G, Patadia V, et al. Postmarketing safety surveillance: where does signal detection using electronic healthcare records fit into the big picture? Drug Saf. 2013;36(3):183–197.
   PubMed Web of Science ®Google Scholar
• Pal SN, Duncombe C, Falzon D, et al. WHO strategy for collecting safety data in public health programmes: complementing spontaneous reporting systems. Drug Saf. 2013;36(2):75–81.
   PubMed Web of Science ®Google Scholar
• World Health Organization. Protocol template to be used as template for observational study protocols: cohort event monitoring (‎CEM)‎ for safety signal detection after vaccination with COVID-19 vaccines; 2021 [Accessed 2023 Mar 3]. Available from: https://apps.who.int/iris/handle/10665/342193
   Google Scholar
• Barbieri MA, Cicala G, Cutroneo PM, et al. Safety profile of biologics used in rheumatology: an Italian prospective pharmacovigilance study. JCM. 2020;9(4):1227. DOI:10.3390/jcm9041227
   Google Scholar
• Rachlis B, Karwa R, Chema C, et al. Targeted spontaneous reporting: assessing opportunities to conduct routine pharmacovigilance for antiretroviral treatment on an international scale. Drug Saf. 2016;39(10):959–976. DOI:10.1007/s40264-016-0434-9
   PubMed Web of Science ®Google Scholar
• Crisafulli S, Cutroneo PM, Verhamme K, et al. Drug-induced urinary retention: an analysis of a national spontaneous adverse drug reaction reporting database. Eur Urol Focus. 2022;8(5):1424–1432. DOI:10.1016/j.euf.2021.07.001
   PubMedGoogle Scholar
• Crisafulli S, Cutroneo PM, Luxi N, et al. Is pegylation of drugs associated with hypersensitivity reactions? an analysis of the Italian national spontaneous adverse drug reaction reporting system. Drug Saf. 2023;46(4):343–355. DOI:10.1007/s40264-023-01277-5
   PubMed Web of Science ®Google Scholar
• Faillie J-L. Case–non-case studies: principle, methods, bias and interpretation. Therapie. 2019;74(2):225–232.
   PubMed Web of Science ®Google Scholar
• Franco V, Barbieri MA, Cutroneo PM, et al. Pediatric adverse reactions to antiseizure medications – an analysis of data from the Italian spontaneous reporting system (2001–2019). Epilepsy Behav. 2021;119:107989.
   PubMed Web of Science ®Google Scholar
• Cicala G, Barbieri MA, Santoro V, et al. Safety and tolerability of antipsychotic drugs in pediatric patients: data from a 1-year naturalistic study. Front Psychiatry. 2020;11:11.
   PubMed Web of Science ®Google Scholar
• Ventola CL. Big data and pharmacovigilance: data mining for adverse drug events and interactions. P T. 2018;43(6):340–351.
   PubMedGoogle Scholar
• Penberthy LT, Rivera DR, Lund JL, et al. An overview of real-world data sources for oncology and considerations for research. CA Cancer J Clin. 2022;72(3):287–300.
   PubMed Web of Science ®Google Scholar
• Ho Y-F, Hu F-C, Lee P-I. The advantages and challenges of using real-world data for patient care. Clin Transl Sci. 2020;13(1):4–7.
   PubMed Web of Science ®Google Scholar
• Cea Soriano L, Soriano-Gabarró M, García Rodríguez LA. Validation of low-dose aspirin prescription data in the health improvement network: how much misclassification due to over-the-counter use? Pharmacoepidemiol Drug Saf. 2016;25(4):392–398.
   PubMed Web of Science ®Google Scholar
• Bower JK, Patel S, Rudy JE, et al. Addressing bias in electronic health record-based surveillance of cardiovascular disease risk: finding the signal through the noise. Curr Epidemiol Rep. 2017;4(4):346–352.
   PubMedGoogle Scholar
• Hazell L, Shakir SAW. Under-reporting of adverse drug reactions: a systematic review. Drug Saf. 2006;29(5):385–396.
   PubMed Web of Science ®Google Scholar
• Palleria C, Leporini C, Chimirri S, et al. Limitations and obstacles of the spontaneous adverse drugs reactions reporting: two “challenging” case reports. J Pharmacol Pharmacother. 2013;4(1_suppl):S66–72. DOI:10.4103/0976-500X.120955
   PubMedGoogle Scholar
• Cutroneo PM, Giardina C, Ientile V, et al. Overview of the safety of anti-VEGF drugs: analysis of the Italian spontaneous reporting system. Drug Saf. 2017;40(11):1131–1140. DOI:10.1007/s40264-017-0553-y
   PubMed Web of Science ®Google Scholar
• Arora A, Jalali RK, Vohora D. Relevance of the weber effect in contemporary pharmacovigilance of oncology drugs. TCRM. 2017;13:1195–1203.
   Google Scholar
• Hung E, Hauben M, Essex H, et al. More extreme duplication in FDA adverse event reporting system detected by literature reference normalization and fuzzy string matching. Pharmacoepidemiol Drug Saf. 2023;32(3):387–391.
   PubMed Web of Science ®Google Scholar
• Stürmer T, Wang T, Golightly YM, et al. Methodological considerations when analysing and interpreting real-world data. Rheumatology (Oxford). 2020;59(1):14–25.
   PubMed Web of Science ®Google Scholar
• Danaei G, Tavakkoli M, Hernán MA. Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins. Am J Epidemiol. 2012;175(4):250–262.
   PubMed Web of Science ®Google Scholar
• Moride Y, Abenhaim L. Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research. J Clin Epidemiol. 1994;47(7):731–737.
   PubMed Web of Science ®Google Scholar
• Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. JMDH. 2016;9:211–217.
   Google Scholar
• Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol. 1990;43(1):87–91.
   PubMed Web of Science ®Google Scholar
• Horwitz RI, Feinstein AR. The problem of “protopathic bias” in case-control studies. Am j med. 1980;68(2):255–258.
   PubMed Web of Science ®Google Scholar
• Abrahami D, Pradhan R, Yin H, et al. Proton pump inhibitors and the risk of inflammatory bowel disease: population-based cohort study. Gut. 2023. DOI:10.1136/gutjnl-2022-328866
   PubMed Web of Science ®Google Scholar
• Suissa S. Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. 2008;167(4):492–499.
   PubMed Web of Science ®Google Scholar
• Suissa S. Immortal time bias in observational studies of drug effects. Pharmacoepidem Drug Safe. 2007;16(3):241–249.
   PubMed Web of Science ®Google Scholar
• Suissa S, Azoulay L. Metformin and the risk of cancer: time-related biases in observational studies. Diabetes Care. 2012;35(12):2665–2673.
   PubMed Web of Science ®Google Scholar
• Gini R, Sturkenboom MCJ, Sultana J, et al. Different strategies to execute multi-database studies for medicines surveillance in real-world setting: a reflection on the European model. Clin Pharmacol Ther. 2020;108(2):228–235. DOI:10.1002/cpt.1833
   PubMed Web of Science ®Google Scholar
• Kent S, Burn E, Dawoud D, et al. Common problems, common data model solutions. Evidence Generation For Health Technology Assessment Pharmacoeconomics. 2021;39(3):275–285. DOI:10.1007/s40273-020-00981-9
   Google Scholar
• Madigan D, Ryan PB, Schuemie M. Does design matter? Systematic evaluation of the impact of analytical choices on effect estimates in observational studies. Ther Adv Drug Saf. 2013;4(2):53–62.
   PubMedGoogle Scholar
• Madigan D, Ryan PB, Schuemie M, et al. Evaluating the impact of database heterogeneity on observational study results. Am J Epidemiol. 2013;178(4):645–651. DOI:10.1093/aje/kwt010
   PubMed Web of Science ®Google Scholar
• Bykov K, Patorno E, D’Andrea E, et al. Prevalence of avoidable and bias-inflicting methodological pitfalls in real-world studies of medication safety and effectiveness. Clin Pharmacol Ther. 2022;111(1):209–217. DOI:10.1002/cpt.2364
   PubMed Web of Science ®Google Scholar
• Heads of Medicines Agencies, European Medicines Agency. HMA-EMA joint big data taskforce phase ii report: “evolving data-driven regulation.“ 2018 [Accessed 2023 Mar 3]. https://www.ema.europa.eu/en/documents/other/hma-ema-joint-big-datataskforce-phase-ii-report-evolving-data-drivenregulation_en.pdf
   Google Scholar
• Food and Drug Administration. Framework for FDA’s real-world evidence program; 2018 [Accessed 2023 March 3]. https://www.fda.gov/media/120060/download
   Google Scholar
• Desai RJ, Franklin JM. Alternative approaches for confounding adjustment in observational studies using weighting based on the propensity score: a primer for practitioners. BMJ. 2019;367:l5657.
   PubMedGoogle Scholar
• Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.
   Web of Science ®Google Scholar