https://orcid.org/0000-0003-2598-1899
Abstract
Courtesy of the development of the Internet, bursts of information technology, and globalization, huge multicenter studies along with meta-analyses have been introduced to the medical sciences society. Meta-analyses and multicenter studies revolutionized modern medicine and drug development, and empowered evidence based medicine by providing extremely high levels of evidence. Nevertheless, there are occasions that while results of local multi/single center studies showed efficacy of a new treatment, larger multicenter studies or meta-analyses failed to show efficacy, and vice versa. Generally, bigger studies are more powerful and we rely on their results in clinical decision making. Nevertheless, we should keep in mind that in certain circumstances, single center studies are of great importance, and are preferred to multicenter studies and meta-analyses. In order to have a better understanding of why and when multicenter studies along with meta-analyses might not be the best options, we have discussed three different scenarios.
Introduction
Courtesy of the development of the Internet, bursts of information technology, and globalization, huge multicenter studies, along with meta-analyses have been introduced to the medical sciences society. By minimizing confounding factors, multicenter studies revolutionized our understanding of many diseases, and helped us to develop and study many new treatment regimensCitation1,Citation2. Meta-analysis studies statistically mimic huge multicenter studies by adding single/multi center studies to each otherCitation3. Nevertheless, there are occasions that while results of local multi/single center studies showed efficacy of a new treatment, larger multicenter studies or meta-analyses failed to show efficacy, and vice versaCitation2.
Multicenter clinical trials with their interesting names and meta-analyses are quit fashionable in the current medical science society; now it is time to ask are they always practical? While we emphasize on the pros (advantages) of these studies, have we ever asked about the cons (disadvantages)?Citation2
Generally, bigger studies are more powerful and we rely on their results in clinical decision making, nevertheless, we should keep in mind that in certain circumstances, single center studies are also of great importance, and are preferred to multicenter studies or meta-analyses. In order to have a better understanding of why and when multicenter studies and meta-analyses might not be the best options, we are going to depict three scenarios.
Drug A scenario
A new treatment (drug A) is under investigation for the treatment of idiopathic pulmonary fibrosis. The results of phase 1, phase 2 and single center clinical trials were promising. A multicenter study in England, as well as a multicenter study in Germany were both in favor of the efficacy and safety of drug A. On the other hand, a multicenter study in Japan failed to show efficacy for drug A, and the results of another multicenter study in China raised major concerns regarding the drug safety.
An American drug company runs a huge multicenter study in 20 countries called DAIPF, and concludes that drug A, is safe and efficient in the treatment of IPF. Everyone is happy, the study is published in the Lancet, and the FDA approved it.
Imagine that you are working in a hospital in Beijing, should you prescribe drug A?
According to the above scenario, studies in China raised some major concerns regarding its safety; it might not be logical to prescribe drug A for your patients in Beijing. Although drug A was well tolerated in most of the other countries, and the results of DAIPF were in favor of the safety of drug A, in the setting of your patients, there are confounding factors which were not presented in the other countries, hence eliminated in the process of statistical analyses of the big data from the different centers. If you and your colleagues decide to order drug A for your patients, years later a phase 4 trial subgroup analysis can find that drug A is not safe in China, because of a less active variation of enzyme X (which is responsible for inactivating a toxic metabolite of drug A in the liver) in 90 percent of people in China.
What if you are working in Kyoto?
It might not be logical to order drug A for your patients. Although the results of DAIPF showed efficacy and safety of drug A, Japanese powerful multicenter studies failed to show its efficacy. This means that there was a confounding factor that is still present, and will affect the results of treatment with drug A. From a practical, pragmatic point of view, no matter what the confounding factors are, they are still there, and will affect the results of treatment with drug A. Ten years later scientists can find that IPF pathophysiology in Japan differs from IPF in European countries.
When there is an observed difference, which means that there is a confounding factor, you should take that difference into account. Which means you are taking that confounder into account, although you may not know what exactly it is.
Clinical trials are designed to eliminate confounding factors; but in multicenter studies and metanalyses, weight of small populations will be low and larger populations, centers, studies can wipe out the effects of smaller counterparts on the final results. Hence, major clinical trials, and bigger centers will dictate the final conclusion of metanalyses and multicenter studies.
Drug B scenario
It has been shown that treatment with drug B (Already FDA approved) significantly decreases post PCI stent restenosis in patients. Single center and multicenter studies, as well meta-analyses were all in favor of treatment with drug B. You conduct a powerful well-designed and well-conducted multicenter study in your country on drug B, and the results of your study are not in accordance with the other published studies.
Do you order drug B for your patients?
Drug B is not working on your patients, and it might not be logical to prescribe it. No matter the results of the other bigger studies that were in favor of drug B, when there is a persistent confounding factor (even if you do not know it), you may not eliminate it from your practice, and you should take it into account. As mentioned above, even if your country was one of the centers of previous multicenter study(s) that tend to FDA approval, probably low weight of your country has been relatively eliminated in the final results. A subgroup analysis of those studies will show that indeed it was not effective in prevention of stent restenosis in your population.
Drug C scenario
As a recent graduate medical practitioner from Harvard Medical School with a profound knowledge in evidence-based medicine, you are working as a clinician in Mumbai. According to the guidelines for the treatment of bacterial sinusitis, amoxicillin is the first line treatment. But after a couple of months you notice that almost none of your patients are getting better with amoxicillin. You conduct a pilot study on bacterial species of your patients with sinusitis and find that almost all of studied microbial species are resistant to amoxicillin, because of secreting β-lactamase, an enzyme that deactivates amoxicillin. You know Clavulanic acid, a β-lactamase inhibitor, hence decide to switch to Augmentin (amoxicillin plus clavulanic acid), and your patients’ response rate will dramatically increase.
We used this very well-known fact as a simple and easy to understand example of variation in populations, and an example of when guidelines are not working. To emphasize that we cannot blindly generalize studies. Fortunately, in this case we are doing great (at least in USA) and almost every state, county, area, or hospital has its own antibiogram. This approach is exactly what we want to encourage in different other areas of Medicine.
Discussion
We would like to clarify that we are not impugning the applicability of multi center studies or metanalyses. We are simply questioning applicability of these studies in certain conditions, and encouraging taking into account personal and population characteristics while interpreting their results and implying in clinical decision making. Nevertheless, while mentioning smaller studies we are talking about well designed, well powered local studies, and not small, and statistically weak studies with low power and compromised design.
We are not writing against evidence base medicine or clinical trials. We want to raise some concerns regarding the sweeping statements of some physicians when they do not interpret results of multicenter studies with respect to the characteristics of their patients, and population.
Pragmatism
A pragmatic approach to the patient care is crucial. Sometimes our understanding of a phenomenon is not complete, but according to repeated experimentation we may predict outcomes. Even though in our setting the results of our experimentation is not in accordance with others, if our methodology is correct, in certain occasions, we should rely on our findings rather than the other studies such as Drug A and Drug B scenario. It would be great if we try to find out the reason for the observed difference, and approach it through our findings (Drug C scenario); although sometimes it is not practical. Let us move back to the Drug C scenario, when evidence based medicine and the first line treatment according to current guidelines are not working. Imagine that you have limited resources, you may not be able to conduct a study, and you may decide to change your antibiotic to Antibiotic-X as one of the other available choices in your area. This approach is not the best approach, and it is not based on your knowledge, but it is practical.
As another example, we would like to refer to the anticonvulsant treatment of resistant seizure cases. Although our understanding of seizures has increased during the past decades, regardless of their mechanism of action and current guidelines, it is quite common in neurology clinics to switch between anticonvulsants when one of them is not working; and yes, it is in some instants in random fashionCitation4. Because our knowledge regarding seizure pathophysiology is extremely limited, we can be forced to rely on experimental personalized approaches in first/second line treatment resistant patients.
Environment polymorphism
As discussed in the third scenario, environmental polymorphism might affect characteristics of our patients, their diseases, and treatment responses. A personalized, localized approach to the patients is warranted. It is especially warranted when there are evidences that showed current algorithms and guidelines might not work in our society (again no matter whether we know the confounding factors or not)Citation5.
Gene polymorphism
Environmental differences, genetics, as well as epigenetic differences play a crucial role in patient presentations, and treatment responses. We should consider that in a larger scale, while each person is different, each society is different as well, and these differences might affect predictability of our findings on multicenter studies, and the meta-analyses that were conducted in the other countries, or the main bulk of their clinical trial population or metanalyses was consisted of a different populationCitation5–8.
Here is a real life example
The sensitivity of commercial BRCA mutation tests like 23andMe are debated. For example, 23andMe’s testing formula is based solely on three genetic variants, which is most prevalent among Ashkenazi Jews, while most people carry other mutations of the gene. This will result in false negative results. 23andMe is a commercial BRCA1 screening test, with more than 10 million customers. If only a small percentage take the test, there will be thousands who could be misled with inaccurate results. As accurately stated by Prof. Mary-Claire King, who discovered the BRCA1 gene, “The F.D.A. should not have permitted this out-of-date approach to be used for medical purposes. Misleading women and falsely reassuring results from their incomplete testing can cost them their lives.” We should keep in mind that although 23andMe is not appropriate for general population screening, it would still be a semi perfect test for the Ashkenazi Jewish population. If you did a study that developed a test or treatment that works for you, although contradicts larger studies, interpret it wisely, do not throw it away, and use it appropriatelyCitation7,Citation9–14.
Idiopathic diseases
Even in the presence of diseases with similar phenotypes, different genotypes and different mechanisms are possible in patients all around the worldCitation10,Citation12. This issue is more prevalent when we are studying diseases with vague/wide definitions; idiopathic diseases are good examples. Actually, the phrase “Idiopathic” refers to our loss of knowledge about the exact mechanism of a particular disease. An example is Idiopathic pulmonary fibrosis (IPF); in fact, IPF might encompass several different unknown diseases with different mechanisms that are clustered under a sole name. Fibrosis could be the end product of several unknown genetic, microbial, viral, and rheumatologic factors that have yet to be discovered. Hence, we should keep in mind that the pathophysiology of a disease like IPF might be diverse in different populations. This fact might critically affect the results of clinical trialsCitation15–18. In the first scenario, drug A which is studied in DAIPF trial, does not address pathophysiology of IPF in Japan, but works on European patients which have a different type of IPF in terms of its pathophysiology.
Other examples are diseases in which their definition is based on an endpoint, such as COPD, or Asthma. There might be hundreds of different pathways that end in obstruction of the pulmonary tree, if it is persistent we call it COPD; if it is not persistent, we call it asthma. However, all of these diseases are a spectrum of diverse disorders clustered under the same umbrella, with a varied combination of airway hyperactivity, over secretion, inflammation, restriction, and sometimes fibrosisCitation19–23.
Subgroups
Patients are not the same, and when there is a small subgroup of a huge study, they might get buried in the balk of the others. It is also possible that the weight of a subgroup positively or negatively affects the results of certain studies. This is why subset/subgroup analyses are important to provide us a precise understanding of our trials. The same might happen when we are entering a small single center study into a meta-analysis, while the weight of this study is extremely low, it will be neglected, and covered by the other powerful studies that were also included in the meta-analysis. This can also be seen when there are neglected confounding factors that were not taken into account at the time of the meta-analysis criterion definition. In the first scenario, Japanese patients were a small subgroup of DAIPF trial, and hence their negative findings have been buried under the bulk of huge study population.
Here is another real life example
A paradoxical result of subgroup analyses of studies on tight blood glucose control in diabetic patients is a perfect example of this issueCitation24. Let us discuss the results of a number of very well known studies on intensive glycemic control of diabetic patients in terms of hypoglycemia, cardiovascular disease (CVD), mortality, and morbidity.
The better you control the blood glucose the better the outcomes of the patients. This statement was the rational basis of a large amount of huge clinical trials (ACCORD, UKPDS, VADT)Citation24. The ACCORD trial is renowned for its premature termination after around 3.5 years because of higher mortality in the intensive group, targeting a hemoglobin A1C <6% (42 mmol/mol)Citation25. Although the intensive versus standard therapy groups had a similar percent increase in the rates of severe hypoglycemia in both age ranges, the older group had higher absolute ratesCitation26. Paradoxically, the age-specific analysis of the ACCORD trial revealed an increased risk (HR = 1.71; 95% CI 1.17–2.50) in cardiovascular mortality the in younger subgroup while no increase has been observed in the older group with intensive therapy. The difference between the two groups was statistically significant (p value for interaction = 0.03)Citation24,Citation25,Citation27,Citation28.
As stated by the ACCORD study group, “As compared with standard therapy, the use of intensive therapy to target normal glycated hemoglobin levels for 3.5 years increased mortality, and did not significantly reduce major cardiovascular events. These findings identify a previously unrecognized harm of intensive glucose lowering in high-risk patients with type 2 diabetes”Citation29.
Compared to epidemiologic studies and the DCCT follow-up study, none of the ACCORD, UKPDS, or VADT trials showed a significant benefit of intensive glycemic control on CVD in patients with type 2 diabetes. Many of the involved patients in these studies were patients with established diabetes (mean duration 8–11 years) with other CVD risk factors, and hence atherosclerosis was already stablished in majority of them. Nevertheless, subset analyses revealed a significant benefit of intensive glycemic control on patients with a shorter duration of diabetes, lower HbA1C, and absence of known CVDCitation24.
As supportive evidence, the DCCT follow-up study showed that intensive glycemic control in relatively young participants (with no or less other CVD risk factors) was associated with a significant (57%) reduction in major CVD outcomes. Additionally, according to the DCCT-EDIC (Epidemiology of Diabetes Interventions and Complications) 9 years of follow-up beyond the end of the DCCT was required to become statistically significantCitation30–33.
You may note that although there are internationally well-known scientists who designed these huge trials, there were many unpredicted confounding factors. Although they tried to define the inclusion criteria, additional heterogeneity of their participants in other factors affected the interpretation of their results. Puzzled by the results of studies like ACCORD, scientist relocated back to the importance of the personalized approach in management of patients with diabetes.
Evidence based medicine, and guidelines
We should treat patients, not their lab findings. Evidence based medicine is not about sticking a mark to a patient, and treating him/her according to the guidelines. Guidelines are “Guide” lines to help us on decision-making, and are not to dictate an inflexible approach. Nevertheless, our clinical decision making should always rely on evidence based medicine.
Personalized medicine
Personalized medicine is a new concept that tries to approach each patient (or a group of patients) through it’s unique characteristics, genetics, and epigenetics. Using genomics with proteomics, personalized medicine scientists are trying to approach patients on a case-by-case basisCitation34. Although the concept is new, the personalized approach to the patients is a principle in patient care traditionally. Patient tailored treatment dates back at least to Hippocrates, and AvicennaCitation14,Citation34–38.
Misunderstanding of evidence-based medicine might negatively influence this wise approach. Alike patients are not the same, groups of patients are not the same, before jumping to personalized medicine we can make a bridge by population centered approached and taking into account differences of our society compared to the available literature, metanalyses, clinical trials and guidelines. We should give value to well designed, powerful trials of our population and take them into account during clinical decision making. These observed differences may tend to better understanding of disease pathophysiology in our population and help us to tailor our approach based on our population characteristics. Furthermore, as discussed above, we should cautiously interpret multicenter trial and metanalyses, because when they are a small subgroup of a huge study they might bury in the balk of the others. The same might happen when we are entering a small single center study into a meta-analysis, while the weight of these studies are extremely low, they will be neglected, and covered by the other powerful studies that were also entered to the meta-analysis.
Just like each patient, society, hospital, and country, each population have unique characteristics that might not be addressed in multicenter studies as well as meta-analyses, and as a result might not be reflected in evidence based guidelines. Hence, results of local well-designed and well-conducted powerful clinical trials are extremely important. From a pragmatic point of view, when a powerful well-designed, well-conducted trial results contradict multicenter studies; we should particularly care about the results of the local study. We may or may not discover the reasoning for the difference in our findings, but practically we should approach our patients with respect to our local study, rather than multicenter larger studies. Because, whatever we observed in our patients will be repeated in clinic. For example, imagine that only the generic product of Drug B is available in your country. Imagine the generic product is not working for whatever reason, or there is a variant of an enzyme that is prevalent in your population, and deactivates the active component of drug B. No matter what the reason is, the results will be repeated in the clinic. Until confounding factor is found, and you approach your patient with respect to that (Drug C scenario), it is recommended not to prescribe drug B for your patients. If the generic product is not working, switch to the brand named drug. If an enzyme is inactivating the drug try to block it, and then prescribe it (Drug C scenario: add clavulanic acid to amoxicillin).
Enzyme polymorphism
We discussed the probability in the differences of wild type enzyme effect towards certain medications, yet statistically not recommendable a case-to-case approach. Deductive and discarding approach would be more feasible. In the absence of response to the typical, resort to the atypical. If genetic studies would be more expeditious, and accessible, case-to-case approach would be ideal. Nevertheless, in near future we might move forward toward more personalized approach. At the moment we can conduct population studies for new medications to tailor them based on characteristics of our society of interest.
Caution: limitations of this type of interpretation to evidence-based medicine and how it could be misused
Meta-analyses do have their time and place. A strong meta-analysis will increase the sample of a study, and hence increase its power. When you increase a sample size, the effects of interests are amplifiedCitation3,Citation39. If done correctly, meta-analyses can also help resolve conflicts/doubts between single studies, and can lead to greater conclusionsCitation40,Citation41. Multicenter studies also have their share of benefits. They make it possible to answer questions that require larger sample sizes, at the same time allowing comparison between different study locations, and provide insight to study generalizabilityCitation42. Multicenter studies also make it possible to develop networking teams, mentorships, and allow investigators to freely share resources, ideas, and expertiseCitation42.
It is crucial to emphasize that we are not writing against evidence based medicine (EBM), multicenter clinical trials and metanalyses. We want to go beyond that. Sometimes our understanding of EBM needs revision. We want to simply discuss those occasional situations when local powerful studies are more important than multicenter studies and metanalyses, and we want to emphasize on the personalized, localized approach in medicine in twenty first century, while all these has to happen based on scientifically proved evidences. This concept should be used with precaution and should never been abused against EBM. Indeed, multicenter trials and metanalyses will stay the foundation of our EBM. We simply want to add some precautions to their interpretation and application in tailoring clinical decision making, developing locally important research, and health policy making.
Conclusion
Although multicenter studies along with meta-analyses revolutionized clinical research, the importance of local, powerful, well designed, and well-conducted clinical trials should not be neglected even if their results contradict the larger multicenter studies and meta-analyses.
A patient/society tailored approach with respect to evidence (emphasize on local valuable findings) is warranted, from a pragmatic point of view, no matter if the results of the other bigger studies (RCTs and meta-analyses) contradict your findings. When or if there is a persistent confounding factor (even if you do not know it) that you can not eliminate it in your practice, you should still take it into account. A discordance between large multicenter clinical trials/Metanalyses and well-designed powerful RCTs in your population should raise concern regarding applicability of their results in your population and calls for cautious interpretation and application of their results. It warrants tailoring your approach, and considering further investigation in your population and may warrant changing health policy and local guidelines.
Transparency
Declaration of funding
This paper was not funded.
Declaration of financial/other relationships
The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties. Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.
Acknowledgements
We would like to thank Dr. M Zakaria Pezeshki and Dr. Vahid Haghpanah for their enlightening comments on this article.
References
- Haines SJ, Walters BC. What is metanalysis? Surg Neurol. 1995;44(6):581–582.
- Messerer D, Porzsolt F, Hasford J, et al. [Advantages and problems of multicenter therapy studies exemplified by a study of the treatment of metastasizing renal cell carcinoma with recombinant interferon-alpha-2c] . Onkologie. 1987;10(1):43–49.
- Llaurado JG. Metanalysis: a new scientific endeavor. Int J Biomed Comput. 1988;22(2):77–79.
- Jobst B. The art of switching antiepileptic medications: keep trying or just let it be. Epilepsy Curr. 2013;13(2):76–77.
- Wang L, Wang LL, Shang D, et al. Gene polymorphism of DNA repair gene X-ray repair cross complementing group 1 and xeroderma pigmentosum group D and environment interaction in non-small-cell lung cancer for Chinese nonsmoking female patients. Kaohsiung J Med Sci. 2019;35(1):39–48.
- Elwan N, Assal F, Elfert A, et al. Genetic susceptibility in family members of Egyptian Hepatitis C virus infected patients: role of interleukin-12 B gene polymorphism. Infect Disord Drug Targets. 2019;19(1):81–87.
- Friedman LS, Szabo CI, Ostermeyer EA, et al. Novel inherited mutations and variable expressivity of BRCA1 alleles, including the founder mutation 185delAG in Ashkenazi Jewish families. Am J Hum Genet. 1995;57(6):1284–1297.
- Francis J, Zvada SP, Denti P, et al. A population pharmacokinetic analysis shows that arylacetamide deacetylase (AADAC) gene polymorphism and HIV infection affect the exposure of rifapentine. Antimicrob Agents Chemother. 2019;63(4):e01964–18.
- Levy-Lahad E, Catane R, Eisenberg S, et al. Founder BRCA1 and BRCA2 mutations in Ashkenazi Jews in Israel: frequency and differential penetrance in ovarian cancer and in breast-ovarian cancer families. Am J Hum Genet. 1997;60(5):1059–1067.
- Bahar AY, Taylor PJ, Andrews L, et al. The frequency of founder mutations in the BRCA1, BRCA2, and APC genes in Australian Ashkenazi Jews: implications for the generality of U.S. population data. Cancer. 2001;92(2):440–445.
- Tung N, Garber JE, Lincoln A, et al. Frequency of triple-negative breast cancer in BRCA1 mutation carriers: comparison between common Ashkenazi Jewish and other mutations. J Clin Oncol. 2012;30(35):4447–4448.
- Walsh T, Mandell JB, Norquist BM, et al. Genetic predisposition to breast cancer due to mutations other than BRCA1 and BRCA2 founder alleles among Ashkenazi Jewish women. JAMA Oncol. 2017;3(12):1647–1653.
- Check Hayden E. The rise and fall and rise again of 23andMe. Nature. 2017;550(7675):174–177.
- Stoekle HC, Mamzer-Bruneel MF, Vogt G, et al. 23andMe: a new two-sided data-banking market model. BMC Med Ethics. 2016;17:19.
- Harari S, Madotto F, Caminati A, et al. Epidemiology of idiopathic pulmonary fibrosis in northern Italy. PloS One. 2016;11(2):e0147072.
- Olson AL, Swigris JJ. Idiopathic pulmonary fibrosis: diagnosis and epidemiology. Clin Chest Med. 2012;33(1):41–50.
- Sauleda J, Nunez B, Sala E, et al. Idiopathic pulmonary fibrosis: epidemiology, natural history, phenotypes. Med Sci. 2018;6(4):110.
- Huang H, Peng X, Zhong C. Idiopathic pulmonary fibrosis: the current status of its epidemiology, diagnosis, and treatment in China. Intractable Rare Dis Res. 2013;2(3):88–93.
- Sun L, Chen Y, Wu R, et al. Changes in definition lead to changes in the clinical characteristics across COPD categories according to GOLD 2017: a national cross-sectional survey in China. Int J Chron Obstruct Pulmon Dis. 2017;12:3095–3102.
- Vestbo J. COPD: definition and phenotypes. Clin Chest Med. 2014;35(1):1–6.
- Petsonk EL, Hnizdo E, Attfield M. Definition of COPD GOLD stage I. Thorax. 2007;62:1107–1108.
- Perez-Padilla R, Wehrmeister FC, Montes de Oca M, et al. Instability in the COPD diagnosis upon repeat testing vary with the definition of COPD. PloS One. 2015;10(3):e0121832.
- Sin DD, Miravitlles M, Mannino DM, et al. What is asthma-COPD overlap syndrome? Towards a consensus definition from a round table discussion. Eur Respir J. 2016;48(3):664–673.
- Skyler JS, Bergenstal R, Bonow RO, et al. Intensive glycemic control and the prevention of cardiovascular events: implications of the ACCORD, ADVANCE, and VA diabetes trials: a position statement of the American Diabetes Association and a scientific statement of the American College of Cardiology Foundation and the American Heart Association. Circulation. 2009;119(2):351–357.
- Calles-Escandon J, Lovato LC, Simons-Morton DG, et al. Effect of intensive compared with standard glycemia treatment strategies on mortality by baseline subgroup characteristics: the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial. Diabetes Care. 2010;33(4):721–727.
- Miller ME, Williamson JD, Gerstein HC, et al. Effects of randomization to intensive glucose control on adverse events, cardiovascular disease, and mortality in older versus younger adults in the ACCORD Trial. Dia Care. 2014;37(3):634–643.
- Schwartz AV, Margolis KL, Sellmeyer DE, et al. Intensive glycemic control is not associated with fractures or falls in the ACCORD randomized trial. Diabetes Care. 2012;35(7):1525–1531.
- Najman DM. Intensive therapy of type 2 diabetes (ACCORD trial) (OCTOBER 2008. CCJM. 2009;76(2):83.1–4.
- Action to Control Cardiovascular Risk in Diabetes Study Group, Gerstein HC, Miller ME, et al. Effects of intensive glucose lowering in type 2 diabetes. N Eng J Med. 2008;358:2545–2559.
- Braffett BH, Dagogo-Jack S, Bebu I, et al. Association of insulin dose, cardiometabolic risk factors, and cardiovascular disease in type 1 diabetes during 30 years of follow-up in the DCCT/EDIC study. Diabetes Care. 2019;42(4):657–664.
- Writing Group for the DERG. Coprogression of cardiovascular risk factors in type 1 diabetes during 30 years of follow-up in the DCCT/EDIC study. Diabetes Care. 2016;39:1621–1630.
- Diabetes C. Complications trial/epidemiology of diabetes I and complications study research G. intensive diabetes treatment and cardiovascular outcomes in type 1 diabetes: the DCCT/EDIC study 30-year follow-up. Diabetes Care. 2016;39:686–693.
- Gubitosi-Klug RA, Braffett BH, White NH, et al. Risk of severe hypoglycemia in type 1 diabetes over 30 years of follow-up in the DCCT/EDIC study. Diabetes Care. 2017;40(8):1010–1016.
- Vogt H, Hofmann B, Getz L. Personalized medicine: evidence of normativity in its quantitative definition of health. Theor Med Bioeth. 2016;37(5):401–416.
- Suwinski P, Ong C, Ling MHT, et al. Advancing personalized medicine through the application of whole exome sequencing and big data analytics. Front Genet. 2019;10:49.
- El-Alti L, Sandman L, Munthe C. Person centered care and personalized medicine: irreconcilable opposites or potential companions? Health Care Anal. 2019;27(1):45–59.
- Eyerich K, Tuting T. [Personalized medicine]. Hautarzt. 2019;70(1):4
- Jorgensen JT. twenty years with personalized medicine: past, present, and future of individualized pharmacotherapy. The Oncologist. 2019;24(7):e432–e440.
- Walker E, Hernandez AV, Kattan MW. Meta-analysis: its strengths and limitations. Cleve Clin J Med. 2008;75(6):431–439.
- Stevens RD, Wu CL. Strengths and limitations of meta-analysis. J Cardiothorac Vasc Anesth. 2007;21(1):1–2.
- Flather MD, Farkouh ME, Pogue JM, et al. Strengths and limitations of meta-analysis: larger studies may be more reliable. Control Clin Trials. 1997;18(6):568–579.
- Cheng A, Kessler D, Mackinnon R, et al. Conducting multicenter research in healthcare simulation: lessons learned from the INSPIRE network. Adv Simul (Lond). 2017;2:6.