Real-world research and the role of observational data in the field of gynaecology – a practical review

Figures & data

Figure 1. Observational study and randomised clinical trial (experimental) designs.

Table 1. Strengths and weaknesses of different types of randomised clinical trials and observational studies [3,5–8].

Study type	Strengths	Weaknesses
Randomised clinical trials
Parallel group: Participants randomly allocated to either treatment/intervention or control/placebo groups for study duration Best for: Investigating the causal effect of an intervention	• Distribution of confounders is unbiased • Potential concealment of allocation (blinding) from participants and clinicians helps to reduce selection, allocation and outcome (performance) biases • Statistical analysis is facilitated • Complete baseline information (e.g., body-mass index, smoking status, etc.) can be obtained	• Costly in terms of time and money • Potential selection bias (e.g., healthy volunteer effect – participants are healthier than general population, leading to lower outcome rates than expected) • Hawthorne effect (an increase in outcome under study in participants who are aware of being observed) • Possibility of ethical problems (e.g., using a placebo group as a control when testing treatment for a life-threatening condition such as haemorrhage) • Restricted population size • Misclassification of exposure, particularly in long-term trials • Difficult or impossible in minors
Crossover: Participants randomly allocated to either treatment/intervention or control/placebo groups, but then switched between groups at a predetermined time point during the study	• Participants act as own controls; reduced sample size required due to statistical advantage • All participants receive treatment at least some of the time • Maintenance of blinding (benefits of blinding and best application are as described above)	• Relevant only if outcome (e.g., symptoms) is reversible over time • Placebo or alternative treatment received by all patients at some point • Prolonged or unknown washout period (time required for elimination of drug and/or its metabolites from the bodies of participants) • Unsuitable for treatments with permanent effects
Field-randomised: Similar to either parallel-group or crossover randomised clinical trials (not necessarily requiring placebo control), but generally involve participants who are living at home in their normal environment, rather than being ‘captive’ in hospitals or outpatient clinics [6] Best for: Investigating preventive measures, such as immunisations or health education	• Less stringent inclusion/exclusion criteria allow wider recruitment pool • Results are more generalisable to broader population (external validity) than those from ‘clinical’ randomised trials	• Less stringent inclusion/exclusion criteria can make data analysis complex
Community-randomised: Participants from specific geographic communities or ‘clusters’ (e.g., caregivers, hospitals, residential homes, etc.) randomised to either treatment/intervention or control arms; generally characterised by small numbers of clusters with a large number of patients in each cluster [5] Best for: Evaluating delivery of health care services, effects of educational interventions, or effects of organisational changes.	• Useful for comparative effectiveness research – focused on understanding effects of an intervention in a routine clinical setting • Pragmatic method of measuring effectiveness of an intervention on a large scale • Useful when randomisation at individual level is inappropriate or impossible	• Potential complexity • Intracluster correlation requires greater patient numbers than in individual randomised clinical trials • Potential for selection bias (patients may not have access to treatments or procedures available only at other locations)
Observational (‘real-world’), prospective or retrospective studies^a
Cohort: No allocation of treatment exposure made by researcher Best for: Investigating the absolute effect of risk factors on outcomes	• Ethically sound • Subject matching possible • Outcome-assessment standardisation possible • Administration simpler and cheaper than randomised clinical trial • Allows inclusion of minors	• No randomisation • Confounders • Blinding is problematic/cannot be done • Prolonged follow-up or large sample sizes required for rare outcomes • Complete background information (e.g., body-mass index, smoking status, etc.) often difficult to obtain/manage; such factors may be a source of bias (Note that the weaknesses above, for a cohort study, apply to any observational study to a greater or lesser extent)
Cross-sectional: Investigates relationship between disease (or other health-related characteristic) and other variable of interest in defined population at same time – exposure and outcomes determined simultaneously Best for: Quantifying prevalence of a disease/risk factor, or diagnostic test accuracy	• Low cost • Ethically sound	• No randomisation • Cannot confirm temporal association • Susceptible to recall bias • Confounders • Potential for imbalanced group sizes
Case-control: Involves selection of patients with certain outcome/disease, and matched controls lacking that outcome/disease, followed by analysis of whether exposure of subjects to an investigative factor has occurred; outcome/disease is fixed but exposure occurs at random [7] Nested case-control: Variant of the case-control study adopted for its cost-effective and objective sampling strategy [8]	• Low cost and rapid • Ethically sound	• No randomisation • Exposure status relies on recall in interview-based studies • Confounders Potential bias in selection of cases and controls
Self-controlled case series: A case-only method in which individuals act as their own controls [7] Best for: Acute events and transient exposures for which periods of exposure risk can be clearly defined	• No separate controls are required • Automatic control of any fixed confounders	• Exposure probability not necessarily affected by the prior occurrence of an outcome • Observation period may not be independent of timing of events

Adapted from Centre for Evidence-based Medicine (CEBM) [3].

^a Prospective studies investigate outcomes (e.g., disease development) during a study period and relate this to other factors (e.g., suspected risk or protective factors); retrospective studies do the reverse, by investigating suspected risk or protection-factor exposure in relation to a predetermined outcome at the study outset. For example, in a prospective cohort study, the incidence or prevalence of the outcome of interest in the subjects is unknown at the time that investigators begin identifying subjects and collecting exposure information, whereas in a retrospective cohort study, the outcomes have already occurred at the time that investigators plan the study and begin identifying subjects.

Table 2. CEBM grades of recommendation.

Grade	Basis of recommendation
A	Level 1 studies (e.g., homogeneous^a systematic reviews of randomised clinical trials or prospective cohort studies; or individual randomised clinical trial with narrow confidence intervals; or CDR validated in different populations)
B	Level 2 (e.g., homogeneous systematic reviews of retrospective cohort studies or untreated control groups in randomised clinical trials; or individual cohort study, including low-quality randomised clinical trial) or Level 3 studies (e.g., homogeneous systematic reviews of case-control studies; or individual case-control studies) or extrapolations from level 1 studies
C	Level 4 studies (e.g., case series, poor quality cohort and case-control studies; superseded or non-independent reference standards; lack of sensitivity analysis) or extrapolations from levels 2 or 3 studies
D	Level 5 evidence (e.g., expert opinions without critical appraisal, based on physiology, basic bench research, or ‘first principles’) or troublingly inconsistent or inconclusive studies of any level

Adapted from CEBM [11].

^a Homogeneity indicates freedom from worrisome variations (heterogeneity) in the directions and degrees of results between individual studies. CDR, clinical decision rule (algorithm or scoring system leading to a prognostic decision or diagnostic category).

Figure 2. The evidence-based medicine triad.

Table 3. Circumstances under which observational studies are valuable and can be supportive of randomised clinical trials.

Circumstance	Study examples relevant to contraception and reproductive health care	Impact on clinical practice integrated into guidelines
When large studies are needed to ascertain an outcome (e.g., to assess infrequent or long-term effects)	• Royal College of General Practitioners' Oral Contraception Study – Cohort study [21,22] Objective: To determine whether the mortality risk among women who have used oral contraceptives differs from that of never users • Nurses’ Health Study – Cohort study [23] Objective: To determine whether use of oral contraceptives is associated with all-cause and cause-specific mortality • Etude Epidémiologique auprès de femmes de l’Education Nationale (E3N) study – Cohort study [24,25] Objective: To investigate risk factors for cancer in women • Risk factors for venous thromboembolism in 1.3 million pregnancies: a nationwide prospective cohort – Cohort study [26] Objective: To quantify risk factors for venous thromboembolism during pregnancy and the puerperal period • Impact of estrogen type on cardiovascular safety of combined oral contraceptives – Cohort study [27] Objective: To assess the risks of serious cardiovascular events (including VTE, PE, ATE, AMI, CVA) associated with short- and long-term use of DNG/EV, oCOC and LNG in a study population representative of actual users of the individual preparations	Provides important information about benefit-risk evaluation for the health practitioner
In instances when treatment adherence might have an impact on outcome	• Prescribing of cyproterone acetate/ethinylestradiol in UK general practice: a retrospective descriptive study using The Health Improvement Network – Descriptive study [28] Objective: To investigate prescribing patterns of cyproterone acetate/ethinylestradiol in the UK before and after the 2013 prescribing guidance • The continuation rates of long-acting reversible contraceptives in UK general practice using data from The Health Improvement Network – Cohort study [29] Objective: To determine the continuation rates of new users of long-acting reversible contraceptive (LARC) methods in the UK, using data from general practice • Age, parity, history of abortion and contraceptive choices affect the risk of repeat abortion – Cohort study [30] Objective: To determine whether use of an IUD/IUS post abortion is more effective than user-dependent contraceptive methods in reducing the need of further induced abortion	Implementation research is very important for obtaining information about the behaviour of the target group in relation to the outcome
When training of providers might have an impact on outcome	• Which factors are associated with trainees’ confidence in performing obstetric and gynaecological ultrasound examinations? – Cross-sectional study [31] Objective: To explore the association between clinical training characteristics and trainees’ level of confidence in performing ultrasound scans independently	Adaptation of programmes/protocols in clinical practice
In instances when a timely result is needed	• Maternal vaccination against H1N1 influenza and offspring mortality: population-based cohort study and sibling design – Cohort study [32] Objective: To investigate the safety of H1N1 influenza vaccination during pregnancy • Oral contraceptives, venous thrombosis, and varicose veins. Royal College of General Practitioners' Oral Contraception Study – Cross-sectional study [33] Objective: To assess the relationship between oral contraceptive use and risk of idiopathic deep and superficial thrombosis in the leg	Important in situations in which it would be politically or ethically unacceptable to deny access to an intervention
When multiple treatment solutions are available	• Population-based evaluation of the effectiveness of two regimens for emergency contraception – Cohort study [34] Objective: To estimate and compare the effectiveness of the levonorgestrel and Yuzpe regimens for hormonal emergency contraception in routine clinical practice • Endometriosis and its treatment with danazol or lupron in relation to ovarian cancer – Case-control study [35] Objective: To test whether exogenous androgens used for endometriosis may be associated with ovarian cancer	Impact on guidelines regarding the use of emergency contraception, and androgen use in endometriosis
When wanting to explore population subsets	• Danish Sex Hormone Register Study: association of hormonal contraception with depression – Cohort study [36] Objective: To determine whether hormonal contraception is positively associated with subsequent use of antidepressants and a diagnosis of depression at a psychiatric hospital	Providing associations and hypotheses, which have to be explored further

Adapted and expanded from Dreyer et al. [20]. AMI: acute myocardial infarction; ATE: acute thromboembolism; CVA: cerebrovascular accidents; EV: estradiol valerate; DNG: dienogest; IUD: intrauterine device; IUS: intrauterine system; LNG: levonorgestrel; oCOC: other combined oral contraceptive; PE: pulmonary embolism; VTE: venous thromboembolism.

Table 4. Limitations of randomised clinical trials versus observational studies.

Randomised clinical trials	Observational studies
Expensive	Lower costs in regulatory and administrative terms
Can be time consuming	Usually less time consuming (important when public safety is at stake, and regulatory actions must be taken)
Strict exclusion and inclusion criteria may cause study population to differ from target population	Representative of target population
Treatment efficacy is assessed, involving strict treatment strategies and/or patient monitoring that may not be practical in a real-world setting	Treatment effectiveness under real-world conditions (routine clinical practice) is assessed
Potentially unethical (e.g., teratogenic effects)	Avoids ethical issues in e.g., the study of noxious substances
Achieving adequate sample sizes for statistical power can be difficult	Large sample sizes^a ensure statistical power (the large sample has to be selected or weighted to represent the population from which sample is drawn)

^a While large sample size can be a strength of observational studies, it does carry a risk of a given outcome being ‘over-dramatised’, particularly if that outcome is negative, such as death.

Figure 3. Causal diagram of factors confounding the relationship between maternal obesity and birth defects. Adapted from Hernán et al. [38].

Figure 4. Flowchart exemplifying typical steps for ascertainment and case validation when using a primary care database in an observational study. GP: general practitioner; GPRD: General Practice Research Database; THIN: The Health Improvement Network.

CEBM.net [Internet]. Oxford: Centre for Evidence-Based Medicine; [cited 2017 Feb 26]. Available from: http://www.cebm.net/study-designs/

 Google Scholar

OUP. Oxford; Oxford University Press. In: Smith PG, Morrow RH, Ross DA, editors. Field Trials of Health Interventions: A Toolbox. 3rd ed.; 2015 Jun 1. Chapter 1, Introduction to field trials of health interventions [cited 2017 Mar 19]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK305510/

 Google Scholar

NPCNOW.org [Internet]. Washington: National Pharmaceutical Council. Making informed decisions: assessing the strengths and weaknesses of study designs and analytic methods for comparative effectiveness research. Velengtas P, Mohr P, Messner DA, editors. 2012 February [cited 2017 Mar 19]. Available from: http://www.npcnow.org/system/files/research/download/experimental_nonexperimental_study_final.pdf

 Google Scholar

Petersen I, Douglas I, Whitaker H. Self controlled case series methods: an alternative to standard epidemiological study designs. BMJ. 2016;354:i4515.

 PubMed Web of Science ®Google Scholar

Cai T, Zheng Y. Evaluating prognostic accuracy of biomarkers in nested case-control studies. Biostatistics. 2012;13:89–100.

 PubMed Web of Science ®Google Scholar

CEBM.net [Internet]. Oxford: Centre for Evidence-Based Medicine; [cited 2017 Feb 26]. Available from: http://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/

 Google Scholar

Hannaford PC, Iversen L, Macfarlane TV, et al. Mortality among contraceptive pill users: cohort evidence from Royal College of General Practitioners' Oral Contraception Study. BMJ.2010;340:c927

 PubMed Web of Science ®Google Scholar

Iversen L, Sivasubramaniam S, Lee AJ, et al. Lifetime cancer risk and combined oral contraceptives: the Royal College of General Practitioners' Oral Contraception Study. Am J Obstet Gynecol. 2017;216:580.e1–580.e9.

 PubMed Web of Science ®Google Scholar

Charlton M, Rich-Edwards JW, Colditz GA, et al. Oral contraceptive use and mortality after 36 years of follow-up in the Nurses’ Health Study: prospective cohort study. BMJ. 2014;349:g6356.

 PubMed Web of Science ®Google Scholar

Fournier A, Berrino F, Clavel-Chapelon F. Unequal risks for breast cancer associated with different hormone replacement therapies: results from the E3N cohort study. Breast Cancer Res Treat. 2008;107:103–111.

 PubMed Web of Science ®Google Scholar

Cadeau C, Fournier A, Mesrine S, et al. Interaction between current vitamin D supplementation and menopausal hormone therapy use on breast cancer risk: evidence from the E3N cohort. Am J Clin Nutr. 2015;102:966–973.

 PubMed Web of Science ®Google Scholar

Virkus RA, Løkkegaard E, Lidegaard Ø, et al. Risk factors for venous thromboembolism in 1.3 million pregnancies: a nationwide prospective cohort. PLoS One. 2014;9:e96495.

 PubMed Web of Science ®Google Scholar

Dinger J, Do Minh T, Heinemann K. Impact of estrogen type on cardiovascular safety of combined oral contraceptives. Contraception. 2016;94:328–339.

 PubMed Web of Science ®Google Scholar

Cea Soriano L, Asiimwe A, García Rodríguez LA. Prescribing of cyproterone acetate/ethinylestradiol in UK general practice: a retrospective descriptive study using The Health Improvement Network. Contraception. 2017;95:299–305.

 PubMed Web of Science ®Google Scholar

Cea Soriano L, Wallander MA, Andersson S, et al. The continuation rates of long-acting reversible contraceptives in UK general practice using data from The Health Improvement Network. Pharmacoepidemiol Drug Saf. 2015;24:52–58.

 PubMed Web of Science ®Google Scholar

Heikinheimo O, Gissler M, Suhonen S. Age, parity, history of abortion and contraceptive choices affect the risk of repeat abortion. Contraception. 2008;78:149–154.

 PubMed Web of Science ®Google Scholar

Tolsgaard MG, Rasmussen MB, Tappert C, et al. Which factors are associated with trainees' confidence in performing obstetric and gynecological ultrasound examinations?. Ultrasound Obstet Gynecol. 2014;43:444–451.

 PubMed Web of Science ®Google Scholar

Ludvigsson JF, Ström P, Lundholm C, et al. Maternal vaccination against H1N1 influenza and offspring mortality: population based cohort study and sibling design. BMJ. 2015;351:h5585.

 PubMed Web of Science ®Google Scholar

RCGPOCS. Oral contraceptives, venous thrombosis, and varicose veins. Royal College of General Practitioners' Oral Contraception Study. J R Coll Gen Pract. 1978;28:393–399.

 PubMedGoogle Scholar

Leung VW, Soon JA, Lynd LD, et al. Population-based evaluation of the effectiveness of two regimens for emergency contraception. Int J Gynaecol Obstet. 2016;133:342–346.

 PubMed Web of Science ®Google Scholar

Cottreau CM, Ness RB, Modugno F, et al. Endometriosis and its treatment with danazol or lupron in relation to ovarian cancer. Clin Cancer Res. 2003;9:5142–5144.

 PubMed Web of Science ®Google Scholar

Skovlund CW, Mørch LS, Kessing LV, et al. Association of hormonal contraception with depression. JAMA Psychiatry. 2016;73:1154–1162.

 PubMed Web of Science ®Google Scholar

Dreyer NA, Tunis SR, Berger M, et al. Why observational studies should be among the tools used in comparative effectiveness research. Health Affairs. 2010;29:1818–1825.

 PubMed Web of Science ®Google Scholar

Hernán MA, Hernández-Díaz S, Werler MM, et al. Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology. Am J Epidemiol. 2002;155:176–184.

 PubMed Web of Science ®Google Scholar