Control of confounding in the analysis phase – an overview for clinicians

Figures & data

Table S1 Summary of the pros and cons of five methods used to control confounding in observational studies

Method	Advantages	Disadvantages
Stratification	Simple and transparent method Provides insight into the data at a detailed level Few assumptions	Limitations on the number of strata that are practically manageable Sensitive to sparse data in strata
Standardization	Specifically developed to compare rates and ratios Few assumptions Variants (direct and indirect standardization) developed to meet limitations in data availability	Sensitive to the choice of standard or reference population Cumbersome analysis with many confounders Sparse data may result in unstable rates across strata
Multivariable regression analysis	Easy to include a large number of potential confounders with standard statistical software Great flexibility when exploring effects^Citation1 Outcome can be of any type (eg, multiple levels)	Many assumptions such as linearity in linear models, no collinearity between factors, normality and homoscedasticity of error terms Less efficient when few events/outcomes per variable^Citation2 In some variants, variable selection is data driven with the risk of discarding important confounders
PS methods	Robust method when exposure is common and outcome is rare^Citation3 Outcome can be of any type (eg, multiple levels) Robust when few events/outcomes per variable (<8)^Citation4 The PS model may be changed until groups are comparable^Citation5 Can be used for calculation of ratios, risk differences and relative risks	Exposure must be a categorical variable (information potentially lost) Appropriate balancing of PSs between comparison groups may not be possible Data set may be reduced when balancing the PSs of comparison groups (information loss) Less efficient when exposure is rare^Citation6
HD-PS method	The very large number of variables included in the analysis may comprise proxies for unmeasured confounders (although there is no guarantee) Outcome can be of any type (eg, multiple levels)	Data greediness Complex procedure with many steps in the selection of confounders Possible increase in the risk of including variables that are not confounders

Abbreviations: PS, propensity score; HD-PS, high-dimensional propensity score.

Table S2 Summary of the methodological pros and cons of four different types of PS methods

Method	Methodology	Advantages	Disadvantages
Stratification	People are assigned to a stratum based upon their PS. Strata are typically defined by percentiles of the PS, eg, quintiles. Hence, within each stratum, treated and untreated people roughly share the same characteristics. A treatment effect is calculated within each stratum, and the overall effect is a weighted average across strata The typical approach estimates ATE^Citation7	Simpler approach in comparison with matching and weighting Across strata, effects are measured	Comparability of treatment groups must be checked for all strata Comparability of all strata may be difficult to obtain Potentially less efficient in removing differences between treatment groups^Citation5 Low number of strata may create residual confounding^Citation7 The range of PS values within strata may create residual confounding^Citation7
Matching	For each treated person, one or more untreated person(s) with a comparable PS are selected. A comparable PS can be defined in different ways, eg, nearest neighbor or caliper width The typical approach estimates ATT^Citation7	Potentially more efficient in providing comparable treatment groups^Citation5	Treated people may not have a match with the untreated people, leading to biased results Only reasonable to use if the untreated-to-treated ratio is large
Covariate adjustment	An outcome regression model is used. As a minimum, the treatment and the PS must be included in the model as independent variables. Other variables may also be included	Simple approach: PS is used to balance treatment groups and is incorporated directly in an outcome regression model	Stronger assumptions than other methods^Citation7 In certain circumstances, it is not clear which effect is estimated^Citation5 No separation of study design and analysis^Citation5
Inverse probability of treatment weighting	Weights are used to create a pseudo-population in which the characteristics are comparable across the treatment groups. Thus, weights are increased for those people who have received the treatment unexpectedly The typical approach estimates ATE^Citation7	Potentially more efficient in providing comparable treatment groups^Citation5	A setting involving treated people with a low PS (or untreated people with a high PS) will generate large weights and variances^Citation8

Abbreviations: PS, propensity score; ATE, average treatment effect for the population (both treated and untreated people); ATT, average treatment effect among treated people.

McNameeRRegression modelling and other methods to control confoundingOccup Environ Med200562750050615961628

 PubMed Web of Science ®Google Scholar

PeduzziPConcatoJKemperEHolfordTRFeinsteinARA simulation study of the number of events per variable in logistic regression analysisJ Clin Epidemiol19964912137313798970487

 PubMed Web of Science ®Google Scholar

ArbogastPGRayWAUse of disease risk scores in pharmacoepidemiologic studiesStat Methods Med Res2009181678018562398

 PubMed Web of Science ®Google Scholar

CepedaMSBostonRFarrarJTStromBLComparison of logistic regression versus propensity score when the number of events is low and there are multiple confoundersAm J Epidemiol2003158328028712882951

 PubMed Web of Science ®Google Scholar

AustinPCAn introduction to propensity score methods for reducing the effects of confounding in observational studiesMultivar Behav Res2011463399424

 PubMed Web of Science ®Google Scholar

GlynnRJGagneJJSchneeweissSRole of disease risk scores in comparative effectiveness research with emerging therapiesPharmacoepidem Dr S201221138147

 Web of Science ®Google Scholar

WilliamsonEMorleyRLucasACarpenterJPropensity scores: from naive enthusiasm to intuitive understandingStat Methods Med Res201221327329321262780

 PubMed Web of Science ®Google Scholar

RobinsJMHernanMABrumbackBMarginal structural models and causal inference in epidemiologyEpidemiology200011555056010955408

 PubMed Web of Science ®Google Scholar