A Comparison of Propensity Score Weighting Methods for Evaluating the Effects of Programs With Multiple Versions

ABSTRACT

This Monte Carlo simulation study compares methods to estimate the effects of programs with multiple versions when assignment of individuals to program version is not random. These methods use generalized propensity scores, which are predicted probabilities of receiving a particular level of the treatment conditional on covariates, to remove selection bias. The results indicate that inverse probability of treatment weighting (IPTW) removes the most bias, followed by optimal full matching (OFM), and marginal mean weighting through stratification (MMWTS). The study also compared standard error estimation with Taylor series linearization, bootstrapping and the jackknife across propensity score methods. With IPTW, these standard error estimation methods performed adequately, but standard errors estimates were biased in most conditions with OFM and MMWTS.

KEYWORDS:

• Inverse probability of treatment weighting
• marginal mean weighting through stratification
• multiple treatments
• optimal full propensity score matching
• propensity score analysis
• quasi-experimental designs
• selection bias