Performance evaluation of regression splines for propensity score adjustment in post-market safety analysis with multiple treatments

ABSTRACT

Observational studies provide a core resource in assessing post-market drug safety and effectiveness. Propensity scores are a predominant method for confounding adjustment to achieve unbiased estimation of average treatment effects in observational data. However, the use of propensity score methods has been limited to comparing two treatment groups, while medical situations frequently present with multiple treatment options. Inverse probability of treatment weighting (IPTW) is a popular propensity score adjustment method, but its performance degrades with decreased positivity leading to extreme weights, a problem that can be amplified with multiple treatment groups. Meanwhile, regression on a spline of the propensity score has shown favorable performance compared to other propensity score methods in recent studies involving two treatments. This project utilizes a simulation study to compare IPTW and propensity score splines as adjustment methods in a three-treatment setting. We test a variety of spline methods, including natural cubic splines with varying numbers of interior knots, and thin-plate regression splines. We vary several parameters across simulations, including the degree of propensity score overlap among treatment groups, treatment prevalence, outcome prevalence, and true marginal relative risk. We assess methods based on their bias, root mean squared error, and coverage of the true marginal relative risk across simulations. We find that all methods perform similarly well when there is good propensity score distribution overlap. However, with even moderate decrease in overlap or low outcome prevalence, IPTW produces more biased estimates and higher variance than propensity score splines. Low treatment prevalence or unequal treatment prevalences across groups also worsens IPTW performance. Overall, a natural cubic spline with a relatively small number of interior knots provides good performance across a range of simulations.

KEYWORDS:

• Propensity score
• multiple treatment
• spline
• inverse probability of treatment weighting

Disclaimer

This manuscript reflects the views of the authors and should not be construed to represent FDA’s views or policies.

Supplementary material

Supplemental data for this article can be accessed here.