Design, Identification, and Sensitivity Analysis for Patient Preference Trials

Abstract

Social and medical scientists are often concerned that the external validity of experimental results may be compromised because of heterogeneous treatment effects. If a treatment has different effects on those who would choose to take it and those who would not, the average treatment effect estimated in a standard randomized controlled trial (RCT) may give a misleading picture of its impact outside of the study sample. Patient preference trials (PPTs), where participants’ preferences over treatment options are incorporated in the study design, provide a possible solution. In this paper, we provide a systematic analysis of PPTs based on the potential outcomes framework of causal inference. We propose a general design for PPTs with multi-valued treatments, where participants state their preferred treatments and are then randomized into either a standard RCT or a self-selection condition. We derive nonparametric sharp bounds on the average causal effects among each choice-based subpopulation of participants under the proposed design. We also propose a sensitivity analysis for the violation of the key ignorability assumption sufficient for identifying the target causal quantity. The proposed design and methodology are illustrated with an original study of partisan news media and its behavioral impact. Supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

Keywords:

• Causal inference
• External validity
• Nonparametric bounds
• Randomized controlled trial

Acknowledgments

We are grateful to Donald Green and David Nickerson for their helpful comments and suggestions. We also thank participants at the 2014 Society for Political Methodology Summer Meeting, MacMillan-CSAP Workshop at Yale University, and the 3rd ISAT/TSE Conference in Political Economy/Political Economy in Toulouse for their feedback.

Additional information

Funding

We acknowledge financial support from the National Science Foundation (SES-1528487 and Graduate Research Fellowship under grant no. 1122374).