More Efficient Policy Learning via Optimal Retargeting

Abstract

Policy learning can be used to extract individualized treatment regimes from observational data in healthcare, civics, e-commerce, and beyond. One big hurdle to policy learning is a commonplace lack of overlap in the data for different actions, which can lead to unwieldy policy evaluation and poorly performing learned policies. We study a solution to this problem based on retargeting, that is, changing the population on which policies are optimized. We first argue that at the population level, retargeting may induce little to no bias. We then characterize the optimal reference policy and retargeting weights in both binary-action and multi-action settings. We do this in terms of the asymptotic efficient estimation variance of the new learning objective. We further consider weights that additionally control for potential bias due to retargeting. Extensive empirical results in a simulation study and a case study of personalized job counseling demonstrate that retargeting is a fairly easy way to significantly improve any policy learning procedure applied to observational data. Supplementary materials for this article are available online.

Keywords:

• Efficient policy learning
• Individualized treatment regimes
• Optimization
• Overlap

This article is referred to by:

Discussion of Kallus (2020) and Mo, Qi, and Liu (2020): New Objectives for Policy Learning

Discussion of Kallus (2020) and Mo, Qi, and Liu (2020): New Objectives for Policy Learning

Supplementary Materials

The online appendix includes additional material on regularizing retargeting weights as well as omitted proofs. Replication code is available at https://github.com/CausalML/RetargetedPolicyLearning.

Acknowledgments

The author thanks the anonymous reviewers and associate editor for the constructive inputs.

Additional information

Funding

This material is based upon work supported by the National Science Foundation under grant no. 1846210.