Residual Weighted Learning for Estimating Individualized Treatment Rules

ABSTRACT

Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introducing machine learning techniques to optimal treatment regimes, it still has some problems in performance. (1) The estimated ITR of OWL is affected by a simple shift of the outcome. (2) The rule from OWL tries to keep treatment assignments that subjects actually received. (3) There is no variable selection mechanism with OWL. All of them weaken the finite sample performance of OWL. In this article, we propose a general framework, called residual weighted learning (RWL), to alleviate these problems, and hence to improve finite sample performance. Unlike OWL which weights misclassification errors by clinical outcomes, RWL weights these errors by residuals of the outcome from a regression fit on clinical covariates excluding treatment assignment. We use the smoothed ramp loss function in RWL and provide a difference of convex (d.c.) algorithm to solve the corresponding nonconvex optimization problem. By estimating residuals with linear models or generalized linear models, RWL can effectively deal with different types of outcomes, such as continuous, binary, and count outcomes. We also propose variable selection methods for linear and nonlinear rules, respectively, to further improve the performance. We show that the resulting estimator of the treatment rule is consistent. We further obtain a rate of convergence for the difference between the expected outcome using the estimated ITR and that of the optimal treatment rule. The performance of the proposed RWL methods is illustrated in simulation studies and in an analysis of cystic fibrosis clinical trial data. Supplementary materials for this article are available online.

KEYWORDS:

• Convergence rate
• Martingale residuals
• Optimal treatment regime
• Residuals
• RKHS
• Universal consistency

Supplementary Materials

The online supplement contains additional simulation results, and the proofs for the lemmas and theorems discussed in the article.

Acknowledgments

The authors thank Editor Nicholas Jewell, an associate editor, and two reviewers for their helpful comments which led to a significantly improved article. Xin Zhou is in the Department of Biostatistics, University of North Carolina at Chapel Hill. Nicole Mayer-Hamblett is at the Seattle Children's Hospital and in the Department of Pediatrics, University of Washington. Umer Khan is at the Seattle Children's Hospital.

Funding

The EPIC trial was funded by the National Heart Lung and Blood Institute (NHLBI) and the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK, U01-HL080310) and the Cystic Fibrosis Foundation Therapeutics. The first and fourth authors were funded in part by the CTSA at the University of North Carolina at Chapel Hill (UL1TR001111) as well as by grant P01CA142538 from the National Cancer Institute (NCI). The second and third authors were funded in part by the CTSA at the University of Washington (UL1TR000423).