Treatment switching in oncology trials and the acceptability of adjustment methods

Abstract

Treatment switching has become an important issue in the development and approval of new drugs, particularly in oncology. Randomized controlled trials (RCTs) represent the gold standard for evaluating the effectiveness of interventions, but often patients randomized to the control group are permitted to switch onto the experimental treatment at some point during the trial. This is important, because standard statistical approaches used to analyze RCTs compare groups as randomized, based upon an intention-to-treat principle. When patients in both groups receive the new drug, such analyses do not provide an accurate estimate of the comparative effectiveness of the two treatments. This may lead to inappropriate decision-making – cost-effective drugs may not be approved. Limited healthcare finances may be used inefficiently. Health-related quality-of-life and lives may be lost.

Keywords:

• economic evaluation
• oncology
• survival analysis
• treatment crossover
• treatment switching

In an oncology setting, treatment switching often occurs after disease progression, particularly when progression-free survival (PFS) is the primary endpoint. Consequently, PFS is not affected, but intention-to-treat estimates of the overall survival treatment effect – critical for health technology assessment (HTA) decision-making – will be confounded.

Statistical methods are available for adjusting for treatment switching, and have been the focus of recent methodological and applied studies [1–10]. Given the confounding caused by switching, it may seem obvious that adjustment methods should be used to inform HTA. However, frequently adjustment analyses have been rejected by decision makers. This editorial discusses why this is the case and offers thoughts on how this can be addressed.

Why does treatment switching occur?

Treatment switching is driven by ethical concerns [1,3]. The Declaration of Helsinki dictates that generation of knowledge cannot take precedence over the interests of individual medical research subjects [11]. If interim RCT analyses suggest that clinical equipoise no longer exists, it may not be defensible to disallow treatment switching for patients randomized to the inferior treatment – particularly if no other non-palliative treatments are available. Switching is also often incorporated in study designs, usually after primary endpoints (such as PFS) have been observed, to boost trial recruitment [12].

How important is treatment switching?

Over half of the technology appraisals of cancer medicines completed by the National Institute for Health and Care Excellence (NICE) and the Australian Pharmaceutical Benefits Advisory Committee have been affected by treatment switching [1], [Mitchell A, Pers. Comm.].

Switching can have a large impact on cost–effectiveness results. For instance, in NICE TA269 (vemurafenib for melanoma), adjusting for switching in 34% of control group patients, reduced the incremental cost–effectiveness ratio from £75,500 per quality-adjusted life-year gained to £51,800 per quality-adjusted life-year gained [13]. At an incremental cost–effectiveness ratio of £75,500 per quality-adjusted life-year gained vemurafenib would almost certainly not have been deemed cost-effective [14]. However, the adjusted analysis was accepted and vemurafenib was recommended.

Adjustment methods

It is commonly accepted that simple adjustment methods (excluding switchers from the analysis, or censoring them at the time of switch) are prone to severe selection bias, because switching is usually related to prognosis [1]. Although these have been commonly used in HTA historically [1], current recommendations are that they should be avoided [1,15]. Attention has turned to more complex methods, primarily the Rank Preserving Structural Failure Time Model (RPSFTM) [16], inverse probability of censoring weights (IPCW) [17] and two-stage adjustment [2]. These may produce unbiased adjustments providing their assumptions hold. To consider the acceptability of these methods, it is important to reflect upon their key assumptions.

The RPSFTM assumes that if no patient in an RCT receives treatment, survival times will be equal, on average, in the randomized groups. In a well-designed RCT, this seems reasonable. The RPSFTM also assumes that the relative treatment effect is equal for all patients no matter when treatment is received (the ‘common treatment effect’ assumption, which is impossible to perfectly test). This is often regarded as a problematic assumption, given that switchers receive treatment later, often after disease progression. Attempts have been made to relax this assumption by developing multi-parameter RPSFTMs, but with little success [18].

The IPCW extends the simple censoring approach. Switchers are censored, but remaining observations are weighted using baseline and time-dependent covariates to remove selection bias. This requires that the ‘no unmeasured confounders’ assumption holds – data must be available on all prognostic factors for mortality that independently predict the probability of switching – otherwise censoring-related selection bias will remain. The plausibility of this assumption depends upon the data collected during the trial. Further insight may be gathered from clinical experts, and by examining data collected in similar trials. However, we can never be sure that all confounders are measured. Nevertheless, as long as the most important confounders are measured, the IPCW may be prone to minimal bias. A frequent problem for IPCW applications to RCT data is that data are collected until disease progression but not beyond. To be unbiased, data collection is required up to the point of switch in switchers, and for the entire trial period for non-switchers.

The two-stage adjustment method was designed according to the switching commonly observed in oncology RCTs – where switching is permitted after a disease-related time-point (e.g., disease progression) [2]. Treatment effects are estimated separately for switchers and for patients originally randomized to the experimental group. The method is only applicable if switching occurs after a specific disease-related time-point, and requires the ‘no unmeasured confounders’ assumption at this time-point. Compared to the RPSFTM, the two-stage method has the advantage of not assuming a common treatment effect. Compared to the IPCW method, it has the advantage of only requiring prognostic data availability at the specified disease-related time-point, not beyond. However, to be unbiased, it requires that switching occurs immediately at the disease-related time-point. This is unlikely to be exactly true, but if switching occurs soon after this time-point, resulting bias may be small.

Evidence suggests that each of these adjustment methods can produce low bias across a range of scenarios [1,2,6], producing better estimates of the true comparative treatment effect than an intention-to-treat analysis. However, the RPSFTM is sensitive to violations of the ‘common treatment effect’ assumption. The IPCW, and to a slightly lesser extent, two-stage methods, are particularly prone to convergence issues and error when the switching proportion is very high, and when sample sizes and event numbers are small. No single method is optimal for all scenarios.

Are adjustment analyses being accepted?

At first sight, it might appear that adjustment analyses are being used inconsistently by healthcare decision makers. Consider the recent NICE technology appraisals affected by treatment switching. In TA263 (bevacizumab for breast cancer), an RPSFTM analysis was not considered appropriate, whereas in TA269 (vemurafenib for melanoma), TA326 (imatinib for gastrointestinal stromal tumors) and TA296 (crizotinib for non-small-cell lung cancer) RPSFTM and IPCW adjustments were accepted, and informed decision-making [13,19–21].

A closer investigation of these appraisals reveals that consideration of adjustment methods may indeed have been consistent. In TA263, the RPSFTM was the only complex adjustment analysis presented, and was considered inappropriate for the case being investigated, largely because it adjusted only for ‘direct’ treatment switching (from the control group to the experimental treatment), while, in fact, ‘indirect’ switches (to other post-study treatments) also occurred [19]. In contrast, in TA269, TA326 and TA296 a comprehensive range of adjustment methods were discussed in detail. Acceptance of the adjustment analyses was associated with consideration of the clinical plausibility of their results, supported by analyses of external data [13,20,21].

This indicates that NICE is willing to accept adjustment analyses, provided methods used are appropriately justified, a range of potentially appropriate methods are considered, and, particularly, if results are shown to be clinically plausible. A similar position is taken by the Australian Pharmaceutical Benefits Advisory Committee and the Scottish Medicines Consortium, where recent examples show a consideration of RPSFTM and IPCW analyses, their assumptions, external evidence and clinical plausibility [22–25].

However, these agencies may be further advanced in their use of adjustment methods than others around the world. For instance, RPSFTM analyses have been submitted to the Pan-Canadian Oncology Drug Review, but have not been considered in subsequent final review documents [26–29]. Further, in Germany, the Institute for Quality and Efficiency in Healthcare rejected an RPSFTM analysis, stating that the method is ‘based on strong assumptions, the fulfillment of which cannot be checked with the available data’ (pg.22, [30]). All complex adjustment methods make untestable assumptions, and thus none may be acceptable to the Institute for Quality and Efficiency in Healthcare. However, in 2014, the Institute for Quality and Efficiency in Healthcare held an ‘In Dialogue’ session specifically discussing adjustment methods [31], and therefore their current position may alter.

The way forward

There are examples of adjustment analyses being accepted and rejected by HTA agencies around the world. This is not necessarily a sign of inconsistency, because adjustment analyses may not always be acceptable. Agencies such as NICE, the Australian Pharmaceutical Benefits Advisory Committee and the Scottish Medicines Consortium seem likely to reject adjustment analyses when they have not been well explained or justified, or when they provide implausible results. It is a concern that some HTA agencies do not appear to have come to terms with the use of adjustment methods. However, this may change with better submissions to these agencies. The emphasis is on manufacturers to ensure that analyses submitted to decision makers are comprehensive and robust, justifying and explaining the methods used.

Uncertainties about the adjustment methods themselves remain, and the confidence that decision makers have in them may be enhanced by further research. It is not possible to perfectly test the ‘common treatment effect’ assumption, but methods could be developed to investigate its approximate plausibility, or to quantify the bias associated with violations of it. Also, IPCW and two-stage analyses are reliant on comprehensive data collection over time, and trial design could be altered to increase the likelihood that adjustment methods can be successfully – and acceptably – applied. Furthermore, although treatment switching may be required in many circumstances to treat study participants ethically, early switching should be avoided when clinical equipoise remains, in order to protect the scientific validity of the trial.

Recent work has focused upon adjusting for ‘direct’ treatment switching. However, ‘indirect’ switching is important. If we adjust for ‘direct’ switching, we should also make adjustments if experimental group patients switch onto other investigational agents. Otherwise, decision makers are likely to disregard adjustment analyses, believing them to be biased in favor of the experimental treatment. In addition, research has focused on making adjustments to survival-time estimates. However other inputs to the economic model may also be affected by switching (e.g. costs, health-related quality-of-life). Research into adjustment methods that may be applied to these outcomes would be valuable – otherwise economic analyses may remain confounded.

Better description and justification of adjustment analyses, together with further research into the methods themselves, will allow adjustment methods to be used more confidently in healthcare decision making. Statistical adjustment methods are not perfect, but in the presence of confounded RCT data, they can be used to provide useful information to enable better decision making for the good of patients and society.

Financial & competing interests disclosure

N Latimer has led consultancy projects conducted within the University of Sheffield funded by GSK, Eisai and the Pharmaceutical Oncology Initiative. In the past 12 months, N Latimer has undertaken private consultancy for Astellas, Sanofi Aventis, Boehringer Ingelheim and Janssen. The author has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.