Using Cost-Effectiveness Evidence to Inform Decisions as to which Health Services to Provide

Abstract

Abstract—This article focuses on three challenges concerning the use of cost-effectiveness thresholds to inform decision making regarding which services a third-party payer will fund. First, how is the appropriate cost-effectiveness threshold or threshold range to be determined or, indeed, should there be a single threshold or multiple thresholds? Second, how can the valuation of health benefits be refined to better capture the value of treatments to patients and to the economy as a whole? Third, how is the tension between cost-effectiveness and the affordability and sustainability of health services to be managed?It concludes that whatever other factors are considered in addition to cost-effectiveness, and whether the decision-making process is more or less deliberative, cost-effectiveness thresholds are important. Though there is a range of sources for identifying appropriate thresholds, using the opportunity cost in terms of the health benefits from displaced activities will minimize the problem of cost-effective interventions not being affordable and will facilitate the efficient use of scarce resources. Finally, although experience using weighted quality-adjusted life years (QALYs) is currently very limited, it is likely to be an important area in the future.

Keywords:

• cost-effectiveness threshold
• decision making
• drug adoption
• health technology assessment
• QALYs

INTRODUCTION

Notions of a cost-effectiveness threshold lie at the heart of any attempt to use cost-effectiveness evidence to inform decisions as to which health services to provide. The aim of this article is to review three challenges concerning the use of cost-effectiveness thresholds to inform decision making regarding which services a third-party payer will fund. First, how is the appropriate cost-effectiveness threshold or threshold range to be determined or, indeed, should there be a single threshold or multiple thresholds? Second, how can the valuation of health benefits be refined to better capture the value of treatments to patients and to the economy as a whole? Third, how is the tension between cost-effectiveness and the affordability and sustainability of health services to be managed? These questions are of course linked, for example, weighting quality-adjusted life years (QALYs) and using a threshold cost per weighted QALY and having multiple thresholds for different groups of QALYs (defined, for instance, by severity of the underlying condition) are alternative means of recognizing that not all QALYs have the same value.

These challenges are certainly ones with which high-income countries (HICs) must grapple, but they are also very much of relevance to low- and middle-income countries (LMICs), although the thresholds at which services are deemed cost-effective will be substantially lower, and the more fundamental challenge for LMICs is to develop and sustain Health Technology Assessment (HTA) decision-making processes rather than to identify appropriate cost-effectiveness thresholds.¹ Cost-effectiveness thresholds are important whenever an organization is concerned with obtaining value for money from its health care spending. They may be of particular relevance with respect to public expenditure because they can increase the transparency and accountability of decision making.

COST-EFFECTIVENESS AND MULTIPLE OBJECTIVES

Before turning to the three challenges addressed in this article, it is worth emphasizing that the relevance of cost-effectiveness thresholds is independent of the specific form of decision making. One of the key strengths of the economic approach is its ability to combine several consequences of an intervention in a single, widely applicable measure. Thus, the incremental cost-effectiveness ratio, in principle, captures the cost of the treatment, potential future cost savings, and impacts on the patient's health status and life expectancy and facilitates comparisons of the implications of spending in different therapeutic areas. Because it is highly unlikely that a decision-making process would rely solely on cost-effectiveness information, the issue arises as to how to combine cost-effectiveness data with other relevant inputs to the decision-making process. One important choice is between a deliberative approach where cost-effectiveness is simply considered alongside other factors or possibly is given priority but its importance and interpretation is influenced by the other factors, and some form of multiple criteria decision analysis (MCDA) that attempts to incorporate these other factors formally and, importantly, is explicit regarding the trade-offs between the differences sources of value.

A comparison of the MCDA approach with the National Institute for Health and Care Excellence's (NICE's) appraisal process emphasizes the many common elements but highlights the key difference at the decision-making stage. NICE engages in deliberative decision making using the incremental cost-effectiveness ratio and other criteria.² The decision of the appraisal committee contains a description of the factors that have influenced the decision, but rarely are these quantified and the weight attached to different considerations is unclear. With an MCDA these other criteria would be quantified explicitly and their relative importance would be reflected in a transparent set of weights.

Whichever approach is adopted, the importance of cost-effectiveness thresholds is undiminished. If an MCDA approach were adopted, cost-effectiveness thresholds would still be relevant because the fundamental principle of opportunity cost remains. Thus, when making decisions with respect to the allocation of a given budget, the benefits expected from a new activity should exceed the loss of benefits from displaced activities if a positive recommendation is to be made. However, the valuation of the benefits would now not be solely in terms of foregone QALYs but in terms of the metric used in the MCDA to value the benefits of the new activity, and the cost-effectiveness threshold would need to be stated in terms of this metric.

DETERMINING THE COST-EFFECTIVENESS THRESHOLD

As long as cost-effectiveness is one of the criteria used to assist decision making, it will be necessary to specify a threshold value (or a range of values) in order to inform assessments of whether a particular intervention generates benefits at an acceptable cost. Several potential sources of values for the cost-effectiveness threshold have been identified: a value implied by past decisions, an estimated societal willingness to pay for additional health benefit, a value related to gross domestic product (GDP) per capita, and the cost per unit of benefit of the services that would be displaced.

Assuming some degree of consistency with respect to previous decisions, it is clearly possible to infer a threshold from previous decisions. This has been done on a number of occasions. Recently, Dakin et al.³ used logistic regression to model NICE decisions to recommend or not recommend particular health technologies. Note that this approach assumes that which incremental cost-effectiveness ratio (ICER) the committee believed when it said “yes” or “no” is known and that all factors relevant to the decision were incorporated in that ICER (or are controlled for in the independent variables). But there is a more fundamental problem: do we want future decisions to be determined by past decisions? To what extent do past decisions reflect assessments of the value of the health benefits displaced (either then or now)? Clearly, it may be appropriate for the threshold to change over time.

Rather than studying the past decisions of a decision maker, an alternative approach is to ask people directly to value additional health benefits. Ryen and Svensson⁴ reviewed 24 studies estimating willingness-to-pay (WTP) for a QALY, containing 383 unique estimates of the WTP for a QALY. The trimmed mean and median estimates were found to amount to 74,159 and 24,226 euros (2010 price level), respectively. They noted the heterogeneity of studies, involving a wide range of countries, individual or societal perspectives, general or specific populations, quality of life or life expectancy, general health or specific condition. They found that WTP for a QALY is significantly higher if the QALY gain comes from life extension rather than quality of life improvements and that the WTP for a QALY is dependent on the size of the QALY gain.

A related stated preference approach is to use information on the value of a statistical life. For example, the Department of Health (DH) in England suggested the use of ≤60,000 per QALY based on making a series of adjustments to the value of preventing a road traffic fatality. The value of preventing a road traffic fatality is estimated to be ≤1,637,420 (based on stated preference estimate of WTP to reduce risk of death). By making assumptions about the average age of male and female fatalities, their predicted remaining life expectancy, assuming a 1.5% discount rate and adjusting for health status and the ratio of male to female fatalities, it is estimated that on average 26.7 QALYs are lost per fatality. Dividing the value of preventing a fatality by the estimated QALY loss gives a value of £61,327.

An alternative approach is to specify the cost-effectiveness threshold as a multiple of per capita income. The World Health Organization's (WHO's) Commission on Macroeconomics and Health 2001 argued that the value of preventing a disability-adjusted life year (DALY) should be at least equal to the per capita income but the true value might be up to three times this due to other factors (such as pain and suffering). Consequently, the WHO identifies three categories of cost-effectiveness on their website: highly cost-effective (less than the GDP per capita); cost-effective (between one and three times GDP per capita); and not cost-effective (more than three times GDP per capita).⁵

Willingness-to-pay valuations (and values related to income) are potentially relevant if what is sought is an estimate of the value placed on additional health benefits whose purchase reduces overall consumption but less relevant if the issue is one of how to spend a given budget. In the latter case, the concern is to ensure that the value of the health benefits displaced does not exceed the value of the health benefits from the new activity.

To quote the NICE methods guide for technology appraisal, “A technology can be considered to be cost effective if its health benefits are greater than the opportunity costs of programmes displaced to fund the new technology.”⁶ This is a useful insight at the conceptual level, but for it to help identify an appropriate cost-effectiveness threshold we need to know what purchasers of health care are giving up when they implement NICE guidance. In the absence of such data we can ask how health outcomes vary in response to changes in health care spending. There are clearly many challenges in making such an estimate: there are likely to be issues of data quality; health care expenditure is likely to be endogenous; health outcomes will be influenced by many factors (in addition to health care spending); and there will be time lags between changes in spending and changes in health outcome.

Claxton et al.⁷ have reported a new central estimate of £12,936 per QALY, which they suggest is “if anything, likely to be an overestimate.” Their probabilistic model indicates that there is an 89% chance the figure is less than £20,000 per QALY and a 97% chance that it is less than £30,000. They go on to argue that

the consequences for the NHS of overestimating the threshold are more serious than underestimating it. In principle, a policy threshold … should be set below its mean value to take account of the non-linear relationship between the threshold and the additional net health benefit offered by a technology.

These estimates have been challenged, in particular by the Office of Health Economics, who claim that they are highly uncertain and sensitive to the use of plausible alternative assumptions.⁸ They particularly take issue with assumptions that patients whose lives are saved will live as long as healthy people of the same age and will enjoy better quality of life than the average patient with the same disease. They argue that the overall effect is to understate the true value of the threshold.

Quite clearly, few countries are currently in a position to undertake similar analyses and thus alternative ways of identifying appropriate thresholds would be of considerable interest. Recently, Woods et al.⁹ have taken the estimated relationship for England between the consumption value of health and the health foregone when National Health Service (NHS) expenditure is displaced and applied this in other countries in order to identify appropriate cost-effectiveness thresholds for these other countries. This approach does require a number of strong assumptions; for example, that the relationship between the consumption value of health and the cost-effectiveness threshold for health (the benefits foregone when health care expenditure is displaced) is common across countries. The authors wisely counsel caution when interpreting their results. The value of this bold paper may well lie as much in the stimulus it provides for further research.

The issue of whether to have a single cost-effectiveness threshold or multiple thresholds is closely linked to that of valuing health benefits and, in particular, weighting QALYs. To have a single threshold, either all QALYs must be valued equally or the QALYs must first be weighted before estimating a cost per weighted QALY gained.

VALUING HEALTH BENEFITS

There is now considerable interest in value-based assessment of new health technologies. Though the emphasis was initially on the relationship between price and assessments of the value of a health technology, over time attention has moved firmly to attempts to identify better ways to assess the relative value of different health technologies. Attention has focused on weighting QALYs to more accurately distinguish the value of the health benefits in different circumstances and on including a broader range of consequences of interventions.

Health economics has a tradition of regarding all QALYs as being of the same significance and value. One example of this is the practice when estimating incremental cost-effectiveness of adding together the QALYs of the entire patient group and ignoring that some patients accrue more QALYs than others and their identity. However, there are several recent examples of cost-effectiveness thresholds differentiated by health loss, such as the supplementary guidance issued to the NICE appraisal committee and the interest shown in various countries in valuing QALYs gained by those in poorer health more highly than QALYs gained by those in better health.

The instruction to the NICE Appraisal Committee in 2009 to treat life-extending, end-of-life treatments differently from other health technologies was a significant departure from the conventional approach. This change was introduced as a means of increasing the proportion of new cancer drugs recommended by NICE.¹⁰ Three criteria must be fulfilled in order for a treatment to qualify as a life-extending, end-of-life treatment: (1) the treatment is indicated for patients with a short life expectancy, normally less than 24 months; (2) there is sufficient evidence to indicate that the treatment offers an extension to life, normally of at least an additional three months, compared with current NHS treatment; and (3) the treatment is licensed or otherwise indicated for small patient populations normally not exceeding a cumulative total of 7,000 for all licensed indications in England.⁶

This started as a form of weighting, but it fairly rapidly metamorphosed into the application of multiple cost-effectiveness thresholds. It represents the simplest form of QALY weighting because only two cases are distinguished: QALYs produced by life-extending, end-of-life treatments, and those generated by all other treatments. To date the criteria have been met on 30 occasions and the committee recommendations have been consistent with using a £50,000 per QALY threshold for these end-of-life treatments (compared to the £20,000 to £30,000 range for nonqualifying treatments). Whether or not counting these health gains as being twice as valuable as those received by other patients reflects societal preferences remains unclear. A recent review of 17 studies concluded that the existing evidence is mixed.¹¹ Leaving aside the merits or otherwise of the policy, the experience with end-of-life treatments demonstrates the feasibility of weighting QALYs (or using multiple thresholds).

There has been increasing interest in valuing health gains differentially depending on the state of health of the patient, and it has been suggested that this might be indicated by the proportional QALY shortfall associated with the condition of the patient. For example, there is agreement in The Netherlands that the cost per QALY gained that is acceptable is greater the higher the proportional shortfall in QALYs. The proportional shortfall is measured by the disease-related QALY loss divided by the remaining QALYs expected in the absence of the disease.¹²

Similarly, in England, NICE proposed that the weight attached to health benefits should be related to the burden of illness. The burden of illness is measured by the proportional QALY shortfall—that is, the shortfall in QALYs considered relative to what people could expect without the condition at the time of treatment. However, following consultation, a decision was taken not to change the technology appraisal methodology in the short run.

The Third Norwegian National Priority Setting Committee¹³ have recommended that the cost-effectiveness threshold should vary according to the health loss experienced by the patient group. This is to be measured by healthy life years lost if given standard treatment compared to a long and healthy life. The underlying justification for the approach is a desire to maximize health and to distribute it fairly. Consequently, it is proposed that a health gain has a higher value the more it benefits the worse-off patients. An illustration is provided with four levels of health loss each with its own cost-effectiveness threshold (<NOK 250,000; <NOK 250,000–500,000; <NOK 500,000–750,000; and <NOK 750,000–1,000,000, where 1 Norwegian Krone or NOK equals approximately $0.11 USD).

A final instance of innovative thinking regarding the definition and measurement of benefits from adopting new health technologies comes from the DH in England. The DH proposal for including wider societal benefits (WSB)¹⁴ is primarily concerned with the effect of treatment on others (the impact on the patient is assumed to be captured through the QALY). The DH defines WSB as the difference between the amount of resources a patient contributes to society (production) and the amount they utilize (consumption). The adoption of any proposed treatment will lead to a change in WSBs (e.g., as one treatment is replaced by another) and if an intervention has a positive incremental cost, other NHS activities will be displaced and these activities will also have associated WSBs. Thus, the proposal concerns capturing benefits not currently reflected in the QALY rather than weighting QALYs per se. However, the NICE proposal (in response) suggested that the wider societal impact could be captured by estimating the absolute QALY shortfall, and this in turn could be used as a reason for weighting the benefits of a treatment more highly.

Though weighting health benefits and using different thresholds for different unweighted health benefits can be viewed as alternatives, there is presumably a limit to how many different thresholds decision makers are comfortable with using. Thus, if it is thought desirable to distinguish many classes of health benefit, it might be easier to calculate weighted QALYs and compare to a single threshold. Similarly, if more fine-grained distinctions between different treatments are sought (for example, recognizing the proportion of the health benefits generated by treatments falling in different categories rather than assigning all of the benefits from a particular treatment to a particular category), then applying a weighting scheme will be more feasible than using multiple thresholds. Thus, with respect to the end-of-life example, rather than all of the health benefits of a treatment being adjudged either to qualify or not, the proportion of patients who would be expected to obtain an extension to life of at least three months could be assessed and taken into account when weighting the QALYs.

COST-EFFECTIVENESS AND AFFORDABILITY

Health technologies can be assessed as cost-effective, but that does not necessarily mean that they are affordable. Affordability and sustainability of health services depend on sufficient funds being made available. The problem arises because there is no automatic link between the size of health care budgets and the cost-effectiveness of health care spending. Health care budgets change over time as a consequence of a wide range of factors and not just changing opportunities to produce health benefit. How is the tension between cost-effectiveness and the affordability and sustainability of health services to be managed?

As noted above, a widely used approach is to specify the cost-effectiveness threshold as a multiple of per capita income.⁵ However, the use of multiples of per capita income ignores opportunity cost and also threatens sustainability. Revill et al.¹⁶ argued that such cost-effectiveness benchmarks lack a theoretical or empirical basis and make many health care interventions notionally cost-effective. Because they take no account of whether resources could be better used elsewhere, the use of WHO thresholds “is likely to reduce overall population health and exacerbate health care inequalities.”

Using a recent UK assessment of health forgone through resources being committed to particular interventions and assuming that the relationship between health care spending and health attainment across countries is subject to diminishing returns, suggests that a suitable benchmark for lower income countries is unlikely to be higher than 0.52 GDP per capita.¹⁶

The relevance of the WHO threshold has recently been examined by Newall et al.,¹⁷ who reviewed the cost-effectiveness of HPV vaccinations in 26 LMICs and of rotavirus vaccinations in 15 LMICs. They found that vaccination programs being found “very cost-effective” (ICER < GDP per capita) did not ensure that they were funded. However, cost-effectiveness may be playing a role in that programs with ICERs more than twice GDP per capita were less likely to be implemented. They concluded that “an intervention having cost per DALY averted less than per capita income was not sufficient for vaccination programme to be funded” and suggested that this results from the difference between affordability and cost-effectiveness.

In England, NICE assesses cost-effectiveness by comparing the cost per QALY gained with a cost-effectiveness threshold range and the potential budget impact is only relevant insofar as “the Committee may require more robust evidence on the effectiveness and cost effectiveness of technologies that are expected to have a large impact on NHS resources.”⁶ It has been suggested that the threshold be set “to optimally exhaust” (maximize the health gain) from the fixed budget.¹⁵ This firmly places the emphasis on a comparison with the cost per QALY of displaced services and has the advantage that it can reduce the tension between cost-effectiveness and affordability by linking the threshold to the predetermined budget.

If the health care budget is relatively fixed in the short run (or at any rate not responsive to changing opportunities to generate health benefits), changing opportunities to produce health will not lead to changes in levels of spending but rather in the mix of activities. A new set of cost-effective treatments is funded by reducing spending on some less cost-effective treatments.

The alternative would be to increase or decrease the resources available for purchasing health services in response to changing opportunities to produce benefit. This would involve making a judgment regarding the value of additional health benefits, which in turn will depend on a number of factors, such as the wealth of the country and the relative importance of improving health vis-à-vis other goods. Cost-effective new treatment opportunities would be funded by increasing spending rather than displacing existing treatments.

In practice, budgets are not entirely exogenously determined, and improved opportunities to buy health over time will encourage expansion of budgets. But to have budgets closely following the changing opportunities to produce benefit leads to difficulties financing health care, and it will be necessary to let changes in the mix of services provided “solve” the problem of changes in opportunities to produce benefit, although this can produce a different set of challenges with respect to the delivery of care.

A recent example of the tension in HICs arises with the directly acting antivirals for treating hepatitis C. Several of these are clearly cost-effective means of treating hepatitis C (at least for a range of patient subgroups and in HICs). But to immediately take these opportunities for the cost-effective treatment of patients requires a marked increase in budgets or substantial reallocation of current spending. In LMICs with very limited budgets for purchasing health care and an arbitrarily defined (overly high) cost-effectiveness threshold there will be a permanent tension between affordability and cost-effectiveness.

THRESHOLDS AND DRUG PRICING

It is recognized that the use of an explicit cost-effectiveness threshold to inform decisions over which health technologies to adopt enables the manufacturer to capture the value of their innovation by pricing to meet the threshold.¹⁸ Drug prices are endogenous and are not generally set independent of the reimbursement decision-making process. Managed entry agreements or patient access schemes that have become common in several countries in recent years provide a mechanism whereby pricing to meet the threshold is facilitated. For example, in England, once an appraisal is underway it becomes clearer how a committee will regard the likely cost-effectiveness of a drug and what size of downward adjustment in price would be required to make a positive adoption decision reasonably likely. Another example of this arises with the commissioning of vaccines in the UK where an initial assessment is made as to whether the vaccine could be cost-effective and the basis for this, and this is known by the manufacturer prior to the tendering process.

The endogeneity of drug prices has led Basu¹⁹ to suggest that cost-effectiveness thresholds are irrelevant. He argues against the use of explicit cost per QALY thresholds on the grounds that it gives manufacturers an incentive to set prices to just “meet” the threshold, thus transferring surplus from the third-party payer to the manufacturer. His “dynamic” alternative using league tables crucially assumes that coverage decisions can be readily reversed. However, to do so might be reputationally costly and problematic in that treatments will be removed from one particular patient group and replaced with a new treatment for another (different) patient group. More fundamentally, the solution to sharing the surplus lies not in discarding explicit cost-effectiveness thresholds but rather in developing policies directed at achieving the desired level of sharing.

Though a higher threshold, other things being equal, will provide a stronger incentive to invest in drug development, cost-effectiveness thresholds are not a suitable means of resolving the broader issues of achieving the desired level and mix of research and development activity. More fundamental measures are likely to be required, such as the divorce of drug production and pricing from the activity of research and development advocated recently by McGuire et al.²⁰

The issue of the sharing of surplus between the health services and drug and medical device manufacturers and its implications for research and development activity, patient access to health technologies (innovative or otherwise), and wider economic concerns such as employment, growth, and trade is beyond the scope of this article. However, it might be observed that, though the cost-effectiveness threshold is a powerful means ensuring health services make the best use of their limited resources, to address these broader concerns requires some additional policy levers.

CONCLUSIONS

Whatever other factors are considered in addition to cost-effectiveness, and whether the decision-making process is more or less deliberative, cost-effectiveness thresholds are important. Though there are a range of sources for identifying appropriate thresholds, using the opportunity cost in terms of the health benefits from displaced activities will minimize the problem of cost-effective interventions not being affordable and will facilitate the efficient use of scarce resources. Experience using weighted QALYs is currently very limited, but given growing interest, this is likely to be an important area in the future.

DISCLOSURE OF POTENTIAL CONFLICTS OF INTEREST

No potential conflicts of interest were disclosed.