Cost-effectiveness modeling for neuropathic pain treatments: investigating the relative importance of parameters using an open-source model

Abstract

Aims: The study objective was to develop an open-source replicate of a cost-effectiveness model developed by National Institute for Health and Care (NICE), in order to explore uncertainties in health economic modeling of novel pharmacological neuropathic pain treatments.

Materials and methods: The NICE model, consisting of a decision tree with branches for discrete levels of pain relief and adverse event (AE) severities, was replicated using R, and used to compare a hypothetical neuropathic pain drug to pregabalin. Model parameters were sourced from NICE’s clinical guidelines and associated with probability distributions to account for underlying uncertainty. A simulation-based scenario analysis was conducted to assess how uncertainty in efficacy and AEs affected the net monetary benefit (NMB) for the hypothetical treatment at a cost-effectiveness threshold of £20,000 per QALY.

Results: Relative to pregabalin, an increase in efficacy was associated with greater NMB than an improvement in tolerability. A greater NMB was observed when efficacy was marginally higher than that of pregabalin, while maintaining the same level of AEs than when efficacy was equivalent to pregabalin, but with a more substantial reduction in AEs. In the latter scenario, the NMB was only positive at a low cost-effectiveness threshold.

Limitations: The replicate model shares the limitations described in the NICE guidelines. There is a lack of support in scientific literature for the assumption that increased efficacy is associated with a greater reduction in tolerability. The replicate model also included a single comparator, unlike the NICE model.

Conclusions: Pain relief is a stronger driver of NMB than tolerability, at a cost-effectiveness threshold of £20,000 per QALY. Health technology assessment decisions which are influenced by NICE’s model may reward efficacy gains, even if they are associated with more severe AEs. This contrasts with recommendations from clinical guidelines for neuropathic pain, which place more equal weighting on improvements in efficacy and tolerability as value drivers.

Keywords:

• Neuropathic pain
• health economic model
• cost-effectiveness
• health technology assessment (HTA)
• National Institute for Health and Care (NICE)
• pregabalin
• R

JEL classification codes::

• I18
• I10

Introduction

Background

The International Association for the Study of Pain (IASP) defines neuropathic pain as “pain caused by a lesion or disease of the somatosensory system”¹. There is cross-country variation in the epidemiology of chronic pain, and currently no accurate estimate of the population prevalence of neuropathic pain². Previous general population study results suggest that between 7–8% of adults have chronic pain that is neuropathic in its characteristics³.

Several therapies are available for neuropathic pain, but pregabalin is the only treatment licensed in the US and Europe for the treatment of both central and peripheral neuropathic pain. It is often used as a first-line treatment in clinical practice, due to its favorable pharmacokinetic profile and potency⁴. The common use of pregabalin is reflected in the prescribing rates, which increased by 17.2% in the UK alone between 2013 and 2016⁵.

Current treatment options do not adequately manage the symptoms associated with neuropathic pain for all patients, and are associated with a variety of adverse effects (AEs)⁶. Moreover, it is difficult to alter a patient’s perception of pain without precipitating other events related to the central nervous system. Moderate-to-severe neuropathic pain has a substantial impact on a patient’s quality-of-life, and has an influence on daily functioning, even though the majority of patients (an estimated 93%) receive prescribed medications to alleviate their symptoms⁷. As such, there remains a need for new treatments that have a beneficial effect on important patient-centered outcomes by providing effective pain relief, a shorter duration until treatment response, and a reduction in the frequent and sometimes intolerable AEs associated with the current portfolio of therapies^7–10.

It is important for developers of neuropathic pain technologies to have an early understanding of the evidence required for Health Technology Assessment (HTA) bodies to demonstrate the value for new pain therapies. For example, in England and Wales, the National Institute for Health and Care Excellence (NICE) assesses and appraises the clinical and economic value of new technologies, and provides opportunities for companies to seek early advice on plans for evidence generation to support the value of future novel therapies¹¹. Early modeling is essential to understand the value of new therapies in development for neuropathic pain.

There are, however, challenges in the health economic modeling of a heterogeneous disease area such as neuropathic pain^10,12. According to Critchlow et al.¹², the only published HTA model for cost-effectiveness analysis of pharmacological treatment of neuropathic pain which is reported in a replicable and reliable format is a model developed by NICE to inform its clinical guideline recommendations⁶. The NICE model allowed for “comparison of the maximum number of drugs for which data are available in a transparent way”, taking into consideration “the potentially serious limitations found in previous economic models of pharmacological treatments for neuropathic pain”. Critchlow et al.¹² also suggest that, in order to encourage further transparency beyond the NICE model, future developers should report data inputs used to inform cost-effectiveness models, provide a clear description of the analysis of uncertainty, and (if possible) produce an open-source model. The need for transparency in the health economic modeling of medicines has been discussed with increasing interest over the last decade, with the aim of promoting “trustworthy, reproducible, validated, comparable and flexible health economic models which could lead to better decisions in healthcare”^13–15.

Objective

The objective of this study was to develop an open-source replicate of the NICE cost-effectiveness model which can provide a framework for exploration of uncertainties in economic modeling of novel pharmacological neuropathic pain treatments. This involved assessing the relationship between clinical efficacy and AEs, in order to determine how key outcome parameters translate to additional benefit. The online availability of the R scripts used to run the model will allow users to assess the economic performance of neuropathic pain treatments across a range of scenarios, potentially facilitating future economic research in the field^16,17.

Methods

Model description

The cost-effectiveness model developed as part of the NICE CG.173 clinical guideline was replicated using R, an open-source and highly extensible language for statistical computing, which is increasingly being used for health economic analysis¹⁸.

The model compared a hypothetical neuropathic pain drug to pregabalin and consisted of a decision tree with branches corresponding to discrete levels of pain relief: less than 30%, 30–49%, or of 50% or more (Figure 1). If a patient experienced less than 30% pain relief, the model assumed that they did not experience any pain relief at all. This approach to categorizing pain relief is recommended by the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) group and commonly used in scientific literature⁹. Each pain relief branch had three sub-branches representing the severity of AEs experienced by the patient: none, minor AEs, and AEs leading to withdrawal.

Figure 1. Diagram of cost-effectiveness model structure⁶. Abbreviation. AEs, adverse effects.

The model utilized a simulation-based approach. Each parameter was associated with a suitable probability distribution, in order to describe the underlying uncertainty in its realized value. A total of 10,000 random draws from these distributions was used to simulate “potential futures”, which were subsequently used to obtain relevant summaries (a procedure often termed “Monte Carlo estimation”). The final output of the model was obtained by summarizing the results of simulations in terms of determining the optimal decision (i.e. the parameter associated with the highest net benefit), as well as by assessing the underlying uncertainty.

Model inputs

Model parameters and their distributions were sourced from the base case of the NICE guideline (Table 1)⁶. The model included dizziness, vertigo, and nausea as AEs, in line with the NICE model (these are commonly experienced by patients who are receiving neuropathic pain therapies and are generally minor in severity, but can lead to discontinuation in some cases).

Table 1. Model parameters.

Parameter	Mean (SE)	Distribution	Source
Pregabalin probabilities (with no dose adjustment
<30% reduction in pain	0.43 (0.079)	Dirichlet	NICE^6
30–49% reduction in pain	0.16 (0.0077)
≥50% reduction in pain	0.41 (0.082)
Withdrawal due to AEs	0.19 (0.033)	Beta
Dizziness	0.36 (0.069)
Nausea	0.12 (0.046)
Utilities
<30% reduction in pain	0.16 (0.036)	Beta	NICE^6, McDermott et al.^7, IASP^22
30–49% reduction in pain	0.46 (0.015)	Beta
≥50% reduction in pain	0.67 (0.015)	Beta
Withdrawal due to AEs (multiplier)	0.86 (0.037)	Uniform (0.8–0.93)	NICE^6, Wilby et al.^23
Dizziness (absolute utility decrement)	0.12 (0.0024)	Beta	NICE^6, Revicki and Wood^24
Nausea (absolute utility decrement)	0.065 (0.0013)	Beta	NICE^6, Sullivan et al.^25
Costs
Pregabalin (20-week cost)	£325.59 (daily rate = £2.33)	–	NICE^6, NHS^20
Cyclizine hydrochloride (50 mg t.i.d)^a	£0.4407	–
GP visit (lasting 17.2 min)	£63.70	–	NICE^6, PSSRU^19
Resource use
Treatment of minor AEs (dizziness/nausea) with antiemetic  medication (cyclizine hydrochloride) - duration (days)	10.5 (2.02)	Uniform (7–14)	NICE^6
No. GP visits for minor AEs (dizziness/nausea)	3 (0.578)	Uniform (1–2)
No. GP visits for withdrawal AEs (dizziness/nausea)	3 (0.41)	Uniform (2–4)

Standard error values were derived from the 95% intervals provided in the report.

^aCyclizine hydrochloride is an add-on treatment to manage the side-effects that are commonly associated with pregabalin treatment (e.g. nausea and vomiting).

The beta distribution applies to a variable that must take values in the range 0–1 (e.g. probabilities), parameterized by two positive shape parameters.

The Dirichlet distribution is the multivariate generalization of the beta distribution that applies to a vector of quantities constrained between 0–1.

Abbreviations. AEs, adverse effects; GP, general practitioner; t.i.d, ter in die.

Efficacy and AE probabilities were obtained from a network meta-analysis conducted for the NICE guideline. As patient level data were not available, the efficacy parameters (which consist of three mutually exclusive values) were modeled using a suitable Dirichlet distribution, a multivariate generalization of the beta distribution. The probability of an AE occurring was modeled independently of that of other AEs and of efficacy (i.e. for each AE, a suitable beta distribution was utilized to confine the parameter estimate to a value between 0 and 1). Efficacy and AE parameters for the hypothetical treatment were determined by applying a scale factor to the corresponding pregabalin parameters (the rescaled value for <30% pain reduction was obtained by subtracting values for 30–49% and ≥50% pain reduction from 1).

A time horizon of 20 weeks was adopted, in line with the NICE model, and consistent with the availability of efficacy data for some treatments⁶. A disutility was assumed for withdrawal due to AEs rather than applying disutilities for individual AEs. Patients who withdrew due to AEs did so after 4 weeks, and costs and utility benefits were accrued until that point. The duration of minor AEs, the length of antiemetic prescriptions for nausea, and the number of GP visits for each AE were estimated using uniform distributions. No discounting was included, due to the short time horizon of the model.

The NICE model analysis was undertaken from the perspective of the National Health Service (NHS) and Personal Social Services (PSS) in England and Wales, in accordance with NICE guidelines methodology. Drug costs were originally obtained by NICE from the NHS Electronic Drug Tariff for March 2013, and the cost of a general practitioner (GP) visit was based on Personal Social Services Research Unit (PSSRU) 2012 figures^19,20. For the purpose of this analysis, costs were inflated to 2014 prices using the Hospital & Community Health Services (HCHS) index²¹. The acquisition cost of the hypothetical treatment was set to be equal to that of pregabalin to facilitate exploration of the net monetary benefit (NMB) resulting from improvements in efficacy and/or tolerability.

Model outcomes

The model provides results for several outcomes: total costs, total quality-adjusted life years (QALYs), incremental costs, and incremental QALYs.

A scenario analysis was carried out to evaluate the NMB of the hypothetical treatment in relation to pregabalin. Scale factors representing the occurrence of AEs (range = 0.6–1.4) and the level of efficacy (range = 0.9–1.3) of the hypothetical treatment relative to pregabalin were adjusted across five scenarios (these ranges were selected in order to produce a variety of plausible scenarios). The model assumed that the hypothetical treatment would improve on pregabalin in at least one of these two areas, and that this would result in a NMB. The model also assumed that increased efficacy in the form of pain relief is associated with more severe central nervous system AEs and vice versa. This assumption was made to illustrate the comparative value that the model assigns to efficacy vs tolerability by providing a set of scenarios in which the two parameters are traded off against each other. The remaining parameters were sampled from their assumed distributions.

NMB is calculated by first assuming a cost-effectiveness threshold, then converting health benefits (QALYs) into a currency metric (in this case, Great British Pounds (GBP; £)). The cost associated with each treatment strategy is then subtracted, resulting in the incremental net benefit of each strategy. This facilitates a monetary assessment of the comparative value of the two products, and should reflect a decision maker’s valuation of additional benefit¹¹. The model used a cost-effectiveness threshold of £20,000 per QALY, the lower bound of the acceptable incremental cost-effectiveness ratio (ICER) range specified by NICE Methods Guidance for products that are not indicated for end of life treatment or ultra-orphan indications¹¹. In general, NICE considers an intervention that doesn’t dominate other relevant strategies to be cost-effective if it costs less than £20,000 per QALY gained than the next best strategy²⁶.

Results

In Scenario 1, the hypothetical treatment was associated with a 10% reduction in efficacy (pain relief), and a 40% reduction in the severity of AEs relative to pregabalin (Table 2). This scenario was associated with a small negative NMB (–£26.75). In Scenario 2, the hypothetical treatment had the same efficacy as pregabalin, but was associated with a 20% reduction in AE severity. This scenario resulted in a £66.63 positive NMB. The NMB in Scenarios 3–5 increased as the efficacy and severity of AEs conferred by the hypothetical treatment relative to pregabalin increased in 10 and 20 percentage point increments, respectively. This resulted in a positive NMB ranging from £160.00–£306.75 (the highest NMB was associated with a 30% increase in efficacy and a 40% increase in the severity of AEs relative to pregabalin).

Table 2. Model scenario analyses results.

Scenarios	Scale Factor		Pregabalin		Hypothetical treatment		NMB (£)
	Efficacy (Pain Relief)	Adverse Effects	Costs	QALYs	Costs	QALYs
1	0.9	0.6	358.72	0.143	345.47	0.141	–26.75
2	1.0	0.8	358.72	0.143	352.09	0.146	66.63
3	1.1	1.0	358.72	0.143	358.72	0.151	160.00
4	1.2	1.2	358.72	0.143	365.35	0.155	233.37
5	1.3	1.4	358.72	0.143	371.97	0.159	306.75

Abbreviations. NMB, Net Monetary Benefit; QALYs, Quality Adjusted Life Years.

In Scenarios 2–5, where the efficacy parameter of the hypothetical treatment was equivalent to or greater than pregabalin, the hypothetical treatment was generally preferred, as it resulted in a greater NMB (Figure 2). In Scenarios 2 and 3, this result held for all values of the cost-effectiveness threshold, while, for Scenarios 4 and 5, pregabalin was only preferred for very low values of the cost-effectiveness threshold. In Scenario 1, where it was assumed that the hypothetical treatment was associated with a marginally lower level of efficacy but a substantially lower severity of AEs, the hypothetical treatment was only cost-effective at a low threshold (up to £6,300). In addition, Scenario 1 became inferior to the other scenarios (the point at which all lines cross) above a cost-effectiveness threshold of ∼ £1,300.

Figure 2. Net monetary benefit of the hypothetical treatment relative to pregabalin. Abbreviation. GBP, Great British Pounds.

Discussion

The results of the replicated model suggest that improved pain relief is a stronger driver of NMB than improved tolerability at a cost-effectiveness threshold of £20,000 per QALY. At this threshold, HTA decisions for pharmacological neuropathic pain treatments that are influenced by NICE’s cost-effectiveness model may reward efficacy gains, even if they are associated with more severe AEs. This finding contrasts with recommendations from clinical guidelines for neuropathic pain, which tend to place more equal weighting on improvements in efficacy and tolerability as value drivers. For example, the NICE guideline cites “major AEs” as critical outcomes and “patient-reported pain relief/intensity reduction” as important outcomes⁶. Furthermore, the IMMPACT guideline states that the incidence and severity of AEs must be considered in an evaluation of the clinical meaningfulness of an improvement in efficacy (i.e. pain relief) of a treatment for neuropathic pain²⁷.

The NICE model contains several limitations, most notably, the adoption of a 20-week time horizon and the base-case assumption that no treatment would occur following withdrawal. The former assumption is attributed to lack of evidence on effectiveness and AEs beyond 20 weeks, and may be important for drugs that have significant long-term AEs⁶. To explore the robustness of the latter assumption, which would be to the detriment of drugs with high withdrawal rates, NICE undertook a scenario analysis in which patients were given amitriptyline (the cheapest treatment considered) after withdrawal. This scenario did not produce qualitatively different findings, thereby suggesting that the base-case assumption is robust⁶. Other limitations include the fact that the model dealt with efficacy as a discrete variable (this was necessary because the available utility data were also discrete), the fact that efficacy data on AEs were limited to withdrawal due to AEs, nausea, and dizziness (these were the only data that were available for all of the drugs considered in the original model), and the fact that multiple-line treatment strategies were not modeled (this could not be undertaken due to lack of evidence about how neuropathic pain treatments work in sequence)⁶.

The study design also has limitations. For example, there is a lack of support in scientific literature for the assumption that increased efficacy is associated with a greater reduction in tolerability. It is, however, worth noting that chronic pain treatments are often poorly tolerated by patients; for this reason, a treatment with a novel mechanism of action could potentially be considered as more clinically meaningful than others with the same mechanism of action as existing treatments²⁷. The fact that the NICE model was used to compare a hypothetical treatment against a single comparator, pregabalin, is another potential limitation. The authors felt that these design elements were acceptable, however, for the purpose of exploring uncertainties in economic modeling of novel pharmacological neuropathic pain treatments. Furthermore, while the model can reasonably incorporate any drug included in the NICE guideline as a comparator, pregabalin was selected as it is the most widely prescribed pharmacological treatment for neuropathic pain in the UK⁵.

To validate the replicated model with the NICE model, results for pregabalin-associated costs and health outcomes were compared against those from the NICE guideline. The cost of pregabalin in the replicated model and the NICE guideline was £358.72 and £363.31, respectively, and the total QALYs related to pregabalin were 0.143 and 0.142, respectively. Based on the similarity of these figures, replication of the model appears to have been successful.

Conclusions

This study has provided an open-source reproducible model based on the model developed for the NICE CG.173 clinical guideline. The developers believe that the open-source model will lead to more certainty in future comparisons of pharmacological treatments for neuropathic pain as other researchers build on the methodologies of this model. This will hopefully aid HTA bodies in making the appropriate resource allocation decisions, and lead to the availability of more treatment options for patients who live with neuropathic pain.

This study also explored uncertainties in economic modeling of pharmacological neuropathic pain treatments. Specifically, it explored the tradeoff between efficacy in the form of pain relief and severity of AEs that would likely result in achieving an NMB within the cost-effectiveness threshold for a hypothetical neuropathic pain treatment. The balance is decidedly in favor of efficacy. Further research should consider whether this balance is truly representative of clinical practice and patient experience. This could include reviewing outcomes for patients suffering from AEs or research into the patient pathway and outcomes for patients forced to withdraw from therapy due to AEs. Alternatively, it could involve extending the time horizon of the model using data from longer-term studies. This approach could be useful for determining whether efficacy improvements are compromised by reduced tolerability of AEs over prolonged periods. The next step following this analysis could be to explore which specific clinical utility and AE parameters drive uncertainty in the decision-making process based on this model.

Transparency

Declaration of funding

This study was sponsored by Mundipharma International Ltd.

Declaration of financial/other relationships

MH and WCND are employees of Mundipharma International Ltd. GB is partially supported as the recipient of a research grant sponsored by ICON plc at University College London. MB is an employee of ICON plc and was reimbursed by Mundipharma International Ltd for consulting services relating to development of the manuscript. AU was an employee of ICON when the manuscript was undergoing development. JME peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

The authors would like to thank Julien Gagnon (ICON plc) for his contribution to the model, as well as Therese Ng (ICON plc), Jack Ziomek (ICON plc), and Tom Hutchings (ICON plc) for their parts in developing this manuscript.

Data availability statement

The data that support the findings of this study are openly available in Zenodo at doi.org/10.5281/zenodo.1212328.