When does economic model type become a decisive factor in health technology appraisals? Learning from the expanding treatment options for relapsing–remitting multiple sclerosis

Abstract

Objectives: Specific economic model types often become de facto standard for health technology appraisal over time. Markov and discrete event simulation (DES) models were compared to investigate the impact of innovative modeling on the cost-effectiveness of disease-modifying therapies (DMTs) in relapsing–remitting multiple sclerosis (RRMS). Fingolimod was compared to dimethyl fumarate (DMF; in highly active [HA] RRMS), alemtuzumab (in HA RRMS) and natalizumab (in rapidly evolving severe RRMS). Comparator DMTs were chosen to reflect different dosing regimens.

Materials and methods: Markov and DES models used have been published previously. Inputs were aligned in all relevant respects, with differences in the modeling of event-triggered attributes, such as relapse-related retreatment, which is inherently difficult with a memoryless Markov approach. Outcomes were compared, with and without different attributes.

Results: All results used list prices. For fingolimod and DMF, incremental cost-effectiveness ratios (ICERs) were comparable (Markov: £4206/quality-adjusted life year [QALY] gained versus DES: £3910/QALY gained). Deviations were observed when long-term adverse events (AEs) were incorporated in the DES (Markov: £25,412 saved/QALY lost, versus DES: £34,209 saved/QALY lost, fingolimod versus natalizumab; higher ICERs indicate greater cost-effectiveness). For fingolimod versus alemtuzumab, when relapse-triggered retreatment was included in the DES, large cost differences were observed (difference between incremental cost is £35,410 and QALY is 0.10).

Limitations: UK payer perspective, therefore societal approach was not considered. Resource utilization and utilities for both models were not derived from the subpopulations; as the focus is on model type, input limitations that apply to both models are less relevant.

Conclusions: Whilst no model can fully represent a disease, a DES allows an opportunity to include features excluded in a Markov structure. A DES may be more suitable for modeling in RRMS for health technology assessment purposes given the complexity of some DMTs. This analysis highlights the capabilities of different model structures to model event-triggered attributes.

Keywords:

• Cost-effectiveness
• multiple sclerosis
• disease-modifying therapy
• economic modeling

JEL classification codes:

• C52
• I19

Introduction

Multiple sclerosis (MS) is an autoimmune disease with an estimated prevalence of over 100,000 people in the UK¹. MS is understood to be caused by progressive neuro-degeneration and is classified by the observed pattern of progression and by the occurrence of periods of sudden deterioration, known as relapses. Relapsing–remitting MS (RRMS) is the most common form of MS at diagnosis². It is characterized by relapses followed by periods where a patient either recovers partially or in full^3,4. As the number of relapses experienced increases, a permanent deterioration occurs over time. It is common for individuals diagnosed with RRMS to change over time to exhibit the secondary progressive form of MS (SPMS). This is characterized by a more continuous deterioration of function with fewer relapses being observed³.

Disease-modifying therapies (DMTs) are the main treatment options for RRMS in the UK. A number of different DMTs are currently recommended by the National Institute for Health and Care Excellence (NICE) in England and Wales^5–11. NICE guidance differs somewhat depending on whether an individual meets the criteria for one of the more active licensing subgroups of RRMS: “highly active” (HA) RRMS or “rapidly evolving severe” (RES) RRMS. HA RRMS is defined as highly active disease despite a full and adequate course of treatment with at least one DMT⁹. Rapidly evolving severe (RES) RRMS is defined by two or more disabling relapses in one year, and with one or more gadolinium enhancing lesions on brain magnetic resonance imaging (MRI), or a significant increase in T2 lesion load compared to recent MRI¹².

Various modeling types can be used for the economic evaluation of health technologies, including decision trees, Markov models and discrete event simulation (DES) models^13,14. The structure chosen is often determined by the relationship between the inputs and output measures required and it is best practice to choose an appropriate modeling technique that naturally models disease progression¹⁵. This should ideally be considered each time a de novo model is built for appraisal in health technology assessment (HTA). However, in practice there is considerable pressure within each decision-making system to retain consistency between appraisals in the same disease area, as a pragmatic approach to modeling¹⁶.

In RRMS, the cohort Markov approach has been used most frequently in HTA assessments in the UK and in other countries such as the USA^5–9,17. The cohort Markov approach considers homogenous cohorts who move between health states over time, using transition probabilities^18–20. As the number of treatment options for RRMS has expanded, the different attributes associated with each of the DMTs, specifically the differences in continuous or event-triggered dosing regimens and long-term implications of certain adverse events (AEs), have been found to be important considerations for cost-effectiveness modeling^21–23.

It has been suggested previously that Markov models may no longer be a suitable model choice when evaluating the cost-effectiveness of treatments in RRMS^23,24. The key limitation of a Markov model is that it is memoryless. This leads to the need for tunnel states to introduce memory to the model, which track events that last longer than one cycle length (for example event-triggered dosing regimens). When a Markov model attempts to capture the increasingly complex treatment pathway in RRMS, it could lead to so-called “state explosion”. A DES model is an example of an individual patient simulation model, in which individual patients, their attributes, events which occur to them and disease state are tracked over time²⁵. Recently, the DES model approach has also been applied to RRMS, in response to the differences in attributes now apparent between DMTs^23,24. As a DES model simulates individual patients through time, it may also be easier to extend the model to multiple sequential lines of therapy, treatment breaks and re-initiation, and model any long-term effects of treatments. It is the natural ability to model these attributes which may make a DES model more suitable for the evaluation of the cost-effectiveness of some DMTs in RRMS. In addition, a DES model conceptualizes the disease in a more natural way than a Markov model, with patients experiencing events instead of “forcing” patients into a number of mutually exclusive health states.

This study sought to use two different model types in the same disease area (RRMS) to investigate how different attributes associated with DMTs, namely event-triggered retreatment and long-term AEs, could impact the cost-effectiveness of DMTs, and whether a shift in model type should be introduced in this disease area. Markov and DES models previously published for fingolimod were used as the starting point^22,23. An overview of the different scenarios undertaken in this analysis is presented in Table 1.

Table 1. Summary of scenarios presented in the Markov and DES models.

Intervention	Comparator	Model	Scenario
Fingolimod	DMF	Markov	Markov HA RRMS base case
		DES	DES HA RRMS base case, which includes PML as a long-term AE for fingolimod
	Natalizumab	Markov	Markov RES RRMS base case
		DES	DES RES RRMS base case, which includes PML as a long-term AE for fingolimod and natalizumab
		DES	PML removed from fingolimod and natalizumab in DES
	Alemtuzumab	Markov	Markov HA RRMS base case
		DES	DES HA RRMS base case, with inclusion of alemtuzumab relapse-triggered retreatment, and long-term AEs for both fingolimod and alemtuzumab
		DES	Retreatment of alemtuzumab removed in DES
		DES	Retreatment of alemtuzumab removed, and PML removed for fingolimod, and renal disorders and long-term thyroid function removed for alemtuzumab; length of all AEs set to match Markov

Abbreviations. AEs, adverse events; DES, discrete event simulation; DMF, dimethyl fumarate; HA, highly active; PML, progressive multifocal leukoencephalopathy; RES, rapidly evolving severe; RRMS, relapsing–remitting multiple sclerosis.

Comparisons between several DMTs were chosen due to the differences in their dosing regimens. As noted, one DMT available for the treatment of RRMS is fingolimod, which is administered orally once daily²⁶. As with all DMTs, fingolimod is associated with AEs and previous economic models have captured these as short-term acute events. Fingolimod is recommended by NICE for the treatment of patients with HA RRMS, who have unchanged or increased relapse rate despite treatment with beta-interferon (the license wording has been revised subsequent to the NICE guidance); it is also licensed for RES RRMS and reimbursed for RES RRMS in Scotland and Wales^9,27–29.

Dimethyl fumarate (DMF) is another oral DMT, taken twice daily, and is recommended by NICE for the treatment of adult patients with RRMS which is not HA or RES RRMS⁸. Attributes associated with DMF treatment are not inherently difficult to capture in a cohort Markov framework, and it is hypothesized that DMF represents an exemplar of a DMT where a similar cost-effectiveness outcome would be achieved irrespective of economic model type chosen.

Natalizumab is a DMT which is administered intravenously once every four weeks. NICE has recommended natalizumab for the treatment of patients with RES RRMS only; it is also licensed for HA RRMS⁷. Natalizumab is associated with a risk of progressive multifocal leukoencephalopathy (PML). This rare but serious AE has consequences which could not be accurately modeled as an acute event and is therefore difficult to capture in a cohort Markov model, suggesting that the DES approach may be relevant for modeling this DMT. Consequently, it has been chosen for this study as an exemplar of a DMT with long-term AEs.

In addition to the PML risk of natalizumab mentioned above, PML has also been observed in a very small number of cases in post-marketing data from those treated with fingolimod or DMF, but not previously treated with natalizumab^30,31. For modeling purposes, however, it may be noted that the observed rates of occurrence of PML with these two DMTs are even lower than that observed with natalizumab and seem a priori unlikely to be influential.

Alemtuzumab has differing attributes compared with those of the aforementioned DMTs. It is administered intravenously as two courses: over five days in the first year, and over three days in the second year³². In the clinical development program of alemtuzumab, a relapse or other sign of disease activity warranted retreatment with a subsequent three-day course of alemtuzumab^33–35. The attribute of event-triggered retreatment does not fit easily with the memoryless Markov modeling approach, and when taken together with modeling long-term AE consequences (for instance autoimmune thyroid disorders), it has been chosen for this study as an exemplar of a DMT with multiple attributes that do not appear to fit easily with a Markov framework. Alemtuzumab is recommended by NICE for the treatment of patients with active RRMS⁶.

Methods

Patient populations and included studies

The baseline characteristics for the patient population in both models were taken from the pooled set of patients from the pivotal phase III trials for fingolimod (FREEDOMS³⁶, FREEDOMS II³⁷ and TRANSFORMS³⁸), irrespective of treatment arm allocation in the trial. Depending on the relevant subgroup for the comparison, the pooled population was from either the HA or RES RRMS subpopulations of these trials.

The baseline characteristics for both subpopulations are summarized in Table 2. The DES model used the individual patient data for each subgroup, whilst the Markov took the average for each characteristic, and scaled these for 1000 patients, which was the size of the cohort.

Table 2. Cohort level characteristics of the IPD used.

Characteristics	Value HA RRMS				Value RES RRMS
	DES		Markov		DES		Markov
Size of the population	1381		1000		528		1000
No. of cohort iterations	2000		–		2000		–
Number of patients in initial EDSS scores
0	82	5.94%	59.4	5.94%	39	7.39%	73.9	7.39%
1	118	8.54%	85.4	8.54%	55	10.42%	104.2	10.42%
2	456	33.02%	330.2	33.02%	203	38.45%	384.5	38.45%
3	309	22.38%	223.8	22.38%	105	19.89%	198.9	19.89%
4	268	19.41%	194.1	19.41%	82	15.53%	155.3	15.53%
5	101	7.31%	73.1	7.31%	26	4.92%	49.2	4.92%
6	47	3.40%	34.0	3.40%	18	3.41%	34.1	3.41%
7	0	0%	0	0%	0	0%	0	0%
8	0	0%	0	0%	0	0%	0	0%
9	0	0%	0	0%	0	0%	0	0%
Female to male ratio	2.8:1		2.8:1		2:1		2:1
Age of cohort (mean)	38.23		38.23		34.85		34.85
Years since diagnosis (mean)	6.42		6.42		4.61		4.61
Source	HA RRMS subgroup from FREEDOMS I, FREEDOMS II and TRANSFORMS		HA RRMS subgroup from FREEDOMS I, FREEDOMS II and TRANSFORMS		RES RRMS subgroup from FREEDOMS I, FREEDOMS II and TRANSFORMS		RES RRMS subgroup from FREEDOMS I, FREEDOMS II and TRANSFORMS

Abbreviations. DES, discrete event simulation; EDSS, expanded disability status scale; HA, highly active; IPD, individual patient data; RES, rapidly evolving severe.

Structure of the models

The structures of both the Markov and DES models have been described in previous publications^22–24. Both models are based on patient Expanded Disability Status Scale (EDSS) for disease status and incorporate the occurrence of relapses. This is in line with previous UK RRMS models²¹. Both models evaluated cost-effectiveness from the perspective of the National Health Service (NHS) and Personal Social Services (PSS) in England. The structures of the Markov and DES models are presented in Figure 1.

Figure 1. The structures of the Markov and DES models. In the Markov model when patients are transferred to SPMS, their EDSS score increases by 1; this assumption was applied in the DES analyses to match the approach taken in both models. Note that in the models, it is possible for a patient to move between states that are more than one EDSS point apart. For example, a transfer from RRMS with EDSS score of 1 to EDSS score of 7 is possible, without going through all intermediate steps. The omission of these transition arrows is for the sake of clarity. Moreover, from each EDSS state patients can transfer to the death state.

Abbreviations. DES, discrete event simulation; EDSS, expanded disability status scale; QALYs, quality-adjusted life years; RRMS, relapsing-remitting multiple sclerosis; SPMS, secondary progressive multiple sclerosis.

The DES model was developed in C++ and Microsoft Excel, whilst the Markov model was developed fully in Microsoft Excel. Each simulated patient in the DES model has four attributes fixed: initial age, sex, initial EDSS score and time since diagnosis of MS²³. The DES model tracks individual patients through time and continues to track their history during the lifetime horizon of the model.

The Markov model uses a discrete time, cohort approach with 21 health states based on 10 EDSS scores (10 for RRMS, 10 for SPMS and the “Death” state)²². Two cohorts of patients are simulated through the Markov model, one for the intervention and one for the comparator. This Markov model uses transition probabilities to calculate when a proportion of the cohort moves from one state to another, and models the cohort annually. The Markov model is inherently memoryless; only when tunnel states are added is it possible to trace patient cohorts through time.

Natural history

To ensure consistency of comparison the natural history data used in both models originated from the same source. Natural history data for both the HA and RES RRMS subgroups were modeled based on the data for each subgroup from the placebo arm of the FREEDOMS³⁶ and FREEDOMS II³⁷ studies. There were insufficient data to calculate the risks at EDSS levels of eight or above; these risks were calculated based on the London Ontario dataset, which was also used to calculate the risks in changing from RRMS to SPMS and transitions within SPMS³⁹. The natural history risks for moving between EDSS states were then adjusted by the relative risk of each DMT treatment versus placebo, using the subgroup analyses where available from the phase III clinical trials of each of the DMTs (Table 3).

Table 3. Efficacy inputs for the Markov and DES models.

DMT	Subgroup	Outcome vs. placebo (over 24 months)		Source
DMF	HA RRMS	ARR ratio	0.57	Pooled DEFINE and CONFIRM HA subgroups^40
		HR for 3 month confirmed disability progression	1.19
Fingolimod	HA RRMS	ARR ratio	0.52	In line with the previous Markov model in HA RRMS^22; this analysis has also been published in Derfuss et al.^41
		HR for 3 month confirmed disability progression	0.66
	RES RRMS	ARR ratio	0.43	FREEDOMS + FREEDOMS II pooled RES subgroups^23
		HR for 3 month confirmed disability progression (definition as per AFFIRM trial)	0.61
Natalizumab	RES RRMS	ARR ratio	0.19	AFFIRM treatment-naïve RES subgroup (Hutchinson et al.)^42
		HR for 3 month confirmed disability progression	0.47
Alemtuzumab	HA RRMS	ARR ratio	0.54	Calculated using Bucher indirect comparison (see Supplementary Table 1)
		HR for 3 month confirmed disability progression	0.58

Abbreviations. ARR, annualized relapse rate; DES, discrete event simulation; DMF, dimethyl fumarate; HA, highly active; HR, hazard ratio; RES, rapidly evolving severe; RRMS, relapsing–remitting multiple sclerosis.

Clinical inputs

All clinical, cost and utility inputs were aligned, where possible, across both models. The clinical efficacy parameters for each DMT are presented in Table 3. There are differences in how AEs have been added to the models, with the DES model allowing for long-term consequences of AEs to be modeled, which is not possible in the Markov model presented here. The addition of relapse-triggered retreatment costs was possible in the DES model.

Cost inputs

All cost and price inputs in both models related to an NHS and PSS perspective, in line with the NICE reference case⁴³. Drug acquisition costs for all DMTs were based on list price, with none of the nationally available patient access scheme (PAS) discounts applied in the base case analysis, as these are all confidential⁴⁴. Costs for resource use in terms of administration, monitoring, AEs and drug acquisition were included in both models (Supplementary Tables 2–5). The possibility of relapse-triggered retreatment with alemtuzumab after the first two doses was only feasible in the DES, as this model type is capable of capturing such events more naturally. The costs associated with relapses in both models were taken from the 2016–17 National Tariff⁴⁵. AEs may also be specified differently between the two models, with the DES model capable of tracking the costs and disutilities associated with long-term AEs whereas the Markov model was limited by its cycle length to one-year events.

The accrual and discounting of costs and quality-adjusted life years (QALYs) are handled differently in each model. In the Markov model costs and QALYs are calculated and discounted in annual cycles, with half-cycle correction. In the DES model, costs and QALYs are applied at the relevant point in time or across the relevant duration of time and discounting is applied on a continuous-time basis.

Utility inputs

As with other inputs, the utility weights for disease status were aligned across both models and were in line with those previously used in other models for RRMS (Supplementary Table 6)^26,34,46,47. These were collected using the EuroQol 5-Dimension (EQ-5D) study, the instrument preferred by NICE⁴³. Disutilities associated with AEs for each DMT were also matched across the Markov and DES models (Supplementary Table 7), with the values largely based on the manufacturer’s submission for each DMT^26,34,46,47. The utility inputs were entirely the same across both models; however, in the Markov they are applied annually, and therefore discounted annually. The DES model applied utility values continuously, and the same approach is taken with discounting. Carer disutility has been included in both models. This disutility was taken from the natalizumab manufacturer’s submission to NICE⁴⁶.

Model outcomes

The primary outcomes of both models were in line with previous health economic models in RRMS, and measured the total costs and QALYs for each intervention and comparator^26,34,46,47. This allowed for the calculation of the incremental cost-effectiveness ratio (ICER). Both costs and QALYs were discounted at an annual rate of 3.5%, as per the NICE reference case⁴³. Both models considered a lifetime horizon, necessarily implemented in slightly different ways; in the Markov this was modeled as baseline cohort age plus 50 years, whereas the DES tracked each simulated patient until death (maximum age capped at 100 years due to data availability)^22,24.

Sensitivity analyses

Deterministic and probabilistic sensitivity analyses (DSA and PSA) were conducted in both the Markov and DES models. Where available, lower and upper confidence intervals were used as sensitivity analysis inputs; if unavailable a ±20% change was applied to the parameters. The PSA was conducted using 1000 iterations in which each parameter was randomly sampled from its probability distribution²³. This was considered a pragmatic limit on the run-time for the largest model (HA RRMS DES model), as this took approximately one week. Inputs for the PSA are listed in Supplementary Table 8.

Results

Base case results

The results for each of the comparisons between the Markov and DES models are presented in Figure 2. The results indicate that there is little change in the comparison of fingolimod versus DMF between the two models in the HA RRMS population. This demonstrates the cross-model internal validity of the Markov and DES models, as it illustrates that the different model structures produce comparable results when based on the same assumptions and inputs. A difference in results was observed for fingolimod and natalizumab in the RES RRMS subpopulations between the two model types. This was due to the addition of PML as a long-term AE for both DMTs in the DES model, which was not straightforward to model in the Markov model. A scenario analysis was conducted where PML was removed from both fingolimod and natalizumab in the DES model, to determine the influence of PML on results. When PML is included for both DMTs in the DES model, fingolimod is cost-effective, as the ICER is in the southwest quadrant of the cost-effectiveness plane, meaning that fingolimod results in a cost saving per QALY lost, which is above the usual upper NICE threshold of £30,000/QALY when compared to natalizumab (£34,209 cost saving per QALY lost). In the Markov model, however, whether fingolimod is cost-effective depends on the choice of £20,000/QALY or £30,000/QALY as the threshold, as the ICER for fingolimod versus natalizumab results in a cost saving of £25,412 per QALY lost, therefore fingolimod is cost-effective at £20,000 per QALY, but not the £30,000 per QALY threshold. With PML removed from both comparators in the DES model, the ICER is more comparable with that of the Markov model (£26,777 cost saving per QALY lost).

Figure 2. Cost-effectiveness plane of deterministic results from comparison of DES and Markov models. All analyses were carried out with the intervention and comparators at their list price. A confidential PAS discount is available for fingolimod and DMF, this was not considered in the model. Values for each point are the incremental QALY and incremental cost for each comparison. Note: ‘No AEs’ indicates the removal of serious long-term AEs from the DES model which were not included in the Markov model (e.g. PML). ‘No retreatment’ indicates the removal of relapse-triggered retreatment for alemtuzumab in the DES model.

Abbreviations. AE, adverse events; ALE, alemtuzumab; DES, discrete event simulation; DMF, dimethyl fumarate; FIN, fingolimod; HA, highly active; ICER, incremental cost-effectiveness ratio; NAT, natalizumab; PML: progressive multifocal leukoencephalopathy; QALY, quality-adjusted life year; RES, rapidly evolving severe; RRMS, relapsing-remitting multiple sclerosis; WTP, willingness-to-pay.

A clear difference was observed in the results between fingolimod and alemtuzumab in the HA RRMS subpopulation, when comparing across the Markov and DES model types. In the base case, both the Markov and DES model report that fingolimod is not cost-effective at list price compared to alemtuzumab (Figure 2; in practice, a nationally available confidential discount is offered for fingolimod in the NHS). However, there are considerable differences between the incremental costs between the two models (Markov incremental cost: £40,442; DES incremental cost: £5032), attributed to relapse-triggered retreatment. Scenario analyses were conducted with relapse-triggered retreatment removed for alemtuzumab in the DES model, which increased the incremental cost to £37,074, moving it closer to the incremental cost observed in the Markov model (£40,442). The incremental QALYs did not change from the base case in both scenarios (-0.18 and -0.08, respectively), as retreatment only affects the cost implications. This demonstrates that there is no other differential uncertainty in the models and cross-validates the models against each other, as also demonstrated in the fingolimod versus DMF example. A further scenario analysis was conducted where several of the long-term AEs associated with alemtuzumab were removed from the DES model alongside retreatment, to ensure that AEs were aligned across both models as much as feasibly possible. This resulted in the incremental QALY from the DES model being in line with the incremental QALY observed in the Markov model (−0.18 in both models).

Sensitivity analyses

A deterministic sensitivity analysis was undertaken on the Markov and DES models, and results are presented in the Supplementary Material. Incremental net monetary benefit at the NICE £20,000/QALY threshold was used for this analysis. For the comparison between fingolimod and DMF, the four most influential parameters in both the Markov and DES models were the same, albeit in a different order. In all comparisons, the most influential factors were the cost of each treatment, and the relative progression risk of each of the DMTs being compared. Probabilistic sensitivity analyses were also carried out in each model type (results in Supplementary Material). Results from each comparison were compared between Markov and DES models. Where the models had been aligned to be similar (for example when PML was removed from natalizumab and fingolimod in the DES model), the cost-effectiveness acceptability curves were comparable (close or overlapping throughout). Systematic differences were observed when the models were not aligned (that is, when utilizing the additional flexibility of the DES), as expected.

Discussion

It is generally accepted that an economic model should take the simplest form possible to adequately reflect the progression of a disease and assess the impact on an intervention or interventions, and that it is best practice to choose an appropriate modeling technique that naturally models disease progression¹³. However, in practice there is considerable pressure for a pragmatic approach to be taken within decision-making processes, to retain consistency in the same disease area¹⁶. The increasing number of DMTs available in RRMS has led to an increasing complexity of treatment options. This paper offers a clear example of how the evolution of more complex treatment options has reached a point where the simplest model form to adequately capture the disease pathway may become a DES model, moving away from the Markov model approach previously applied in this disease area. This can be inferred from the difference in ICERs observed in the fingolimod versus alemtuzumab comparison between the Markov and DES models, and the impact which relapse-triggered retreatment and the modeling of long-term AEs had on the results. The challenge of modeling treatment in RRMS is not only due to the increasing number of treatments available, but that each treatment will bring its own complexities, in this case event-triggered retreatment requirements or more severe AEs with long-term sequelae. Long-term AEs can be modeled within a Markov structure; however, the inclusion of a large number of tunnel states in a Markov model is not considered to be a practical approach, as this quickly becomes unmanageable leading to “state explosion” and further increasing the runtime of such a model. An alternative within the Markov framework that has been used to avoid “state explosion” is to incorporate an estimate of the outcomes of long-term AEs without modeling them as tunnel separate states; however, the published example of this approach still failed to capture the costs and consequences of PML beyond one year¹⁷.

Important differences are observed when comparing the results from the Markov and DES models across different DMTs. Results for fingolimod versus DMF are comparable between both models, with the slight deviations observed likely due to the differences in application of discounting between models. In the comparison of fingolimod and DMF the Markov modeling approach may be considered most suitable as it is a less complex model and the choice of model type does not seem likely to influence the final decision made.

For the other comparisons made in this paper, there are aspects of the decision problem which cannot be naturally captured in a Markov model, and therefore a DES model may be the more suitable approach, as suggested by Allen et al.²¹. A divergence in the results is observed with the addition of PML as a long-term AE for both the fingolimod and natalizumab exemplars in the DES model although, as noted, there are approaches to incorporating estimates of long-term AEs such as PML within a Markov framework. The divergence becomes even more apparent with the addition of relapse-triggered retreatment to the DES model for the comparison between fingolimod and alemtuzumab, leading to considerable variation in the discount required for fingolimod to be cost effective versus alemtuzumab between the two model types. As the DES model allows for these economically relevant attributes of treatment to be modeled, it may indicate that a DES model is the more appropriate model type for this comparison.

Strengths and limitations

The strengths of the analysis presented here include that the comparison was based on two published peer-reviewed models^22–24, both of which have been used for UK HTA appraisals^9,48, providing clear relevance to HTA decision-making in the UK. Additionally, the development of the DES model was identified as desirable as the outcome of a published systematic review of UK economic models for RRMS²¹. Development of both the Markov and DES models by one team allowed the models to be closely aligned in all respects except where differences in the model structure represented specific and desirable changes^22–24. The comparison presented thus represents a rare opportunity to isolate and investigate the effect of model type from the effects of other model differences; thus, although the results relate to the specific decision problem presented, they remain of broader interest.

The DES model is a more complex approach and requires additional time to build. The increased computational requirements of running the DES model and programming accounted for most of this time. Model run-time was not considered an obstacle, as using compiled C++ code is computationally efficient and resulted in the analyses running relatively quickly (i.e. enabling simulation of in the order of 2,000,000 patients per arm in several minutes). Using C++ allowed undertaking probabilistic sensitivity analyses for 1000 replications using the full sets of simulated patients in a realistically achievable time, not having to restrict to a reduced set as is sometimes the case with DES models; it was also possible to run pairwise analyses in parallel to make the process more efficient. It would be preferable to run 5000 or 10,000 replications; however this was not tested due to time constraints. The choice of software used to enable swift execution of a DES model is vital for use in time-constrained formal appraisal processes. NICE has stated that a practical approach to implementing a DES model is required⁴⁹.

In terms of limitations, the decision problem discussed is limited to one subset of patients in one disease area, with particular focus on the approach taken by payers such as NICE in the UK. Other regions may take a societal or other approach in their analysis of the decision problem, which may lead to alternative results to those presented in this paper and may limit the generalizability of these results. General limitations for both models themselves, rather than the comparison between them, include the fact that both the costs and utilities are not derived from the specific HA or RES RRMS subpopulations, as reported in previous publications; however, as the focus of this study is on the features of the decision problem that make model type itself a significant factor in the outcome, input limitations that apply to all models in the disease area are less relevant^22–24. Currently there are no data available within this disease area that match the lifetime time horizon taken within this model, meaning that extrapolation of results from clinical trials was necessary. This is a limitation for all models developed for RRMS. However, for the purpose of comparing the Markov and DES models it is assumed that aside from the differences in attributes investigated, there is no differential uncertainty to consider between the models. This is substantiated by the results for the comparison of fingolimod versus DMF in the Markov and DES models, where outputs were similar, as well as the scenario analyses for the other comparisons. When comparisons are made for scenarios without the additional features captured in the DES model, the outputs were comparable between both models. Whilst the possibility of incorporating long-term AEs into the Markov model using non-state methods has not been investigated here due to additional work required, it is anticipated that results would be comparable between models.

Strengths and limitations of the DES and Markov models themselves have been described previously^22–24.

Implications for health care decision-making

Considering health economic modeling in general, in situations with new treatment options available or considered the most relevant, different choices of model types from those adopted historically in the given disease area may need to be made. For example, in rheumatoid arthritis, DES models have been used as an alternative approach to Markov models, and, having been initially accepted by HTA bodies, more recently it has been suggested in appraisals that the cohort Markov approach does not reflect the disease^50,51.

It has been suggested previously that comparing a simple model structure to that of a DES may, in fact, provide an understanding of the benefits of a DES model, and insight into when a DES model may be more suitable⁵². The results observed in the present study suggest that DES models may be more appropriate than Markov models for the decision problem now faced by payers in RRMS, where relapse-triggered retreatment with alemtuzumab is pertinent to the analysis. The ability to model event-triggered retreatment has potentially significant consequences on the cost-effectiveness results for comparisons that payers must make. The variation in the incremental costs and QALYs observed between the DES and Markov models for the comparison of fingolimod and alemtuzumab is a clear exemplar of why a DES may most adequately model the treatment options now available for RRMS. This DES approach has been accepted by the All Wales Medicines Strategy Group (AWMSG) in a recent appraisal in RRMS^28,48.

The differences observed between the two models are important for HTA decision makers in the UK, for example NICE, the Scottish Medicines Consortium and AWMSG. The results clearly show that the model type used can influence results for HTA bodies for the treatment of RRMS in cases where model type allows more realistic assumptions to be made in the analysis. Here, the benefits of DES modeling outweigh the additional resources and time associated with building a more complex model type for use in decision-making processes.

Conclusions

This study looked specifically at the comparison of cost-effectiveness model types for exemplar DMTs in RRMS. The results presented demonstrate that choice of model type, where it allows modeling of more realistic assumptions, does have considerable impact on cost-effectiveness outcomes, and therefore could influence reimbursement decisions. As the complexity of treatment options for RRMS grows with the introduction of new DMTs and relapse-triggered retreatment, a DES model structure may more naturally model the cost-effectiveness of treatments for RRMS than a “simpler” Markov model structure, and therefore be more appropriate for cost–utility analyses in RRMS for HTA and reimbursement purposes in the future. Such a change of modeling approach may affect some reimbursement decisions in RRMS. This analysis may provide insights which are generalizable to other disease areas.

Transparency

Declaration of funding

This study was funded by Novartis Pharmaceuticals UK Ltd.

Declaration of financial/other relationships

M.A.K. and N.E.A. have disclosed that they are paid employees of Novartis. K.M.N. and S.M.M. have disclosed that they are paid employees of Costello Medical Consulting Ltd, Cambridge, UK, which was contracted by Novartis to undertake the economic modeling and medical writing.

A peer reviewer on this manuscript received consulting fees from PhRMA for MS-related projects on quality of life in MS and was also a recipient of a PhRMA Foundation Post-Doctoral Fellowship related to MS disease modeling. The remaining peer reviewers have no relevant financial or other relationships to disclose.

Acknowledgements

This work was built on two pre-existing economic models. Creators of these economic models were authors on the primary publications for the models (Maruszczak et al. 2015²²; Montgomery et al. 2017²³).