Evaluating discrete choice experiment willingness to pay [DCE-WTP] analysis and relative social willingness to pay [RS-WTP] analysis in a health technology assessment of a treatment for an ultra-rare childhood disease [CLN2]

ABSTRACT

Background

Conventional cost-effectiveness analysis [CEA] using cost per QALY thresholds may counteract other incentives introduced to foster development of treatments for rare and ultra-rare diseases. Therefore, alternative economic evaluation methods were explored, namely Discrete Choice Experiment Willingness to Pay (DCE-WTP) and Relative Social Willingness to Pay (RS-WTP), to value interventions for an ultra-rare childhood disease, Neuronal Ceroid Lipofuscinosis type 2 (CLN2).

Research Design and Methods

Treatment for CLN2 was valued from a citizen’s (‘social’) perspective using DCE-WTP and RS-WTP in a survey of 4,009 United Kingdom [UK] adults. Three attributes (initial quality of life, treatment effect, and life expectancy) were used in both analyses. For DCE-WTP, a cost attribute (marginal income tax increase) was also included. Optimal econometric models were identified.

Results

DCE-WTP indicated that UK adults are willing to pay incremental increases through taxation for improvements in CLN2 attributes. RS-WTP identified a willingness to allocate >40% of a pre-assigned healthcare budget to prevent child mortality and approximately 15% for improved health status.

Conclusions

Both techniques illustrate substantive social WTP for CLN2 interventions, despite the small number of children benefitting. This highlights a gap between UK citizens’ willingness to spend on rare disease interventions and current funding policies.

KEYWORDS:

• CLN2
• Rare Disease
• Relative Social Willingness to Pay
• RS-WTP
• Discrete Choice Experiment Willingness to Pay
• DCE-WTP
• Discrete Choice Experiment
• DCE
• Willingness to Pay
• Conjoint Analysis

1. Introduction

1.1. Defining rare disease

Definitions for rare or orphan diseases (conditions with low prevalence) differ from country to country [1], and there is no universally accepted definition of rare disease [2,3]. A global review of rare disease terminology found 58% of definitions included a prevalence threshold, with an average global threshold of 40 to 50 cases per 100,000 people. Moreover, the qualitative descriptors that were used most frequently were ‘life-threatening’ (15%), followed by ‘debilitating’ (10%), ‘severe‘ (3%), and ‘intractable’ (1%) [3]. According to a European Community [EC] regulation, the European Union [EU] defines an orphan disease as one affecting less than 1 in 2,000 individuals [4]. In the USA, orphan diseases are defined as having a prevalence of fewer than 200,000 affected persons [5]. The term ‘ultra-orphan disorders’ has also been used though there is no current globally recognized definition for this term. It has been used by the United Kingdom [UK] official Health Technology Assessment [HTA] body National Institute for Health and Care Excellence [NICE] to refer to drugs indicated for conditions with a prevalence of fewer than 1 per 50,000 persons [5,6].

1.2. The need to apply alternative HTA valuation frameworks applicable to rare disease

Our application is neuronal ceroid lipofuscinosis type 2 (CLN2), a very rare childhood disease and form of Batten disease, which is a pediatric neurodegenerative disease with onset in late infancy. While the epidemiology of CLN2 disease does vary between countries and regions, incidence generally falls below or close to the NICE threshold for an ultra-rare disease of 1 per 50,000 live births [7–11]. Estimated incidence of CLN2 disease is 0.5 per 100,000 live births (0.15 to 0.78 per 100,000 reported in European countries [7–9] and up to 9.0 per 100,000 in Newfoundland [10]), and a prevalence of ~0.6–0.7 per million in Scandinavia [7,11].

Unique challenges are presented when developing and utilizing an appropriate methodology for the evaluation of a very rare disease of childhood like CLN2. A number of HTA bodies, including NICE and the Scottish Medicines Consortium [SMC], now utilize different processes to facilitate special consideration for rare diseases [2,12–14] these processes themselves remain in a continuous state of reevaluation and redesign.

This paper discusses the application of two valuation techniques [Discrete Choice Experiment Willingness to Pay (DCE-WTP) and Relative Societal Willingness to Pay (RS-WTP)] that might be particularly appropriate for the evaluation of rare diseases or ultra-rare diseases. These techniques explore the potential value of assessing social preferences for treatments of rare diseases, in order to value and prioritize treatment interventions.

One of the main reasons for using these two techniques to value an ultra-rare childhood disease is that conventional cost-effectiveness analysis [CEA] models, used for HTA purposes, are associated with a number of limitations [15]; the cost-utility analysis [CUA], a variant of CEA, also incorporates an assumption of strict proportionality between the number of patients benefitting and the social value implied by the model [15]. This approach flies in the face of evidence, which suggests that, when people adopt the social perspective of a citizen, they have a preference for sharing the health budget in such a way that it does not exclude patients who require services that are not cost-effective [16]—which would include many orphan medicinal products [17]. By establishing social citizen’s preferences using DCE-WTP or RS-WTP, a social perspective, which does not exclude individual preferences for potential sharing can be facilitated.

1.3. Treatments for rare diseases: a small but increasing proportion of international pharmaceutical expenditure

Concerns have been raised that therapies for orphan diseases might impose ‘substantial and increasing costs [to healthcare systems]’ [18]. In many countries, policies have been adopted to encourage the development of such therapies [19,20]; total and proportional expenditures on rare disease therapies are continually rising across both developed and developing economies, due to the resultant steady growth in their development and commercialization [2]. Almost by definition, however, such development costs can only be recouped from a relatively small market [6], leading to price tags often disproportionately higher than those of therapies indicated for more common diseases. As a result, therapies for ultra-orphan diseases are unlikely to meet the conventional cost-effectiveness thresholds often stipulated by pharmacoeconomic-driven HTA bodies such as NICE [£20,000 to £30,000/QALY in the UK] [5].

In many countries, policies have been adopted to encourage development of treatments for rare or orphan diseases [19,20]. Both total and proportional expenditures on rare diseases are continually rising due to steady growth in their development and commercialization [2]. That being said, a recent systematic review [6] of nine studies on the budgetary impact of orphan medicinal products (OMPs) across 11 countries concluded that despite per-patient annual treatment costs averaging nearly 300,000 euros, per-capita spending upon OMPs was small. Spending on OMPs, as a percentage of overall pharmaceutical expenditure, was shown to be relatively low among the countries sampled, ranging from 0.8% in Latvia to 7.8% in Bulgaria.

A patient-directed policy framework for managing Orphan and Ultra-Orphan drugs, facilitating the incorporation of patient preferences into decision-making [21], was developed with the involvement of patients and families from the rare diseases community in Canada. This framework may prove useful for clinical problems affecting adults. However, for diseases affecting young children, more relevant may prove to be the preferences of parents whose children are affected, the general public, and treating physicians, as young children—and in particular, those with neurodegeneration—may not be able to register preferences themselves. Alternatively, if a question relating to the distribution of societal resources for health care being addressed, a social perspective may be very valuable [22,23]. Such a social perspective recognizes that a society’s willingness to support people with rare diseases ought to be integral to the development of reimbursement policies [22].

1.4. Implied perspectives of alternative health economic analyses

The choice of appropriate health economics outcome measures cannot be meaningfully made without considering the perspective of the analysis adopted. The Quality Adjusted Life Year [QALY], which is the most prevalent outcome measure and underpins CUA, fits with a QALY maximization objective for the healthcare system [24], and has in the past been used as a measure of social value. This is because it can provide a measure of ‘individual utility’ as a result of health (gains), which in turn can be aggregated such that the sum total is believed to be a direct reflection of the social value [25]. It has therefore been argued that QALYs represent society’s preferences over different health-care outcomes [26,27]. However, the QALY maximization approach has been described as ‘descriptively flawed’ [24] in part because ‘Rather than being linear in quality and length of life, it would seem that social value diminishes in marginal increases in both’ [24].

A further limitation of the CUA approach with respect to application to children with CLN2 is that direct estimates of QALYs cannot be obtained from young children, and if adults provide them on behalf of their child, this could introduce bias if adult respondents do not accurately discern the degree of pain or discomfort experienced by their offspring. Moreover, due to the limitations of established CEA/CUA methods in evaluating rare diseases, it has been suggested instead that ‘social preferences’ for rare disease therapies should be used [23]. The main justification for this lies in the fact that conventional CEA/CUA thresholds adopt a narrow perspective on the preferences of patients, and these are inevitably driven by self-interest. Another limitation of the CUA approach is that it only takes quality of life and life-years into account [28], when other criteria, which can be valued by preference elicitation methods may not be appropriately established using CUA utility measures [29]. If a preference elicitation technique is applied, such as DCE-WTP [30,31] or RS-WTP [30–35], then the preferences elicited can consider the wider societal willingness to support people with rare diseases via reimbursement policies [22]. Such societal preferences have the advantage that they can incorporate the preferences of the payer. This is because many members of society are also taxpayers.

In light of these limitations of conventional CEA/CUA methods, a case can be made for instead applying preference elicitation methods. However, when applying such techniques, it is vital to address the issue of adopting an appropriate perspective of analysis. Paul Dolan and colleagues suggested [36] a two-dimensional framework to classify perspectives applied when eliciting preferences within health-care studies. In terms of this theoretical framework [36], both the DCE-WTP analysis and RS-WTP analyses, which we have applied involved a social ex-ante perspective, as they are both applied to establish taxpayer (social) willingness to pay for changes with respect to uncertain health states. Another more recent analysis [37] has expanded upon this framework, adding complexity to it. In terms of this newer framework, our DCE-WTP and RS-WTP analyses could be said to have adopted a ‘Non-use’ perspective. This is because none of the respondents to our analyses had CLN2 disease, and since it is a progressive childhood disease, none of them will acquire it. Therefore, none of the respondents will be direct beneficiaries of treatment interventions for CLN2 disease. Yet, our analysis does not fit logically into any of this paper’s [37] remaining categories relating to the context/timing of health outcomes. Our DCE-WTP and RS-WTP analysis relates to the consideration of ex-ante health outcomes. However, this analysis [37] relates to states defined by four ex-ante cases, and none of these cases fit with the particular characteristics of CLN2 disease. This is because all the four ex-ante cases are defined either by probability statements (case 3 and case 4) or certainty in relation to the likelihood of becoming ill [cases 1 and 2]. Unfortunately, because CLN2 is a rare disease, its evidence base is limited, so neither probability statements nor statements about certainty of many CLN2 health states can be reliably presented to respondents within DCE-WTP and RS-WTP scenarios. So, we concluded that even this complex summary of perspectives for healthcare research [37] lacked the complexity to cater for this very rare disease affecting children. This justifies the application of stated preference techniques such as RS-WTP [32–34,38] and DCE-WTP [30,31] without including information about probabilities or certainty (with respect to all health states) within the choice scenarios presented to respondents.

1.5. Preference elicitation methods and alternative health research techniques

Although most CUA studies (for example, those involving the application of the EQ-5D and EQ-VAS measures) concern themselves with patients’ personal ex-post valuations, some of them involve the application of standard gamble [SG] [28], which considers ex-ante patient health outcomes under conditions of uncertainty. Others involve the application of the time trade-off [TTO] methodology [39], which can be used to assess certain and uncertain outcomes through time.

Numerous other health outcome studies involve the application of person trade-off [PTO] [26,35,36,39–41]. PTO studies ask respondents to trade-off treatment of one group of patients, with one set of characteristics, against the treatment of another group of patients with another set of characteristics [40]. Such an approach can therefore be used to simultaneously address efficiency concerns (extent of health improvement obtainable), alongside equity (distribution of health improvements) considerations. Moreover, it can also be used to assess societal preferences for representative members of the adult population [40]. So, this approach fits in with the conceptual idea [42] that there may be an underlying social welfare function [SWF] relating to both efficiency and equity that PTO can assess. However, a limitation of the mainstream PTO approach [35,39,40] for our policy question was that we wanted to generate a monetary valuation for a drug to treat a very rare childhood disease. RS-WTP can be used to establish trade-offs using a monetary valuation [35], and DCE-WTP has been extensively applied to generate societal monetary valuations for health interventions [31]. Another approach, which we could have adopted [41] incorporates both public concerns for fairness/equity issues and also addresses the particular value that respondents place upon saving the life of a young person (known as ‘a SAVE’). It involves applying a non-QALY-based variant of a PTO framework [41]. However, a limitation of this approach [41] is that it is ill-equipped to take rarity issues into account, although this variant of PTO has the advantage that it can produce monetary valuations [41].

1.6. Preferences and rare diseases

It has also been argued [43] that when systematically reviewing a large number of healthcare-based contingent valuation [CV] studies, ‘only 24% of the studies have asked the general population,’ because they were advancing the view that ‘citizen preferences’ are relevant for social valuations. It has also been pointed out that when using stated preferences to elicit valuations, the choice of the payment vehicle should be made with reference to how it would be financed if it were implemented [44]. As a pharmaceutical product is being evaluated, this would imply that social preferences (citizen preferences) rather than patient preferences ought to be elicited. It also follows that, if you are eliciting preferences with the objective of estimating benefits to inform the ‘benefits’ side of a Cost-Benefit [CBA] analysis (for example, to evaluate a health-care technology/pharmaceutical product) that because CBA requires a societal perspective, then citizen (general public) preferences are relevant to inform CBA [45].

A compelling case can be made for a radical re-think of health economics evaluation, involving the modification of approaches to economic evaluation in order to take ‘fairness’ into account [46]. Such an approach, in the context of state-funded health care the UK’s National Health Service [NHS] would involve a focus upon ‘social transfers’ between taxpayers and beneficiaries, where the nature and scope of the transfers is determined by the level of ‘social generosity.’ It follows that techniques such as DCE-WTP and RS-WTP analyses can be applied to establish the appropriate level of social generosity and help address the shortfalls of established methodologies [47].

The DCE method is increasingly being applied, as reflected and recorded in systematic reviews of the health-care DCE literature [31,48,49] and is gaining an increasing acceptance as an HTA tool [31]. It has the advantage over mainstream contingent valuation/willingness-to-pay analyses in that it elicits information about how much changes in different component parts of healthcare provision might be valued. This means that the impact of different configurations of changes in service provision can be assessed, rather than just one particular defined change in service provision [30].

In addition to being able to adopt a traditional HTA patient-based perspective, by selecting alternative respondent groups for a DCE, the method can be used as an instrument to take the preferences of other respondent groups into account [50], ascertaining and comparing different stakeholder group perspectives. Moreover, whilst the perspective of traditional cost-effectiveness threshold analysis is limited because it relates to patients and quality of life/length of life issues, DCEs often also take aspects of utility, which extend beyond health outcomes into account, for example, relating to process outcomes [30,49,51]. Of importance for the purposes of this analysis, DCEs can also be used to ascertain a citizen’s societal preferences [43], which can feed into the ‘benefits’ side of Cost-Benefit Analyses [45,52]. RS-WTP analysis in theory can be used for the same purpose.

2. Objective

The objective was to evaluate social or citizen preferences using DCE-WTP, a technique which is becoming increasingly established as a preference elicitation technique for HTA [31], and which has been used already to evaluate rare diseases [53–58]. The DCE-WTP technique can be used to calculate the average respondent marginal willingness to pay for defined changes in attribute levels. The aim was to use this technique, alongside the newer preference elicitation technique RS-WTP [32–35,38] (which has particular applicability to a disease such a CLN-2, which is potentially fatal because it also values fatalities avoided), to establish relative social willingness to pay or absolute social willingness to pay [SWTP] for a medication to treat CLN2 disease.

A sample of UK adults was selected because adults were capable of making meaningful decisions, which trade off increases in income tax with defined improvements in CLN2 treatment effects or, CLN2 quality of life, or overall life expectancy.

3. Respondents and methods

3.1. Study design

Both the DCE-WTP and RS-WTP analyses were used on the same sample (n = 4,009) of the UK adult population so that they could be directly comparable. Further, detailed information was obtained about respondent characteristics, respondent understanding of the issues, and other relevant respondent-related information. The survey was administered online under the direction of Laser Europe Limited.

Although we could have attempted to use a sample size calculation for the final DCE-WTP analysis [59], we did not consider this necessary because we knew, through analyzing a sample of 286 pre-pilot responses that all the attributes and levels were significant at the 1% level for analysis of data for the sample overall. Instead, because funding for large-scale DCE-WTP and RS-WTP was available, we only sought to obtain a sample of around 4,000 respondents overall for the final main analysis. Such a sample would be large enough to also facilitate substantive sub-group analysis.

Before survey respondents faced questions about their preferences using both DCE-WTP and RS-WTP methodologies, the survey respondents were given information about the nature of CLN2 disease and its implications. It stressed the fatal and neurodegenerative (especially if not appropriately treated) nature of the disease.

3.2. Survey instruments/experimental designs

3.2.1. DCE-WTP methodology

The DCE-WTP analysis for this project relating to ‘Survey instruments/Experimental designs’ can be explained by dividing it into constituent parts:

1. Selection of attributes and levels, and also specifying text for the DCE-WTP preamble: The role of ‘framing effects’ (the impact of the way in which DCE choices are presented upon the choices subsequently made) can be important for DCEs [60], and so considerable attention was paid to ensuring that attributes were appropriately framed during DCE development. The survey instruments together with the attributes and levels were tested and validated in a pre-pilot survey with 103 respondents, and then in a subsequent pilot survey with a further 286 respondents, sampled to be representative of the UK general population (for further details refer to Appendix 1, under the sub-heading ‘Selection of attributes and levels for DCE-WTP’).

   As CLN2 is an ultra-rare disease that many people will be unaware of it was necessary to ensure that the questionnaire preamble explained the condition and its impact upon children to help avoid respondents making assumptions or bringing outside information into the decision-making process [61]. The questionnaire preamble and the DCE attributes and levels were all subject to rigorous pre-pilot and pilot DCE analysis, before finalizing the main DCE-WTP survey.

• (2) Selection of levels of the monetary ‘tax’ attribute: It is difficult to know-how to select appropriate monetary levels for a monetary ‘tax’ attribute. However, as the appropriate tax levels are the unknown being estimated in the DCE-WTP analysis, it is unclear what levels should apply a priori. There is some evidence [44] that estimates of WTP can be sensitive to the levels and range assumed for the price levels used for the monetary attribute. Therefore, the pilot DCE analysis was used to establish whether adopting different levels for the price vector (in this case hypothecated tax) might affect estimated marginal willingness to pay [MWTP], and so informed guesses were used relating to realistic levels for this attribute. Given the considerable uncertainty surrounding appropriate levels for this attribute, two different sets of levels for the tax attribute for the pilot DCE-WTP analysis were tested. The ‘lower bound’ version were assigned monetary levels (£0, £25, £50, and £100) which were 50% of the levels for the ‘upper bound’ version (£0, £50, £100, £200).

• (3) Need for an ‘opt-out’ in the DCE-WTP survey: Unless respondents must consume the good/service, choice amongst hypothetical pairs may be problematic as it implicitly assumes respondents choose to consume the good/service [61]. Moreover, it has been argued [62] that by not including a nonparticipation alternative and forcing respondents to make a choice, this might mean respondents would not normally lower their initial level of utility (e.g. starting level of utility or status quo). This would violate the assumption underpinning Hicksian compensating measures [62] that welfare gain (loss) is obtained by analyzing shifts from the respondent’s initial level of utility. Therefore, an opt-out was included.

• (4) Experimental design for the pretest DCE-WTP survey: A ‘fractional factorial’ DCE design was used. This allows a sufficiently large sample of scenario choices from the ‘full factorial’ being selected so that the effects of interest can be estimated. At a minimum, these should include the main effects [61]. The SAS statistical package was used for this purpose, and a D-optimal main effects design used in other DCEs involving social preferences [40,63] was applied. D-optimal designs are employed to try to increase statistical power to determine the significance of attributes and their levels for a given number of DCE responses.

• (5) Selection of attributes and levels for the final DCE-WTP survey instrument: The pilot DCE results suggested that the attributes (and their respective levels) were appropriate, namely, ‘initial quality of life at treatment initiation;’ ‘treatment effect on child’s capabilities;’ and ‘extra years of child life expectancy.’ As a result, the same attributes and levels were used in the final version without revision. To obtain conservative estimates of WTP, the lower bound set of levels for the tax attribute (£0, £25, £50, and £100) were used.

• (6) Experimental design of the main DCE-WTP survey: This was conducted using the same approach adopted in (4) above.

• (7) Creating suitable questions relating to respondents’ socio-economic statuses and other relevant characteristics of the respondent sample: Information was required to assess the representativeness of the sample obtained for the DCE-WTP analysis, and so key information about respondents’ socio-economic characteristics and other relevant characteristics was gathered.

3.2.2. RS-WTP methodology

The key steps involved in conducting the RS-WTP analysis were similar to those used for DCE-WTP:

1. Selection of attributes and levels: The RS-WTP survey instrument was developed alongside the DCE-WTP instrument and tested and validated in a pre-pilot survey (103 participants) and subsequent pilot survey (286 respondents). The approach used for attribute and level selection was very similar to the approach adopted for DCE-WTP. The same attributes were relevant for RS-WTP analysis, except for the monetary tax attribute, which was not required for RS-WTP analysis. The starting point is that the respondent allocates a pre-allocated budget between two different services [35] unlike DCE-WTP where there is no fixed budget. This provides the basis for determining WTP with RS-WTP, rather than establishing the marginal rate of substitution between non-monetary and monetary attributes. The pre-allocated budget assumed for RS-WTP corresponds to the lower boundary valuation for the QALY threshold used by NICE [64]. As there is evidence that the proportion of the budget allocated between Service 1 and Service 2 may be insensitive to the size of the budget [35], it was not considered that there was a need to experiment with different pre-determined budgets in pilot analysis.

2. Experimental design for the pretest RS-WTP survey: Although previous RS-WTP designs [32–35,38] had not used a D-optimal design to produce an RS-WTP instrument, in the interest of rigor, both the pilot RS-WTP questionnaire and the final RS-WTP questionnaire used a D-optimal main effect design in SAS.

3. Selection of attributes and levels for the final RS-WTP survey instrument: After conducting the pilot RS-WTP analysis, the RS-WTP design did not require modifications before proceeding to the final RS-WTP.

4. Creating suitable questions relating to respondents’ socio-economic statuses and other relevant characteristics of the respondent sample: The RS-WTP survey was conducted at the same time as the DCE-WTP survey, and the same dataset about respondents’ socio-economic and other key relevant characteristics was therefore utilized to establish the representativeness of RS-WTP respondents.

3.2.3. Details of DCE-WTP instrument attributes and levels

An example of a DCE-WTP question is presented in Figure 1. Fuller details of the range of final attributes and levels for the DCE-WTP design are in Table 1.

Figure 1. Example DCE-WTP preamble and question.

Table 1. Details of DCE-WTP instrument attributes and levels

Attribute	Levels
Additional income tax (hypothecated for CLN2)	Levels of £0, £25, £50, or £100.
Increase in life expectancy as a consequence of treatment	Levels of 0, 2, 5, 10, 15, or 20 years.
Quality of life of affected child at treatment initiation	- Levels of: - Not affected - Badly controlled seizures - Frequent seizures - Controllable seizures
Treatment effect on child’s capabilities	- Levels of: - No effect - Yes—Slow down - Yes—Remain - Yes—Improved

3.2.4. Details of RS-WTP instrument attributes and levels

An example of RS-WTP question is presented in Figure 2. Fuller details of the range of final attributes and levels for the DCE-WTP design are detailed in Table 2.

Figure 2. Example RS-WTP preamble and question.

Figure 2. Continued.

Table 2. Details of RS-WTP instrument attributes and levels

Attribute	Levels
Increase in life expectancy as a consequence of treatment	- Levels of 0, 5, 10, or 20 years
Quality of life of affected child at treatment initiation	- Levels of: - Not affected - Badly controlled seizures - Frequent seizures - Controllable seizures
Treatment effect on child’s capabilities	- Levels of: - No effect - Yes—Slow down - Yes—Remain - Yes—Improved

3.2.5. Key differences in the design of DCE-WTP and RS-WTP instruments

As previously indicated in Point 1 relating to the RS-WTP methodology for RS-WTP (‘Selection of attributes and levels’), there is no need to include a monetary attribute. Instead, a pre-determined budget of £20,000 corresponding to the lower threshold valuation of a quality adjusted life year [QALY] by NICE was assumed. As there is some evidence that the proportion of the pre-determined budget allocated to both service A and service B in an RS-WTP may be linearly related to the size of the original budget allocation [35], and the focus is upon the proportion of the original budget allocated to Service A versus Service B, the size of the pre-determined budget allocation was not considered for the RS-WTP analysis. Another difference between the DCE-WTP and RS-WTP approaches was that the number of levels assigned to child life-expectancy was higher for the DCE-WTP analysis, which had six levels (0, 2, 5, 10, 15 or 20 years), compared to RS-WTP, which had only four levels (0, 5, 10, or 20 years) for the same attribute.

3.3. Econometric modeling

Appropriate approaches to select the econometric analyses were applied to both the DCE-WTP and RS-WTP data (for further details please refer to Appendix 1, under the sub-heading ‘DCE-WTP model selection’).

Also, presented within the same appendix, are full details of the econometric methodology deployed including a description of the underlying utility function for DCE-WTP and of the associated probability assumptions/distributions for the analysis that we executed using STATA using random parameter logit. The details of the equations deployed for the econometric analyses deployed for DCE-WTP are as follows:

Linear (model 1)

$U_{ij}={\beta }_{1}TAX+{\beta }_{2}L{E}_{\hspace{0.17em}}+{\beta }_{3}QUA{L}_{na}+{\beta }_{4}QUA{L}_{cs}+{\beta }_{5}QUA{L}_{fs}+{\beta }_{6}TREA{T}_{ysd}+{\beta }_{7}TREA{T}_{yr}+{\beta }_{8}TREA{T}_{Yi}+{\epsilon }_{ij}$

Although the linear random parameter model (model 1) was run as it was outperformed in terms of measures of goodness-of-fit applied (e.g. AIC, BIC, and pseudo log-likelihood) by a non-linear model (model 2), in the interests of brevity only the results of model 2 are presented within this paper. The definition of variables for all the models presented is as follows:

TAX: Relates to hypothecated taxation for CLN2 treatment via tax.

LE: Relates to extra years of life expectancy.

QUAL_na: Quality of Life ‘Not affected,’ relative to the base case of badly controlled seizures.

QUAL_cs: Quality of life affected by ‘Controlled Seizures,’ relative to the base case of badly controlled seizures.

QUAL_fs: Quality of life affected by ‘Frequent Seizures,’ relative to the base case of badly controlled seizures.

TREAT_ysd: With respect to a treatment effect, yes, there is a slowdown in impairment of child’s remaining capabilities (slowdown), relative to the base case of ‘no effect.’

TREAT_yr: With respect to treatment effect, yes, there is preservation of child’s remaining capabilities (remain), relative to the base case of ‘no effect.’

TREAT_yi: With respect to treatment effect, yes, there is preservation of child’s capabilities with a noticeable improvement in their affected capabilities (improved), relative to the base case of ‘no effect.’

Non-linear in Life expectancy (model 2)

$U_{ij}={\beta }_{1}TAX+{\beta }_2{\sqrt{LE}}_{\hspace{0.17em}}+{\beta }_{3}QUA{L}_{na}+{\beta }_{4}QUA{L}_{cs}+{\beta }_{5}QUA{L}_{fs}+{\beta }_{6}TREA{T}_{ysd}+{\beta }_{7}TREA{T}_{yr}+{\beta }_{8}TREA{T}_{yi}+{\epsilon }_{ij}$

Here, the variables are again as defined for model 1. For this non-linear model, the life expectancy variable relates to the squared root of life expectancy, which is the best fitting non-linear specification for this continuous variable. This non-linear functional form for the life expectancy attribute (squared root of life expectancy) implies that the utility associated with extra life expectancy remains positive but steadily falls, implying diminishing marginal utility, which fits well in terms of plausible behavioral health economics theory. For the DCE-WTP data, the delta method for calculating confidence intervals [CI] around estimates of MWTP was used.

3.3.1. RS-WTP

RS-WTP analysis of preferences (Model 3) is based on assuming a linear relationship between the amount of money allocated to Service 1 and the relevant attribute and level that describe different scenarios:

$Service{1}_i={\alpha }_{\hspace{0.17em}}+{\beta }_{1}L{E}_{\hspace{0.17em}}+{\beta }_{2}QUA{L}_{na}+{\beta }_{3}QUA{L}_{cs}+{\beta }_{4}QUA{L}_{fs}+{\beta }_{5}TREA{T}_{ysd}+{\beta }_{6}TREA{T}_{yr}+{\beta }_{7}TREA{T}_{yi}+{\epsilon }_{ij}$

The variables are as defined previously for Model 1.

With respect to the RS-WTP analysis, we used the package R, which provides linear 95% Confidence Intervals for the estimated variables.

4. Results

4.1. Descriptive statistics

For the final survey, a total of 4,009 responses from members of the UK adult population were obtained by Laser Europe Limited, who administered the computer-based questionnaire.

The data obtained as part of the questionnaire survey relating to respondent characteristics can be cross-referenced with data from UK population census data [65] to establish the representativeness of the sample relative to the UK population. Details of this are tabulated in Appendix 2 (Table A1). These data suggest that respondents are broadly representative of the UK population with respect to geographical location. The sample was not as representative as we would like with respect to gender, age, marital status, household composition, family type, and economic activity. There was a lack of appropriate data for comparison with respect to both respondent highest level of education and respondent annual salary.

4.2. DCE-WTP results

The DCE-WTP econometric results established that with respect to the monetary [TAX] attribute, a continuous linear variable was the optimal model, as determined by the lowest pseudo-log-likelihood, AIC, and BIC scores. However, the best performing DCE-WTP model in terms of these scores for the child life expectancy variable involves the square root of changes in the CLN2 child life expectancy variable.

Econometric results for this model are presented in Table A2 in Appendix 2, and estimates of MWTP for attributes derived from this model are shown in Table 3. All attribute-related variables for mean valuations showed the expected signs and were statistically significant at well below the 1% level. There is also evidence, from this parametric model, of statistically significant preference heterogeneity with respect to five of the eight attribute variables, including those relating to [TAX]; the square root of child life expectancy; two out of the three ‘quality of life’ attribute levels (‘Not Affected’ and ‘Controllable Seizures,’ but not ‘Frequent Seizures’); and one out of the three ‘Treatment Effect’ variables (‘Improved’ but not ‘Slowdown’ or ‘Remain’).

Table 3. Estimates of MWTP in £s—Whole sample using Random Parameter Logit including all responses except ‘unwilling to choose’ responses but including ‘indifference’ responses. Non-linear model using squared root of child life expectancy

Dependent variable.	Marginal Willingness to Pay [MWTP].	P>[z].	95% CI.	95% CI.
Child life expectancy square root.	£32/Square root of number of years of additional life expectancy.	<0.01
Quality of Life (compared to the base category of ‘Badly controlled seizures’)
Not Affected.	£187	<0.01	£166	£208
Controllable seizures.	£123	<0.01	£109	£137
Frequent Seizures.	£76	<0.01	£65	£86
Treatment effect (compared to the base category of ‘No effect’)
Slowdown.	£125	<0.01	£111	£139
Remain.	£149	<0.01	£132	£166
Improved.	£185	<0.01	£165	£205

The MWTP figures (see Table 3) can be summarized as follows.

Extra years of life expectancy are valued positively, but the MWTP value of an extra life-year decreases as life expectancy increases, as depicted in Figure 3.

Figure 3.. Non-linear decrease in MWTP in £5 for an additional life year with respect to life expectancy.

Improvement in initial CLN2 patient quality of life at treatment initiation is also positively valued relative to the base case of ‘badly controlled seizures:’ by an average of £76 for ‘frequent seizures;’ an average of £123 for controllable seizures; and an average of £187 for ‘not affected.’ These findings seem sensible because they indicate that as quality of life progressively improves following treatment initiation, and then society places an increasing value upon improvements, which is rational.

In relation to ‘treatments effects,’ compared to ‘no effect’ a slowdown in the impairment of an affected child’s remaining capabilities was positively valued at £125; preservation of the child’s remaining capabilities was positively valued at £149; and an improvement in the child’s remaining capabilities was valued at £185. Once again, these findings seem sensible because as the extent of treatment effects improves, the value society places upon the improvements increases.

4.3. RS-WTP results

We refer the reader to an important explanatory footnote under the RS-WTP results table (Table 4). The important point made in this footnote is that the absolute WTP figures for RS-WTP are a by-product of assuming a very low and arbitrary initial budget allocation of £20,000 per year for treatment of a child with CLN2. The figures for the implied percentage of an initial £20,000 budget are therefore of policy-making relevance, rather than the absolute WTP figures discussed here.

Table 4. RS-WTP Econometric results—WTP in £s for Service 1 (Imperfect Health)—Linear OLSContinuous Life Expectancy

Variable	Implied Percentage (%) of the budget/Estimated coefficient	Standard Error	Pr (>[t])	95% CI	95% CI
Intercept (constant) %	40.11%			39.08%	41.13%
Intercept (constant)	(£8,022)	104.500	<0.01	(£7,817)	(£8,227)
Child Life Expectancy – %	0.15%			0.10%	0.20%
Child Life Expectancy	(£30)	4.850	<0.01	(£21)	(£40)
Quality of Life
Not affected – %	5.60%			4.61%	6.61%
Not affected	(£1,121)	101.778	<0.01	(£922)	(£1,321)
Controllable Seizures – %	3.83%			2.83%	4.83%
Controllable Seizures	(£766)	101.778	<0.01	(£567)	(£966)
Frequent Seizures – %	1.42%			0.42%	2.41%
Frequent Seizures	(£284)	101.471	<0.01	(£85)	(£482)
Treatment Effect
Slowdown – %	1.84%			0.85%	2.84%
Slowdown	(£369)	101.622	<0.01	(£170)	(£569)
Remain – %	2.40%			1.41%	3.40%
Remain	(£481)	101.620	<0.01	(£282)	(£680)
Improved – %	2.79%			1.80%	3.78%
Improved	(£558)	101.467	<0.01	(£360)	(£757)
AIC	653,431
BIC	653,506
Pseudo – Log -Likelihood	−326,706

Table 4 footnote: The absolute regression estimates and confidence intervals in () within this table are a by-product of assuming a small £20,000 budget available for allocation. Since there is also evidence to suggest (Richardson et al 2014) that the value of coefficients may be linearly related to the size of the initial arbitrary budget allocation, the implied percentage figures are therefore arguably of more relevance.

The RS-WTP results in Table 4 indicate that the intercept (constant) and all attribute variables are highly significant at well below the 1% level.

The high positive value for the intercept of £8,022 suggests that over 40% (40.11%) of the £20,000 budget is allocated to respondents with imperfect CLN2-related health status in order to avoid certain death of the affected child. If the child avoids certain death, he or she will instead be in an imperfect health state (defined by differences in the attribute levels presented to respondents). This willingness to allocate a budget to the imperfect health state indicated by the intercept is unrelated to changes in extra life expectancy, CLN2 initial quality of life, or treatment effects (which are separately valued by the other attributes detailed below). The valuation of the intercept (equivalent to 40.11% of the overall £20,000 budget) therefore relates to the valuation placed upon CLN2-related mortality avoidance.

Each extra year of life expectancy is valued positively but at a low level of just £30 per year. This is only equivalent to about 0.15% of the initial budget allocation. So, there is the paradox that whilst respondents would devote just over 40% of the budget to avoid death, they would only allocate 0.15% of the budget for an extra year of life expectancy once death is avoided. Improvements in ‘quality of life at treatment initiation’ compared to the base case of ‘badly controlled seizures’ is also positively valued relative to the base case of ‘badly controlled seizures;’ by an average of £284 for ‘frequent seizures,’ equivalent to 1.42% of the initial budget allocation; an average of £766 or 3.83% of the initial budget allocation for ‘controllable seizures;’ and an average of £1,121 for ‘not affected,’ equivalent to 5.60% of the initial budget allocation. These findings, like the equivalent DCE-WTP findings, seem sensible because they show that as quality of life at treatment initiation progressively improves, society places an increasing valuation upon this.

In relation to ‘treatment effects,’ compared to ‘no effect’ a ‘slowdown’ in the impairment of an affected child’s remaining capabilities was positively valued, at £369, equivalent to 1.84% of the initial budget allocation; ‘preservation’ of the child’s remaining capabilities was positively valued at £481 or 2.4% of the initial budget allocation; and an improvement in the child’s remaining capabilities was valued at £558, equivalent to 2.79% of the initial budget allocation. These RS-WTP findings are in line with the findings obtained using DCE-WTP analysis because as the extent of ‘treatment effects’ improves, the value society places upon this also increases.

5. Discussion

Many of the objectives of this research have been achieved. The DCE-WTP estimates of MWTP have illustrated that members of society who responded to the survey are willing to pay a lot for defined improvements in CLN2 disease attributes. Additionally, the RS-WTP analysis has illustrated that members of society will devote a large proportion (just over 40%) of a budget allocation, which could be allocated to improvements in other attributes to treat children with CLN2 disease, to instead avoid immediate death.

5.1. Comparison of results using DCE-WTP and RS-WTP analysis

The relative weighting of levels in the attributes for CLN2-related quality of life and treatment effects are similar between the DCE-WTP and RS-WTP. The results of the DCE-WTP, which adopts a social (general public) perspective, suggest that DCE-WTP respondents have on average placed a very high monetary valuation upon improvements in attributes for CLN2 patients relating to extra life expectancy, higher CLN2-related quality of life, and better treatment effects. These results imply that an effective drug intervention, which can generate improvements in terms of the attributes defined in the DCE-WTP analysis may be highly valued by UK citizens.

With respect to the RS-WTP [32–35,38] valuations of the CLN2-related quality of life at treatment initiation and treatment effects attributes, the findings are broadly in line with those from the DCE-WTP analysis. With respect to the quality of life attribute, both the DCE-WTP and the RS-WTP analysis show that as quality of life improves (compared to the base case of ‘badly controlled seizures’), both the DCE-WTP and RS-WTP valuations imply a progressive increase in estimates of WTP attributable to them. Likewise, with respect to treatment effects (compared to the base case of ‘no effect’), as the extent of the treatment effect improves, once again both the DCE-WTP and RS-WTP valuations suggest a progressive increase in estimates of WTP. The fact that a broadly similar pattern of relative valuations for improvements in CLN2 attributes was obtained using both methodologies is reassuring. These findings, taken together, are supportive of the suggestion that society’s valuation of improvements in disease attributes for an ultra-orphan disease such as CLN2 may be considerable.

The differences observed in the percentage allocated to a life-saving intervention (40%) compared to the life extension (0.15% using RS-WTP, for an extra year of life expectancy) need some further attention. While the RS-WTP results may not be inconsistent with the DCE-WTP results, we may ask if the question was well understood or whether there is a true marked difference between WTP for salvage interventions, rather than for life-extending interventions. One possibility is that precisely because the life-saving intervention averts an otherwise certain outcome of death, respondents may consider this much higher priority compared to simply further extending the life of someone who is not necessarily in immediate danger of death.

However, this needs to be qualified by saying that both sets of WTP results may also suffer from upward bias due to insensitivity to size [66], which means that estimates of WTP may be insensitive to the size of health outcomes. These authors [66] compared estimates of WTP for different sizes of health-care goods both across samples and within samples. Their results showed no statistically significant differences in WTP for different-sized health effects. They claim this finding casts doubt on the reliability of estimates of WTP as a means of comparing social values for competing health-care programs. In fact, this finding may have the implication that the high valuation, which members of society seem to place upon treatments for rare diseases within this CLN2 DCE-WTP analysis may involve an overstatement of respondents’ true valuations. Such an overstatement may arise if respondents are incapable of adjusting their valuations appropriately to reflect the fact that few people will benefit (e.g. because the intervention involves the treatment of a rare or ultra-rare disease).

With respect to DCE-WTP, only MWTP figures are presented for the reasons discussed below. The estimates of actual MWTP values obtained using the DCE-WTP analysis relate to the average valuation per respondent for changes in attributes. This means that to obtain an overall valuation of SWTP for the UK, aggregation would be needed to take account of the total number of adults in the UK. In contrast, SWTP valuations obtained using RS-WTP assume an initial socially determined budget; this means that the valuations obtained already relate to the valuations of society overall, rather than just to individuals.

Aggregated estimates of SWTP obtained using RS-WTP depend upon the actual changes in attribute levels that might arise, comparing the situation without this new CLN2 treatment with the counterfactual in which attribute levels change for the 50 to 60 children in the UK with CLN2 disease because of improvements conferred by treatment. Also, RS-WTP—unlike DCE-WTP—places a valuation upon avoiding death.

Moreover, there are key differences in the elicitation settings comparing DCE-WTP and RS-WTP, which might impact upon the findings reached. The RS-WTP analysis, compared to DCE-WTP, implies a different starting point for the WTP analysis. An initial societal budget allocation for both Service 1 and Service 2 is assumed as part of the RS-WTP analysis. In the analysis presented here, an initial allocation of £20,000 was assumed. As this societal budget allocation is predefined, it is no longer appropriate to aggregate estimates of WTP according to the number of adults in the UK population in order to obtain an overall estimate of SWTP for society. Instead, average estimates of WTP obtained using RS-WTP analysis already provide estimates of SWTP for marginal changes in attributes, without any need to aggregate WTP across the overall population.

With RS-WTP, since there is evidence that estimates of WTP may be linearly related to the value of the initial budget constraint imposed [35], and since the initial £20,000 budget allocation was arbitrary, it therefore makes sense to instead look at the proportions of the original £20,000 allocation divided amongst the intercept and changes in attribute levels. Therefore, RS-WTP valuations are expressed in terms of the percentage of initial budget allocation they imply. As the RS-WTP scenario explicitly relates to a budget allocation that can be allocated wholly or in part to an individual to achieve imperfect health rather than death or wholly or in part to an individual to obtain perfect health, the budget allocation process for this analysis fails to acknowledge the fact that resources spent on CLN2 patients can only benefit a small proportion of the overall population. Therefore, the aforementioned concerns about the insensitivity of estimates of WTP to the size of the health outcome effect [66], may also be relevant to both DCE-WTP and RS-WTP estimates of WTP for CLN2 disease treatment.

5.2. Information relating to the interpretation of DCE-WTP results

Any attempt to use MWTP estimates obtained using DCE-WTP to estimate the overall SWTP is problematic [60]. In fact, there are very strong methodological reasons to suppose that any attempt to estimate overall SWTP from the DCE-WTP results would be invalid (for details refer to the section headed ‘Information relating to the interpretation of DCE-WTP results’ in Appendix 3).

For RS-WTP, one key methodological limitation is that because the initial budget assumed for allocation is arbitrary, this means that the actual estimates of SWTP obtained are a by-product of an initial arbitrary budget allocation. This is why, given that there is some evidence [35] to suggest that the proportion of the budget allocated to different attributes or to death aversion may be insensitive to changes in the size of the budget, there is a strong case for placing the emphasis upon the percentage of the budget allocated to different criteria.

Unfortunately, also for both DCE-WTP and RS-WTP, we cannot ascertain from our analyses whether the high valuations associated with improvements in attributes related to CLN2 disease are upwardly biased because respondents do not perceive scale effects [66] or alternatively whether these valuations reflect a properly informed decision for an ultra-rare disease.

5.3. Discussion of methodological limitations of approaches

A wide range of methodological limitations associated with the application of stated preference techniques have been identified in the literature (see Appendix 3 under the sub-heading ‘Discussion of methodological limitations of approaches’ for a detailed discussion). Problems that can be associated with simplifying heuristics [63,67] or attribute nonattendance [68–70] have been minimized in the DCE-WTP and RS-WTP analyses we have conducted here because both the DCE-WTP and RS-WTP designs were thoroughly piloted and involved a small number of attributes.

Concerns have also been raised that the way in which the attributes are described in DCE-WTP analyses can influence monetary valuations [71]. The pre-pilot and pilot exercises for this study were carefully considered, therefore, to ensure that attributes and levels were appropriate and that the clinical reality of CLN2 disease was presented accurately, with the view of minimizing uncertainty in interpretation by respondents. The confidence intervals of WTP point-estimates observed in this study—for both DCE-WTP and RS-WTP—are considered by the authors to be relatively narrow, suggesting that the approach taken was able to go some way in addressing these concerns.

Other concerns relating to monetary valuations for DCE-WTP relate to the fact that a functional form, which assumes that the marginal utility of income is constant, may be wrongly assumed [72] for DCE-WTP. However, for the DCE-WTP, non-linear functional forms for the monetary [TAX] attribute were tested, and we only assumed linearity when a range of non-linear functional forms seemed to be inferior (as demonstrated by log-likelihood, Akaike Information Criteria [AIC] and Bayesian Information Criteria [BIC]). Concerns relating to a monetary attribute do not apply to RS-WTP as it does not involve a monetary attribute.

Another concern relating to the monetary attribute for DCE-WTP is that respondents may react mainly to ‘the concept of non-zero payment and not to the magnitude of payments’ [73]. However, changes in the monetary attribute (for the monetary levels specified) were found to be linearly related to utility, indicating that respondents were reacting to the magnitude of tax payments and not just to the concept of a non-zero payment [73].

Concerns have also been raised that estimates of MWTP obtained using DCE-WTP might be highly sensitive to the range specified for the monetary attribute [44]. Therefore, the ‘lower bound’ range for the monetary income tax attribute was used rather than a ‘higher bound’ range, in the interests of generating ‘conservative’ rather than ‘exaggerated’ estimates of MWTP. Analogous to this for RS-WTP, we assume a relatively low pre-assigned budget of £20,000. Further, when interpreting the RS-WTP results, emphasis was placed on the percentage share of the initial budget allocation assigned to different objectives, as opposed to the actual nominal monetary amounts allocated.

Concerns have also been raised about the validity of DCE-WTP valuations with respect to health-risk reduction [70,74]. So, we circumvented the need for respondents to value risk by getting respondents to instead value certain states that would apply (for example, six different certain levels of CLN2 life expectancy for DCE-WTP), and a similar approach was adopted for RS-WTP.

The concerns expressed by Lancsar and Louviére [75], relating to DCE ‘rationality’ tests and their inherent ‘irrationality,’ were noted. Therefore, in line with the conclusions of this paper [75], dominance, non-satiation, or tests for lexicographic preferences were not included in either the DCE-WTP or RS-WTP analyses. However, some ‘test-retest’ tests [74,76] that may still be appropriate for DCE-WTP were applied. The results suggested a high degree of conformity between ‘test’ and ‘retest’ analyses for DCE-WTP, which was reassuring. It was not possible to conduct such a test-retest analysis for RS-WTP due to resource constraints, which is unfortunate as it is a new methodology and tests of its validity and reliability could be very informative.

6. Conclusions

A number of previous studies in Norway [77–79], Canada [55,80], and the United Kingdom [40,53] did not identify substantive additional value associated with disease rarity. One of the Norway-based studies involved a cross-sectional study of the Norwegian population (n = 1,547 respondents) [78]; another involved a trade-off analysis, again applied to the Norwegian population (n = 3,359) [79]; whilst the other involved a trade-off analysis survey [77] of doctors (n = 551) comparing results with a Norwegian population sample detailed in one of the author’s other papers [79]. The results of all three papers suggested very little evidence that respondents would value prioritizing medical care for those with rare diseases.

One Canadian study [55] used a DCE methodology in relation to rare/orphan diseases, using a convenience survey of university students (n = 213). The DCE included a cost-per-patient monetary attribute and a total budget attribute, alongside others. It concluded that, all things being equal, respondents were not willing to pay more per life-year gain for a rare disease than for a common disease [55]. The other Canadian study [80] used a larger sample (n = 2,005) of Canadian citizens. It particularly focused on the issue of whether there might be preference instability with respect to funding orphan drugs. The authors also raised the concern that some respondents might not accept the ‘zero-sum’ framework in which the opportunity cost of funding for rare diseases is often framed in terms of the opportunity foregone to use the funding potentially more productively to tackle common diseases. They therefore used this framework to see whether it might be associated with high levels of recorded indifference with respect to questions. The findings suggest that overall there appears to be a preference to fund common diseases, and this preference is greatest when the unequal cost of treating rare diseases is made explicit in trade-off scenarios. However, the authors [80] also reported that there was low engagement with the orphan-drugs funding issue, which led a significant proportion of respondents to avoid making a choice.

The United Kingdom studies [40,53] both applied choice experiments. The earlier study [53] was applied to a large society-based sample (n = 4,118 respondents), in which preferences were established by asking respondents to allocate fixed funds between different patient and disease types indicative of nine different prioritization criteria, which could be used by NICE (including special funding status for rare diseases). However, the findings of this DCE did not support the NICE approved special funding status of rare diseases [53]. A more recent survey [40] applied both the DCE and Patient Trade-Off [PTO] methods to both a stakeholder sample (comprising 14 informal caregivers, 16 health-care professionals, and 24 policymakers) and a large sample of the UK population (n = 3,950 respondents). The DCE considered the impact of rarity by specifying higher cost-per patient per year for rare diseases relative to common diseases, allowing consideration about respondent willingness to permit a cost-per patient ‘premium’ for rare diseases. The PTO similarly specified higher costs for the treatment of rare diseases. Having considered the results emerging as a result of the application of both analyses, the analysis concluded that the UK general public does not value rarity as a sufficient reason to justify additional funding for orphan/rare drugs [40].

The approach used in the current analyses is different. First, the DCE-WTP and RS-WTP questionnaire emphasized an application to a specific childhood disease (CLN2), which is an ultra-rare disease as opposed to just a rare disease (analyses [40,53,55,77–80]). It is possible that both the fact that the disease affects children and the fact that it is an ultra-rare disease (as opposed to just a rare disease) might mean that interventions for CLN2 are valued more than those for more ‘common’ rare diseases, and/or for those mainly affecting adults.

Second, the two preference elicitation techniques [DCE-WTP and RS-WTP] were applied after the questionnaire preamble had outlined the fact that respondents might place additional value upon interventions to treat an ultra-rare disease such as CLN2, as otherwise there would be a lack of incentives for firms to invest in research and development for such interventions (thereby emphasizing the case for special consideration for a CLN2 treatment). In short, the aim was to address concerns [80] that a lack of priming in some previous studies might have led to ill-informed preferences being expressed.

Third, both the DCE-WTP and RS-WTP analyses were applied in such a way that respondents were encouraged to allocate scarce resources (hypothecated income tax with respect to the DCE-WTP or parts of a finite-predefined budget for RS-WTP) in order to address health problems associated with CLN2 disease. In contrast to the earlier studies [40,53,55,77–80], an explicit attempt was made to establish willingness to pay for CLN2-related interventions. This approach represented a shift from approaches to date [40,53,55,77–80], which have explicitly tried to establish willingness to fund interventions for orphan diseases by considering the opportunity cost (framed in terms of the opportunity foregone to use the funding to treat other common diseases). With the DCE-WTP, the ability to fund treatment through hypothecated taxation was assumed, and therefore funding for CLN2 was not related to an opportunity cost framed in terms of alternative funding foregone. The RS-WTP analysis assumed a pre-assigned budget for CLN2 treatment, in which opportunity cost was central to decision-making but framed in terms of distributing a fixed CLN2 budget, either to patients who could avoid death in imperfect health or to a greater number of surviving patients experiencing perfect health. Therefore, opportunity cost in terms of opportunity foregone to fund common diseases was not considered either within the RS-WTP.

Both the DCE-WTP and RS-WTP analyses have been undertaken rigorously, taking on board a range of methodological issues, which are pertinent when applying these techniques to obtain social valuations relating to a rare or ultra-rare disease. A clear finding from the DCE-WTP results contained in this paper, is that in the context of CLN2 disease (an ultra-rare disease affecting children), members of society appear to value funding drugs highly, as measured by their MWTP, if medications positively affect the life expectancy and quality of life of affected children. This finding was obtained despite respondents being told there are only 50–60 children with CLN2 disease in the UK who might benefit. The results of the RS-WTP analysis suggest that, in the context of a disease like CLN2, which is currently incurable but which can be treated to improve life expectancy and quality of life, avoiding mortality and improving other disease-related attributes are both valued by members of society. Moreover, this significant valuation persists even when treatment for CLN2 disease cannot confer perfect health. Therefore, these two sets of results from the two different preference elicitation techniques are broadly consistent.

The limitations associated with the methodology do suggest a need for caution, to avoid an overly positive interpretation of these results. One concern is the fact that unlike many of the other studies that tried to establish the value to society or other stakeholder groups of ‘rarity’ [40,53,55,77–80], there was no comparison between prioritization for common diseases versus rare diseases. In these earlier analyses [40,53,55,77–80], such a comparison provided a benchmark against which to establish whether rarity ‘per se’ was valued. Since in our analyses the intervention for CLN2 disease was valued in isolation, there was no equivalent benchmark control group for the analysis. Also, precisely because the questionnaire presented the case for special consideration for this ultra-rare childhood disease, this might have made some respondents feel self-conscious when making responses, resulting in ‘social desirability’ or ‘warm glow’ upward bias in their expressed WTP [81]. The counter-argument to this view is that some of the earlier analyses might have been subject to downward bias with respect to the valuation of rarity for example, if analyses failed to set out the empirical reasons why therapies for rare diseases might be considered a special case before seeking valuations [80]. This could mean that some respondents might have been ignorant of the possible reasons for prioritizing therapies for rare diseases.

Another problem with the methodology, however, is that precisely because this analysis aimed to establish SWTP for an intervention to treat an ultra-rare disease, it may be subject to insensitivity to size bias [66]. Such bias will inevitably be more of a problem when valuing an intervention, which impacts upon a small number of people (for example, with an ultra-rare disease such as CLN2), especially if there is no within-analysis comparison relating to the scale of impact between a common disease versus a rare disease intervention, as was the case in our analyses here. It could be argued that the DCE-WTP used an unrealistic decision-making framework because when making decisions about funding for rare diseases, decisions are made with reference to the opportunity cost measured in terms of other healthcare provisions which must be foregone to fund it. It is not the case (as assumed in the DCE-WTP) that a separate hypothecated taxation funding stream for ultra-rare diseases such as CLN2 exists in the UK, which would facilitate additional funding for CLN2 disease without adversely impacting upon other forms of healthcare provision. Moreover, RS-WTP assumes a fixed budgetary allocation for the treatment of CLN2 disease, when inevitably such a budget allocation in the UK would be made with reference to other funding priorities across the NHS. The differences in methodological approaches may in part underpin the differences in our findings compared to the existing rarity literature, which tends to suggest that rarity may not be valued. However, it may also be that ultra-rare diseases such as CLN2 may be valued more than rare ones, especially if they affect children. This paper nonetheless makes a unique contribution to the health economics literature, in that it demonstrates the high valuation that members of society may place upon developing treatments for ultra-rare diseases affecting children, using an established health economics preference elicitation technique [DCE-WTP analysis]. This paper also illustrates—using innovative newly emerging RS-WTP methodology, which also specifically establishes respondent valuations associated with avoiding mortality—that members of UK society appear to value avoiding mortality in children with ultra-rare diseases, and value attributes improving their quality of life and health status.

To our knowledge, this is the first economic evaluation to deploy both DCE-WTP and RS-WTP methods simultaneously. Notably, methodologies can provide complementary insights in relation to SWTP for medical interventions to treat rare or ultra-rare diseases in children. We would argue that techniques such as DCE, DCE-WTP, and RS-WTP may be particularly valuable in terms of providing additional insights that may be required when evaluating rare or ultra-rare diseases affecting children.

Key issues

• The objective was to establish the value from the perspective of society of the benefits associated with a drug treatment for an extremely rare life-threatening neuro-degenerative disease affecting children.

• This disease is known as Neuronal Ceroid Lipofuscinosis type 2 (CLN2).

• Conventional health economics evaluation techniques (including CEA and Cost-Utility Analysis [CUA]) fail to adequately cater for the additional valuation that members of society might have for prioritizing patients suffering from such rare diseases affecting children.

• Therefore, our research employed two complementary preference elicitation techniques (Discrete Choice Experiment Willingness to Pay [DCE-WTP] analysis and Relative Social Willingness to Pay [RS-WTP] analysis), to value preferences for prioritizing the treatment of children with CLN2.

• Before undertaking the survey, the fatal and neurodegenerative (especially if not appropriately treated) nature of CLN2 disease was explained to survey respondents in a preamble to the questionnaire.

• Both valuation techniques were applied to a large sample (n = 4,009) of adult members of the United Kingdom population, in an attempt to establish potential Social Willingness to Pay [SWTP] for a drug treatment for CLN2.

• Considerable experimentation was undertaken with alternative econometric models, to arrive at ‘optimal’ econometric ‘Results’ models for both our DCE-WTP and our RS-WTP analyses contained within this paper.

• Due to the methodological limitations associated with aggregating SWTP for a DCE-WTP, when a DCE-WTP analysis incorporates an ‘opt-out’ option, it was impossible to obtain a robust overall estimate of SWTP from the DCE-WTP data.

• Nonetheless, the findings obtained from our ‘optimal’ DCE-WTP econometric model indicate that on average respondents are willing to pay high amounts (in terms of additional hypothecated taxation for CLN2) for defined improvements in CLN2 attributes.

• Absolute estimates of SWTP obtained using RS-WTP, have the limitation that estimated SWTP may be linearly related to the arbitrary size of the initial hypothetical budget allocation that respondents are asked to assign to either Service 1 [saving a life] or Service 2 [Improving health status]. Therefore, absolute estimates of SWTP obtained using RS-WTP are not informative.

• However, the average estimates of the relative percentage share of SWTP obtained using the ‘optimal’ RS-WTP model, imply a willingness to allocate just over 40% of a pre-assigned health-care budget to prevent child mortality, and approximately 15%, for improvements in health status.

• In light of the acknowledged methodological limitations of these methods, we tentatively conclude that societal willingness to pay for CLN2 treatment may be high.

• This analysis therefore indicates a gap between United Kingdom citizens' willingness to pay for interventions to treat rare life-threatening childhood diseases, and the valuations obtained using conventional techniques of health economic evaluation, which do not consider rarity.

Declaration of interest

D Moro has declared that he has received funding (originally from BioMarin, via Analytica Laser and Certara) to work on the design, data analysis, and interpretation of the DCE-WTP and RS-WTP findings detailed in this paper, and upon the production of this paper. Also, some CLN2 related work-related travel expenditure when attending meetings in London and Frankfurt has been covered by funding emanating from BioMarin.

M Schlander has declared that his institution (InnoVal^HC) received funding for research projects under an unrestricted educational grant policy for providing consultancy for the present project and for related projects, by BioMarin and Sanofi/Genzyme. His institution has also been receiving unrestricted educational grants from a number of biopharmaceutical companies, including Amgen, BIT Pharma, Celgene, Galenica, Johnson & Johnson (J&J), Novartis, Novocure, Roche, and Vertex, as well as from government, professional, industry, and payer organizations including BPI, curafutura, FAMH, IQWiG, Interpharma, and santésuisse. He also holds shares in J&J (unchanged since 1999).

H Telser has declared that his institution (Polynomics) received consultancy payments from BioMarin relating to this submitted work, this was for sitting on the expert advisory board for designing and conducting the discrete choice experiment for this study.

O Sola-Morales has declared receipt of payments from BioMarin relating to the preparation of this manuscript, and for other purposes not directly related to this matter, but which could be considered as potentially influential.

M Clark has declared he has received funding (originally from BioMarin, but paid via Apple Education Limited) for his work upon interpretation of data for this project, and for his input into this publication. Also, some CLN2 related work-related travel expenditure (traveling to work for work on this project) has been covered by funding emanating from BioMarin.

A Olaye was a salaried employee and shareholder of BioMarin when this study was undertaken.

C Camp, M Jain, and T Butt are salaried employees and shareholders of BioMarin, who have contributed to this CLN2 project and this paper whilst working for BioMarin.

S Bakshi who works for Certara Evidence & Access (EvA) disclosed that Certara EvA (previously Analytica Laser) has received financial support in the form of Consultancy payments from BioMarin toward the design and implementation of the study detailed in this paper and for medical writing support.

The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Authors contributions

D Moro, A Olaye, M Jain, T Butt, and S Bakshi were involved in the conception and design of the study. M Clark was involved in pinpointing key methodological literature to assist D Moro with study design. D Moro, A Olaye, T Butt, and S Bakshi were involved in the analysis of the study data. D Moro, M Schlander, H Telser, O Sola-Morales, M Clark, A Olaye, C Camp, M Jain, T Butt, and S Bakshi were involved in the interpretation of the data. All authors were involved in revising the paper critically for intellectual content, and agreed for the final version of the manuscript to be published.

A statement of originality of this research

This paper has been submitted for consideration for publication as an ‘Original Research’ paper within the journal ‘Expert Review of Pharmacoeconomics & Outcomes Research.’ The research underpinning this paper can lay legitimate claims to originality in a number of respects.

First, it relates to new primary research that the co-authors of this paper have been involved in undertaking, which elicits the preferences and monetary valuations of United Kingdom members of the general population relating to the value of treating a very rare disease affecting children (Neuronal Ceroid Lipofuscinosis type 2 - CLN2).

Second, to our knowledge, this is the first joint application of both Discrete Choice Experiment Willingness to Pay [DCE-WTP], and also Relative Social Willingness to Pay [RS-WTP] analysis that has ever been applied to the same sample of respondents, to value the same condition (in this case CLN2).

Third, we have conducted this experimental research (which applied both DCE-WTP alongside RS-WTP analysis) because of our perception that most other conventional techniques for health economics evaluation are ill-equipped to assess any additional value that members of society might place upon treatments targeted at rare conditions. A key issue that this paper therefore addresses is that if you develop a new treatment for a rare disease, by implication because the potential market for such an intervention is very limited, this may provide a disincentive for commercial organizations to develop drugs to treat rare diseases such as CLN2. This would be the case unless commercial organizations could charge sufficiently high prices to reimburse them for the research and development costs incurred. This novel research was therefore conducted to establish whether members of the general population might be willing to fund higher prices for products that treat rare life-threatening diseases affecting children (such as CLN2), through hypothecated taxation.

Finally, our RS-WTP analysis to our knowledge was the first to use an appropriate main effect design (D-optimal RS-WTP design for RS-WTP attributes) to facilitate assessment of the impact of changes in RS-WTP attribute levels upon econometric estimates obtained using the RS-WTP technique. Therefore, the application of multivariate analysis for RS-WTP to ascertain the impact of changing attribute levels upon allocations to the imperfect CLN2 health state contained in this paper is underpinned by an appropriate underlying RS-WTP questionnaire design.

Complying with research ethics requirements

This study was conducted using a sample of the UK general public who did not suffer from CLN2. The issue of whether UK National Health Service ethical approval was required was considered before this research was conducted. However, it was concluded that NHS ethical approval was not required. This was because it was a survey of general population preferences, and our sample did not contain CLN2 patients, CLN2 carers, or prisoners. Moreover, our enquiries indicated that no other forms of approval were needed for this UK-based study.

Data deposition

As indicated above, this is ‘Not applicable.’

Reviewer disclosures

Peer reviewers in this manuscript have no relevant financial or other relationships to disclose.

Data availability statement

As the underlying preference data cannot be utilized without elaborate econometric modeling, and without providing considerable time-consuming explanations of the data, the underlying data supporting this analysis is not being made publicly available alongside this publication.

However, in principle if peer reviewers or others request access to the data within 2 years of the date of first publication, then we may be prepared to facilitate access to data relating to the actual RS-WTP or DCE-WTP analyses, which are outlined in detail within this paper. Any requested disclosures that we could facilitate, must, however, be in line with any relevant legislation relating to data protection (e.g. including the United Kingdom General Data Protection Act 2018).

Supplemental data

Supplemental data for this article can be accessed here.

Additional information

Funding

This research and the production of this manuscript was funded by a pharmaceutical company (BioMarin) who have developed and marketed a drug to treat CLN2.