Incorporating public and wider stakeholder views in the design of health state valuation studies in adults and young people: an undervalued resource?

KEYWORDS:

• Health state valuation
• health technology assessment
• patient and public involvement
• preferences
• QALY
• qualitative research
• stakeholder engagement

1. Introduction

Health state valuation (e.g. calculating numerical values representing people’s relative preference for living in particular health states) is an essential ingredient for health technology assessment (HTA). It allows for the generation of utilities to calculate health-related quality of life outcomes and adds the ‘QA’ to the QALY, or quality-adjusted life year [1]. For HTA, valuing health states uses standardized methodologies, such as time trade-off (TTO) or discrete choice experiment (DCE) with duration, with a targeted sample of people to derive their preference (or utility) weights for living in particular states of health relative to preference for survival or life expectancy. The sample used for valuing health states differs depending on locality and HTA body, but in the UK a sample of the adult (taxpaying) public is recruited, representative on key demographic criteria, such as age, sex, ethnicity, and socioeconomic status [2].

Elsewhere in health research, the endorsement and use of patient and public involvement (PPI) has been flourishing over the past two decades, but uptake in health economics has arguably been slower [3,4]. PPI describes ‘an active partnership between patients and the public and researchers in the research process, rather than the use of people as “subjects” of research. Patient and public involvement in research is often defined as doing research “with” or “by” people who use services rather than “to,” “about” or “for” them’ [5]. More broadly, holistic stakeholder engagement in research can incorporate a range of individuals who have a stake in the research process or outputs, as defined, for example, by the ‘7Ps Framework’ as: patients and the public (including informal carers and advocacy organizations), providers (of care/services), purchasers, payers, policy makers, product makers, and principal investigators (researchers and funders) [6].

Complementary to, but by no means in place of, collaborative involvement and engagement with stakeholders, qualitative research into the acceptability and design of methods can systematically elicit public and wider stakeholder views on key methodological decisions made in the research process. An important distinction here is that views are elicited from stakeholders as research participants as part of a structured research exercise and not as equal collaborators based on a peer-to-peer relationship with stakeholders, as in PPI [3].

In health state valuation there has been some published evidence of PPI and wider stakeholder involvement, but this appears muted relative to the large production of value sets (e.g. for commonly used preference-based measures, such as EQ-5D, in many different countries). Further, published qualitative research eliciting public and wider stakeholder views on valuation methods is relatively limited. In this editorial, I discuss the issue of incorporating public and wider stakeholder views in health state valuation, both through dedicated PPI provision and through qualitative research studies. Here, I am concerned with the valuation process specifically and not the generation of the outcome measures or descriptions of health states that are valued, such as vignettes, where involvement of stakeholders is commonplace [7]. First, I review some impactful examples of stakeholder involvement in health state valuation projects and emerging qualitative research in the field. Second, I discuss why public and wider stakeholder input into methods research may have been relatively neglected in the field of health state valuation. Third, I present some of the ways greater public and wider stakeholder input in the design of health state valuation studies may benefit the field.

2. Incorporating public and stakeholder views in the design of health state valuation studies

2.1. How have public and stakeholder views been considered in the design of health state valuation methods?

Impactful examples of different forms of PPI and wider stakeholder involvement can be found in published health state valuation work (i.e. treating members of the public or other stakeholders as equal research collaborators rather than research participants). In one notable project, an expert panel of patients with multiple sclerosis was formed to help assess implausibility in health state combinations from a condition-specific preference based measure to enhance the design and validity of a valuation task [8]. In a 4-year research project designed to develop methods to facilitate the inclusion of informal carer outcomes in economic evaluation, a PPI group was instrumental in informing study methods. This included choosing person trade off rather than DCE methods and helping to revise instructions, graphical methods, worked examples, and scale and duration of benefit for the task [3]. A final example is an interaction with service user members of the UK National Institute for Health and Care Excellence (NICE) Public Involvement Programme, whereby input was solicited on the use of the EQ Valuation Technology (EQ-VT) protocol [9] to value the novel EQ Health and Wellbeing (EQ-HWB) instrument [10]. Amongst other insights derived was that the ‘wheelchair example’ (used at the start of the EQ-VT protocol) was not deemed by service users as appropriate for use [11]. In the absence of stakeholder consultation, relatively simple design issues such as this may persist unchecked.

Published research qualitatively soliciting public and/or stakeholder views on the methods of health state valuation is more commonplace than instances of collaborative involvement and engagement, yet most studies of this nature are a relatively recent phenomenon. Qualitative methods have been used to help better understand how participants make decisions in health state valuations (e.g. [12–16]). The focus in these studies has typically been how or why participants make the valuation decisions they do, with their views on the suitability or design of the valuation methods themselves often a subsidiary issue. Nevertheless, some important methodological insights have been observed in such studies, including hypothetical health states (including ‘full health’) being too difficult to imagine for members of the public, incompatible or unrealistic health states being problematic to interpret, or other aspects of TTO being too complex or confusing (e.g. [13,16]).

More recently, an emerging set of qualitative studies has explored public views on normative issues in the design of health state valuation exercises, including valuing hypothetical versus experienced health states [17] and issues of who should be asked and/or who should be thought about when valuing health in young people [18–20]. Accepted methods to value health in young people are less well developed than for adults and this represents a particularly fertile area for incorporating public and stakeholder views, including those of young people themselves. To the author’s knowledge, the above mentioned research represents the first time that members of the public have been consulted for their views on normative issues in health state valuation and the findings have been published.

Lastly, greater engagement with stakeholders in the HTA process has recently been recommended by the EuroQol Research Foundation as part of future EQ-5D-Y (EQ-5D youth version) valuation studies, allowing for consultation on methodological and normative issues at a local level [21]. However, it was recognized that further guidance on how this process should be done and with whom is required. One of the first examples published from this process in the US was a piece of qualitative research, reporting on a 14-stakeholder roundtable exercise (featuring industry and HTA representatives, researchers, and two lay people experience caring for or working with children) [22]. Interesting insights were derived from this exercise that appear contrary to the recommended approach in the EQ-5D-Y-3 L valuation protocol [23], including support for eliciting preferences from adolescents rather than exclusively (taxpaying) adults and concerns over the ’10-year-old child’ perspective that was advocated.

2.2. Why have the public and wider stakeholders not been more involved in methodological decisions in health state valuation research?

While the relative paucity and recency in the examples of public and stakeholder involvement in health state valuation methods cited above are not proof that it is not happening (and just not being reported in publications), it is indicative of the historically low value placed on this kind of activity. A multitude of reasons may exist for why views and input from the public and wider stakeholders have not historically had greater transparent input into methodological/design decisions in health state valuation research, but at least three are pertinent: complexity, rarity, and efficacy.

First, health state valuation for HTA is a complex process and necessitates deriving a relative value (or quantified preference) for living in different (often complicated) states of health over duration of life. Participants often find health state valuation tasks with a duration element difficult to understand [13,16], and it could be argued that engaging with the design and methods of such tasks is a less accessible medium for people without an expert subject knowledge. Indeed, at the recent EuroQol workshop on valuing EQ-5D-Y where greater stakeholder engagement was recommended, there was a tendency to favor engagement with HTA bodies ‘who are likely to be the best informed and have internal processes for dealing with methods issues’ [21]. Accordingly, informing lay stakeholders sufficiently so they understand the basic underlying economic theory, goal(s) of health state valuation, and existing methods issues could be assumed to be resource-intensive, difficult to achieve, and impractical. Critics of such an argument may point to projects where informing lay people and soliciting their input has had notable success [3,8], that good quality stakeholder involvement often involves collaborators deepening their knowledge over a period of time [3], and that the onus is on researchers to develop best practice guidance and approaches on how stakeholder involvement in health state valuation can be best facilitated [24].

Second, the involvement of the public and wider stakeholders has not been commonplace in the discipline of valuing health, yet it has an established role in the HTA process more broadly [25]. HTA bodies have not mandated patient and public or broader stakeholder involvement in health state valuation and no real precedent has been set on this as a meaningful or customary activity in health state valuation research. This is despite HTA bodies, such as NICE, having established patient and public involvement policies in their work [26].

Third, the efficacy or cost-benefit of involving public and wider stakeholder views may not be clear to health state valuation researchers [27]. For researchers to want to engage in this type of activity it either needs to be mandated, or the benefits need to be clear to them. This is related to the complexity of the task and the extent to which stakeholders can be perceived to offer meaningful insights into methodological decisions that are restricted by the goal of the exercise. For example, how can the fundamental design of a DCE with duration or TTO be changed based on stakeholder feedback? Nevertheless, there are different examples of methodological decisions that have been meaningfully informed by soliciting public and wider stakeholder views as attested above [3,8].

2.3. How would health state valuation studies benefit from greater public and stakeholder involvement in methodological decisions?

Better incorporating the views of the public and wider stakeholders in the design of health state valuation studies is likely to be beneficial for the field in a number of ways. First, there is a moral and political argument for incorporating the views and preferences of the public in health research and health state valuation methods are no exception. This is represented in the right of citizens to have a say in publically funded health research and the ethical right of people to have a say in health research that affects or is being done to them [28]. This may be considered analogous to the ‘taxpayer argument’ for prioritizing the preferences of the adult general public when valuing health states for economic evaluation in the UK; it is not too much of a stretch to suggest that we should also consult their perspectives and opinions on how valuation research is undertaken [27], including normative issues, for example, of who to ask to value health states.

Second, there is growing evidence that incorporating PPI and the views of wider stakeholders in health economics improves the quality of study design and thus is likely to enhance data quality and validity [3,27,29]. In the examples above, methodological decisions were informed by stakeholder views to improve study design in a number of areas, including choice of method and how it was presented [3], and eliminating implausible health state combinations [8]. This helps to ensure the valuation task is understood as intended by participants and thus is likely to help improve the ultimate validity of data collected (i.e. the elicitation of genuine preferences).

Third, greater public and stakeholder involvement is likely to lead to greater investment in the research process from those who may be affected by and/or who are ‘end-users’ of the research. There is evidence that PPI can enhance recruitment strategies and study enrollment, and also can improve the dissemination and communication of findings to others [30]. Greater stakeholder collaboration is also likely to enhance study impact and use, and the accessibility and relevance of study outcomes to key stakeholders [31]. Doing so reduces the potential for research waste and lack of impact (e.g. due to limitations in design, reporting, or dissemination [32]). For example, by collaborating with the ‘end-users’ of value sets, including representatives of HTA bodies, there is a greater likelihood that local stakeholder requirements and principles will be embedded in valuation study design and thus the outputs will be more appealing to the stakeholders involved [21]. The degree of relative autonomy HTA bodies should afford as public institutions to decide on what criteria and methods HTA should operate is out of scope of this Editorial. However, most HTA bodies, such as NICE, typically consult with patients and the public as part of their decision-making processes [33].

Finally, consultation and incorporation of public and wider stakeholders’ views in health state valuation methods is likely to be particularly valuable in areas where methods guidance and normative conventions are not yet well established. This is particularly pertinent in valuing health states in young people, where the methods are evolving, and significant normative decisions exist, including who to ask and what perspective they should take (when completing the valuation task) [34]. Establishing the opinions and normative position of the wider public, including young people themselves, is likely to be of use for researchers and wider stakeholders (such as HTA bodies) to come to a consensus viewpoint on which approach to recommend in this area. The valuation protocol of the EQ-5D-Y-3 L [23] included normative methodological decisions that were taken in the absence of stakeholder involvement; articles are now emerging that suggest wider stakeholders may disagree with some of the normative positions adopted [22].

3. Conclusions

Published evidence of incorporating public and wider stakeholder views in the design of health state valuation studies is, at present, a relatively limited and recent phenomenon. Yet, promising examples exist and there is evidence to suggest PPI and related activities can make a meaningful contribution to health state valuation research projects. While there are challenges to stakeholder engagement in this field, such as the resource required and complexity of the tasks (the cost-benefit of such activities has yet to be formally established [35]), there is a moral and political argument for doing more of it, as well as clear emerging benefits for individual studies and the field as a whole. As the evidence base in this area continues to expand, the field would benefit from the generation of best practice guidance and principles for when, how, and with whom health state valuation research can best benefit from public and wider stakeholder involvement. Only then will we begin to make systematic progress in the field benefiting from what has been arguably a historically undervalued resource.

Declaration of interest

P Powell has received prior funding from the EuroQol Research Foundation for a separate project reviewing stakeholder engagement in health state valuation.

The author has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

The author would like to thank Professor Donna Rowen for her comments on an earlier version of this Editorial.

Additional information

Funding

This paper was not funded.