Measurement and valuation of the attributes of innovation of healthcare technologies: a systematic review

Abstract

Objective

Innovative technologies (e.g. treatments) play a pivotal role in improving patient’s well-being and in consequence population health outcomes. However, there is lack of consensus and comprehensive summary what constitutes innovation. Additionally, valuing them using traditional cost-effectiveness analysis is unlikely to capture the full range of benefits of these innovative technologies. This review aims to understand how innovation attributes were measured and/or valued in healthcare.

Materials and methods

We systemically searched four databases, PubMed, Embase, PsycINFO, and Econlit, from inception to April 2021. Studies were included if they measured and/or valued the attributes of innovation for healthcare identified in our previous systematic review. Any other potential recommended methods to measure and/or value the innovation attributes were also extracted.

Results

Of 546 articles, a total of 17 articles were finally included in this review. If attributes were measured and traded-off relative to costs, then it was considered as valuation of those attributes. Two specific attributes of innovation, i.e. substantial benefits and convenience and/or adherence were measured using adherence rate and life year or QALY gain. When innovation attribute was non-specific it was described as “overall innovation” and measured using overall innovativeness scale (e.g. point/binary scale). QALY-based cost-effectiveness analysis (CEA) was commonly used to assess and value substantial benefit attribute. Other valuation approaches were (i) rating, (ii) the economic value of life year gain, (iii) multiple criteria decision analysis (MCDA), (iv) incremental net health benefit (INHB), and (v) quality-adjusted cost of care (QACC). ICER threshold adjustment and multiple-criteria decision analysis (MCDA) are two common recommended approaches to capture the innovation comprehensively. We found that MCDA approaches often promoted and discussed but were sub-optimally used to incorporate different value attributes into decision-making.

Conclusions

Existing methods used by payers to measure and value the innovation component of a new product do not reflect the full range of health and cost impacts. They generally do not consider the alternative perspectives of patients, providers, caregivers, and society. Key challenges remain to appropriately measure and value innovation attributes and incorporate them into HTA decision making.

Keywords:

• Innovation
• value of innovation
• measuring innovation
• systematic review

This article is related to:

Characterizing attributes of innovation of technologies for healthcare: a systematic review

Introduction

Innovative technologies (e.g. treatments) play an important role in healthcare. Under limited healthcare resources, the value they bring to patients, carers, health care systems and society as a whole has to be appropriately assessed as they tend to be costly. A multitude of complex factors are involved for this phenomenon including, high research and development costs bore by the manufacturers and the exclusive right on their product, which provides investment incentives and encourages them to reinvest in future innovation^1,2. Policymakers are under growing pressure to ensure access to innovative drugs at a reasonable price as well as to determine the actual value of the drugs³. Some important innovation attributes including Quality of Life, Life years gained are frequently considered when assessing the value of these treatments. Cost-Effectiveness Analyses (CEA) tend to rely on Quality Adjusted Life Years (QALYs) as measure of benefit. A QALY attempts to combine life years gained and Quality of Life into a single unit. The ISPOR Special Task Force (STF) has recommended to U.S. public and private payers to use the cost per quality adjusted life-year (QALY) metric as a starting point, since this measure cannot capture full-scale benefit^4–6. They identified other value elements beyond QALY with proposing possible valuation methods on how to incorporate these elements in healthcare decision making⁷. For example, the value of hope is crucial and highly relevant for those individuals eager for the arrival of innovative technology with long survival promises. Another element, real option value proposed that, Individuals might be benefitted from innovation will be unlocked in the future by extending their survival⁸. For example, the incremental QALYs gain increased by 6–8%, and the ICER decreased by 0%–2%, after accounting for real option value that arises from technology advancement in treating metastatic melanoma with ipilimumab⁹. The concept of "scientific spillovers" has been recognized as a way for the first drug to lay the groundwork for further innovations and impact future generations of patients⁸. The recent COVID-19 pandemic has provided a great example of this phenomenon, where the years of research on mRNA vaccines laid the groundwork for the lightning-fast development of COVID-19 vaccines^10–12. Existing value assessment frameworks (VAF), such as ASCO framework, NCCN guidelines, and ICER value framework, have largely overlooked the value of scientific novelty. In addition, the ISPOR STF has acknowledged that their proposed elements are not the exhaustive list and omitted other elements such as “family spillovers,” which includes caregiver quality of life (QOL) and “absence of alternative effective therapies or unmet need”¹³. Without assessing these elements, the full value of this new treatment approach could be underestimated. These less recognized elements can promote well-being and aggregate more value through expanded, appropriate utilization at the population level⁸. However, the characteristics of innovative technologies, particularly new medicines, has never identified and valued.

In our previous systematic review¹⁴, eight attributes were conceptualized to describe innovation based on similar concepts or characteristics, definitions, or features of innovation from the included studies (details in Table 1). These eight innovation attributes, namely, (i) novelty, (ii) step change, (iii) substantial benefits, (iv) an improvement over existing technologies, (v) convenience and/or adherence, (vi) added value, (vii) acceptable cost, and (viii) uncounted benefits, deserve to be valued to reflect the true value of the innovative technologies. The newly approved drug, first-in-class therapy, was counted as a novelty. Drugs that brought radical changes in the current standard medical practice or introduced breakthrough technology were considered a step change. Substantial benefit is the attribute of innovation that might overlap with novelty and step change; in developing this attribute, we relied on a clear statement from the included articles regarding drugs’ substantial impacts on disease (such as substantial and positive impact on survival or quality of life). An improvement over existing technologies was referred to as the new drug that did not necessarily provide novel or “ground-breaking” benefits but rather a modification and improvement over an existing drug. Less dosage, administration frequency, and aid to enhance adherence of medical products were considered a convenience and/or adherence attribute of innovation. Alleviating social burden in a concerning disease, improving the population’s health, or reducing inequalities in healthcare would be added value to the healthcare system and society overall. Delivering better healthcare with cost-effective treatment was considered an acceptable cost attribute of innovation. Finally, the demonstrable and distinctive nature of technology not already captured in the technology’s economic evaluation as the HTA bodies tends to incorporate only healthcare perspectives was referred to as uncounted benefits of innovation. However, the valuation of these attributes was never identified. Also, it would be interesting to explore how these innovation attributes were measured since they were newly described and might not be previously valued. The measurements of these attributes would guide their future valuations. Thus, in this systematic review, we aimed to understand how innovation characteristics identified in the previous systematic literature review were measured or valued.

Table 1. Innovation attributes related to healthcare.

Attributes	Descriptions
Novelty	Novelty was the most stated attribute in describing innovation in healthcare. Novelty can address different healthcare demands, including first-in-class drug,^15–18 new approach to solve emerging and persistent health issue,^19^,^20 or something new/novel to fulfill the unmet^21–26. Broadly, any novel drug, products, services, and clinical practices that introduced over time into the provision of healthcare is a novelty^27^,^28.
Step change	Step change is a feature used to describe innovation. Technologies that produce lifesaving effects^29^,^30 or immediately convincing^31 were referred to as Step change. Instead of incremental improvement, technologies with breakthrough effect^32–34 that cure^35, prevent^35, or changes the course of disease states^36 were considered to offer a step change in terms of outcomes of patients^37.
Substantial benefit	As both novelty and step change attributes might potentially overlap with substantial benefit, so only those definitions that directly claimed a technology bringing significant, substantial, or considerable positive impact were considered under this attribute. The technology resulting in a substantial increase in overall survival,^38 substantial contributions to patient safety or quality,^39^,^40 large and positive impact on survival,^41 health and quality of life^42 were few examples found in works of literature that counted as substantial benefit attribute of innovation in this analysis.
Improvement over existing technologies	Exhibiting clinically relevant therapeutic advantages over the existing therapies was another attribute of innovative technologies^43. Compared to existing technologies, improvement in efficacy, effectiveness, safety, bioavailability, and existing products’ quality was referred to as improvement over existing technologies^20^,^44–48. Another aspect of this attribute was to reduce morbidity in the sick state, increase longevity and improve quality of life^49–51.
Convenience and/or adherence	Convenience, which is closely tied with adherence, are identified in several works of literature as innovation attributes. Optimal route,^19^,^52 decrease the dosage,^53 less frequent dosing (once daily or once weekly depending on conditions),^54^,^55 the capability of simplifying the multidrug administration along with maintaining appropriate drug release,^56 or improved palatability^57 were viewed as a convenience and/or adherence attribute of innovation. In the case of medical devices, portability was recognized as an essential feature of convenience that resulted in widespread acceptance^45^,^46.
Uncounted benefits	Innovative technology that produces a demonstrable and distinct benefit that may not have been adequately captured in the technology’s ICER calculation,^58 NICE demarcated these unnoticed and unrecognized benefits as uncounted benefits of innovation.
Acceptable cost	Several works of literature have recognized and hoped that the innovation in health care will offer better treatment, better patient care at a reduced cost or at least at the same cost compared to existing technologies^15^,^24^,^59–61. It was reported innovation at reduced cost would improve access to technologies^37^,^59.
Added value	Another feature of healthcare innovation is added value, which means innovative technology is expected to alleviate patient/social burden in the concerned disease^62. Value can be added to the system through several ways; increased productivity and knowledge spillovers (other due to innovation will produce indirect societal benefit^15 or innovative delivery system that allows a more decentralized health system to improve healthcare access and reduce health disparities are examples of added value^36^,^55^,^63.

Methods

This systematic review was reported following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines⁶⁴, following a predetermined protocol (PROSPERO 2021 CRD42021233799).

Searching strategy

To capture peer-reviewed articles that measured and/or valued the innovation attributes, we performed a systematic search in four different bibliographic databases, PubMed, Embase, PsycINFO, and Econlit, from inception to July 2022. Our search keywords were related to innovation, attributes of innovation, and measurement to comprehensively identify any articles that measured and/or valued innovation in healthcare. Detailed search strategies are provided in Supplement 1. In addition to systematic search, reference lists of eligible studies were hand-searched to locate additional relevant studies that might not be captured through direct search. All identified publications were managed with EndNote version X9 (Clarivate Analytics) and Microsoft Excel (Microsoft Corporation).

Study selection

All search results were downloaded and imported into the EndNote citation manager, and duplicate references were removed. Then titles and abstracts were scanned to choose articles for full-text review. Any studies that have measured and valued the attributes of innovation were included in the final selection. We excluded any articles focused only on the measurement of the attributes of innovation without discussing or at least recommending the potential method of valuing those attributes. Based on these eligibility criteria, full-text articles were assessed, and articles were finalized for data extraction. Two reviewers (NP and SS) performed the study selection process independently, and any disagreement was resolved with help of the third reviewer (NC).

Data extraction

A standardized data extraction form was utilized to capture the following information: author, year of publication, attributes of innovation described, perspective, quotation related to the attribute of innovation, measuring method of the attribute, valuation method of the attribute, and recommended method to value the attribute. Two reviewers (NP and SS) independently extracted data from the included documents. All disagreements were resolved through discussion with the third reviewer (NC).

Qualitative synthesis

We summarized how the attributes of innovation measured and/or valued or recommended the method to value it in the included articles. One article might present the same attribute from different perspectives that were included in this review. Only measuring the attribute of innovation, ignoring the way of valuing it does not facilitate understanding the current gap, so those articles were excluded. Measurement was defined as how certain attribute is measured through quantitative or qualitative scale. The valuation was determined as approaches to assess outcomes relative to costs⁶⁵, or trade-off between benefits and cost⁶⁶, or relative comparisons between two interventions. Since the current study was intended to capture only the measuring and/or valuing methods of the attribute of innovation described without a need to determine the internal validity of the findings, we chose not to perform methodological quality assessment of included articles.

Results

Study selection

Searches in four bibliographic databases had produced 896 articles. After excluding 254 duplicate references, 642 unique articles were scanned based on titles and abstracts to identify articles for the full-text review. Following the full-text review of 46 articles, eight articles were selected to include in this review. In addition, ten documents (includes both peer-reviewed articles and white papers) were identified to include in this review through manual citation searching. In total, 17 documents were deemed eligible for this review (Figure 1). The measurement and valuation approaches to capture the attributes of innovation were described in detail in Supplement 2.

Figure 1. PRISMA diagram of study selection process.

Study characteristics

Of the included articles, most of them (8/17) were narrative reviews^67–75, four were qualitative studies^76–79, three quantitative studies^43,80,81, and one commentary⁸². The majority of the included articles were from countries in Europe and North America, including UK, Germany, Italy, Spain, the USA, Bulgaria, Switzerland, and Canada. One study was conducted in Israel, a country in the Middle East⁷⁸.

The majority of the studies (10/17) did not specify the product types and talked about innovation in general. Among eight other studies, three studies were related to cell and/or gene therapy^70,71,83. The remaining four studies mentioned innovation in hepatitis C virus (HCV) treatment⁸¹, inhaler technology⁷⁹, cancer treatment with Nivolumab⁸⁰, and a digital medical management system called ReX⁷⁸.

Most of the studies (11/17) did not mention specific attributes related to the innovation. Substantial benefit^76,80 and convenience and/or adherence^67,78 attributes were examined in four studies. The other two studies captured uncounted benefits^69,83 and step-change⁶⁹.

Six studies measured the attributes of the innovation from the payers’ perspectives^{69,70,74,81–83}. Other studies measured the attributes of the innovation from the researcher’s perspective^{71–73,79,80}, decision-makers or regulatory agencies such as NICE⁶⁷ and AIFA - the Italian Medicines Agency⁴³, industry⁶⁸, provider⁷⁷, and patients⁷⁸. One study was conducted from multiple perspectives, including patients, providers, policymakers, caregivers, and the general population⁷⁶.

Overall description of the method used to measure and value the attributes of innovation

Two attributes of innovation, substantial benefits, and convenience and/or adherence, were measured and/or valued in the included studies. Some included studies also measured and/or valued overall innovation in general without describing any specific attributes of innovation.

Measurement of innovation in healthcare

Broadly, three tools were applied to measure only two out of eight innovation attributes that we conceptualized in the previous systematic review published elsewhere. The measurement tools of the innovation attributes in healthcare are listed in Figure 2 (details in Supplement 2). Three measurement tools were: (1) adherence rate, (2) life year or QALY gain, and (3) overall innovative scale (point/binary scale). First, the adherence rate was calculated by the subject’s report, remaining pill count, and ReX (an innovative medication management system)⁷⁸ record of measuring convenience and/or adherence. Second, life year or QALY gain was applied to measure the substantial benefit^76,80,82. Life year gain measures the survival gain due to the innovative treatment, while QALY gain measures the improvement of length of life adjusted by QoL as a result of the use of new technology. Third, the overall innovative scale was another tool that attempts to measure innovative status based on the presence of specific preset criteria^43,67,82. For example, AIFA Italy determines the innovative status of technology if it meets three criteria (i.e. unmet therapeutic need, added therapeutic value and quality of the evidence) and categorizes them either as fully innovative, conditionally innovative, or non-innovative⁴³. The binary scale determines innovativeness based on fulfilling high vs. low unmet needs⁸². Similarly, the added value provided by a technology can be measured in terms of the “improvement in expected benefits” in comparison with another, on a scale from I to V (level V is for no improvement)⁶⁷.

Figure 2. Identified measurement, valuation approaches, and recommended methods to capture healthcare innovation.

Of the included studies, one study has suggested a potential measurement approach of healthcare innovation. The idea of “impact inventory” was originally proposed by the Second Panel on Cost-Effectiveness in Health and Medicine to capture all of the innovation consequences both inside and outside of the formal healthcare sector by applying a structured checklist⁷⁰. Incorporating these innovation attributes or other novel elements would augment the CEA to be more comprehensive in line with the Second Panel’s “Impact Inventory.”⁷ Accounting for these indirect healthcare benefits of an innovative therapy, such as impacts on caregivers and social services and economic productivity, is crucial as it can profoundly impact whether treatment would be cost-effective at a given threshold⁷⁰. Another similar idea was to develop a checklist for the value of innovation of gene therapy to determine whether and to what extent key elements of gene therapies have been measured⁷⁰. This checklist would aid in determining whether the attributes of innovation or other novel elements of value flower such as real option value, caregiver burden, insurance value, scientific spillovers, and severity of the disease was measured.

Valuation of innovation in healthcare

Six different approaches were observed in the included studies to value the attributes of innovation (Figure 2, details in Supplement 2). These approaches included (1) QALY based cost-effectiveness analysis (CEA), (2) rating, (3) the economic value of life year gain, (4) multiple criteria decision analysis (MCDA), (5) incremental net health benefit (INHB), and 6) quality-adjusted cost of care (QACC). Primarily, the QALY-based cost-effectiveness analysis (CEA) was used for the valuation of convenience and/or adherence and substantial benefit attributes. The economic value of life year gain was applied to value the substantial benefit. All other approaches were used to value non-specific attributes, i.e. innovation in general. Also, while these approaches could be interconnected, this systematic literature review presented them as the literature did to provide a clearer picture.

First, CEA was the most frequently cited method to value innovation^76,77,82. It aggregates the survival gains and improved quality of life into quality-adjusted life years (QALY). This valuation derived part of the QALY from the innovation attributes. It incorporated it in CEA to calculate the incremental cost-effectiveness ratio (ICER), which reflected how much an additional cost was needed to obtain an additional QALY due to changing from one technology to another technology. For instance, Claxton et al. included the impact of convenience and/or adherence attribute on health outcomes in a decision model⁶⁷. The substantial benefit attribute would be another example. Give this method only measured substantial benefit and convenience and/or adherence attributes, it begs the question of how appropriately the QALY framework can measure the attributes of innovation.

Second, the rating used categorical measures, e.g. Haute Autorité de Santé (HAS) criteria or AIFA innovation criteria^43,67,70,82, to assess the innovative status or degree to which the innovative product was added therapeutic value, in general. For example, the HAS in France used two criteria, including the “expected benefit,” which measures clinical utility, and the “improvement in expected benefit,” to measure the added value provided by technology, compared to another technology, and then promptly valued technological innovation.

Third, the economic value of life year gain converted the improvement over survival derived from innovative treatments into monetary terms by applying an arbitrary monetary value of a life year to the additional life-years gained. For instance, each additional life-year gain was valued at $150,000⁸⁰.

Fourth, one study used the multiple criteria decision analysis (MCDA) to aggregate the benefits of a technology, including the innovation attribute such as unmet need⁸². It is important to differentiate MCDA from other approaches as it is often promoted and discussed but sub-optimally used to incorporate different value attributes into decision-making. Assuming all relevant attributes have been identified and measured appropriately, participants then can allocate different weights to each attribute to generate an aggregated value.

Fifth, the incremental net health benefit (INHB) was used to quantify the total lifetime value of an innovative technology⁷⁴. It unified two CEA dimensions (health benefits and costs) into one economic dimension, which was the INHB. Similar to the CEA, the innovation attributes could be included through the health benefits in this approach. Typically, the associated decision rule of using the INHB was to invest in the new technology if the INHB was higher than 0⁷⁶.

Last, the quality-adjusted cost of care (QACC) evaluated the value of innovation in hepatitis C virus (HCV) treatment. QACC provides a straightforward approach for incorporating value into calculations of cost growth⁸⁴. It can be estimated in following steps: first, compute the incremental cost due to innovation; second, measure the incremental benefit in value to patients in terms of monetized gains in quality-adjusted life-years (QALYs)⁸⁴. The difference defines the quality-adjusted cost of care as illustrated in the following equation: QACC = (incremental cost)-(monetized increase in QALYs, defined as incremental QALYs × value of QALY)^81,84. Younossi et al. 2016 estimated the value of one QALY at $50,000 to calculate the QACC⁸¹. When the estimated QACC was less than 0, cost growth with innovative technology was regarded as justified by the associated value to society⁸¹.

In addition to the valuation of the innovation attributes, Figure 2 also shows the list of studies that recommended potential methods to value innovation in healthcare in general (details in Supplement 2). These methods were recommended by the literature but had not been used. A single study might recommend multiple methods. These potential approaches to value the attributes of innovation included 1) ICER threshold adjustment and incorporation of innovation into CEA, 2) discrete choice experiment (DCE), and 3) multiple criteria decision analysis (MCDA). In this case, MCDA was proposed as a potential valuation method.

First, the ICER threshold adjustment suggested modifying the current cost per QALY approach, e.g. using a sliding ICER scale⁸². It is a commonly recommended valuation approach to accommodate healthcare innovation^70–72. The ICER threshold adjustments could take the form of a “sliding ICER scale” with clear rules based on product performance or applying a higher ICER threshold in case of innovation. Besides the ICER threshold adjustment, the incorporation of innovation into CEA proposed to integrate innovation more comprehensively beyond the traditional QALY-dependent measure⁷⁰. Several potential ways have been recommended to incorporate innovation within CEA. One study suggested using the primary clinical outcomes in their natural units and let policymakers make the trade-offs between them to reward innovation⁶⁹. Dynamic CEA was another potential approach where the benefits to future (incidence) patients and the cost savings due to price erosion and off-patent price dynamics were taken into account in addition to conventional CEA components to quantify the total lifetime value of an innovative drug⁷⁴.

Second, the DCE aimed to explore individuals’ valuations of the various attributes of treatments⁶⁹. The DCE was used to assess the added value of technologies, comparing several attributes of treatment relative to one another. It allowed examining clinical outcomes and other aspects such as convenience and duration of treatment to capture the value of innovation.

Finally, the MCDA aggregated the different innovation attribute analyses to be used in the decision-making process. Since a single dimensional approach, e.g. QALY, might not be able to value the full benefit of innovation, various studies suggested using MCDA to measure the value of additional elements^69–71,83.

Discussion

This systematic review investigated the methods of measuring and valuing innovation attributes in the decision-making process after identifying the attributes of innovation from different perspectives in our previous systematic literature review¹⁴. In general, the innovation attributes in healthcare were measured in terms of the product benefit. Traditionally accepted life-year gain or QALY gain approach and overall innovative scale (e.g. scale from I to V) were used to capture or measure the level of improvement in health or fulfillment of the unmet need. However, these approaches failed to capture innovation consequences comprehensively.

The traditional belief is that HTA should maximize health benefits given scarce health financial resources by capturing only benefits and costs deemed relevant from a healthcare system perspective. However, it has been argued that other sectors should also account for their spillover benefits on health. Indeed, the COVID-19 pandemic and the broad benefits arising from effective vaccines are beginning to unveil the nature of healthcare’s broader benefits. But benefits outside the formal healthcare system were not considered while valuing a technology. A broad definition of health has been proposed to include physical, mental, and social domains in addition to the conventional view of the absence of disease as health that further certifies the necessity to capture the benefits accrue beyond the healthcare system⁸⁵. Alternative approaches such as impact inventory were thus suggested as a potential alternative measurement approach to capture the health benefit of innovation outside formal healthcare.

We found that the CEA was the most frequently used technique to value the innovation, where innovation was captured via survival/efficacy gain and savings in cost. Interestingly, the economic value of life year gain, INHB, and QACC approaches used a similar concept based on either life year or QALY gains. Primarily, these approaches applied a certain value or willingness-to-pay for life-year or QALY as a means to value the innovation attributes. While these approaches could be consolidated or defined as a CEA or CEA-related approach, this systematic literature review separated them to show more details of these valuation approaches.

The categorical rating and MCDA were the other two possible valuation approaches. They were not used as often as the CEA and CEA-related approaches. One of the reasons was that both approaches were likely based on the perspectives of decision makers, and they could be considered more subjective. However, MCDA was suggested as a potential valuation approach by several researchers and policy makers as a potential tool to value new technology because it is capable of including multiple attributes, including the innovation attributes, which can lead to a more comprehensive value assessment^70,83. In addition, other potential valuation methods were mentioned in this review. Most of the possible methods suggested revolved around adjusting the CEA, e.g. varying the ICER threshold or QALY weighting^{67,69–71,83}. However, the ICER threshold adjustment will only account for those attributes that have been identified. Dynamic CEA method was recommended that would allow accounting for subsequent price erosion and off-patent price. The price of healthcare innovation resembles a mortgage payment as the society will eventually accrue healthcare innovation after patent expiration and will benefit from often lower prices due to subsequent competitors (not necessarily generics) entering the market⁸⁶. DCE was another mentioned method to capture the preference-based value of innovation⁶⁹. These two approaches were relatively intuitive. Since the decision of CEA or CEA-related approaches was primarily based on the value or willingness-to-pay for life-year or QALY, the varying ICER threshold would allow decision makers to capture the value of innovation. Similarly, the DCE could accommodate the inclusion and valuation of the innovation attributes and cost to allow decision makers to value the innovation.

Although a few methods have been proposed to capture the innovation in healthcare, they primarily measured or valued some innovation attributes identified in the previous systematic literature review, including uncounted benefit, convenience and/or adherence, substantial benefit, and step change attributes. Thus, the measurement and valuation of the other identified innovation attributes were still needed. Also, it was interesting to find that only a few studies measured the innovation attribute (i.e. convenience and/or adherence) based on the patient perspective. One of the reasons was that the innovation attributes were not clearly defined, and therefore they were not ready for the patients to discuss the value of these attributes.

Overall, there is a broad number of healthcare innovation attributes exist, and some attributes which are often not identified (and thus not measured/or valued) by HTA or Value assessment frameworks. Of the identifiable innovation attributes, some are quantifiable/measurable and some of them are difficult to quantify/measure or remain uncounted. Being part of the latter group does not mean they are not worth it. More importantly, it can be argued that HTA bodies lack a thorough process to discern which attributes can be measured and/or valued, and which cannot thus remain uncounted or missing. This process is likely to be heavily influenced by various factors such as, specific disease, population, healthcare setting, etc. Moreover, it will require active engagement of patients, caregivers, family members, providers, and other relevant stakeholders (e.g. employer, insurer, etc.) to make this process robust.

Given the context of our findings major challenges related to healthcare innovation could be: (i) to identify all relevant innovation attributes when conducting HTA, (ii) to appropriately measure and value those quantifiable attributes (e.g. assuming a QALY paradigm does the job perfectly), (iii) to find ways to measure and value those uncounted attributes to incorporate into HTA decision making (iv) to identify, measure and value healthcare innovation attributes which have an impact beyond the healthcare setting (e.g. spillover benefits to national economy, other economy sectors).

Our study suffered from three major limitations. First, for some studies, the innovation attributes were not stated explicitly, and they were interpreted at researchers’ discretion. Second, some measurement tools were not adequately described and interpreted to the researchers’ best understanding. Third, although we have performed manual searches of the included studies’ reference lists and included the eligible white papers in this review, comprehensive exploration of the grey literature was out of the scope; thus, we might not capture other valuation approaches if they existed.

Conclusion

Several studies included various valuation approaches of the innovation in healthcare in general, based on multiple perspectives. Primarily, the existing valuation approach centered around CEA with QALY. Some studies also recommended valuation approaches such as ICER threshold adjustment, MCDA. Only a few studies included the patient perspective, and only two innovation attributes identified from the previous systematic literature review were valued while most of them are difficult to measure and/or value and remain uncounted. Key challenges remain with respect to appropriately measure and value these identified attributes, devise a way to identify, measure and value uncounted attributes, and the impact of health care innovation to incorporate them into HTA decision making. Our review will benefit researchers in developing or adopting an approach that values innovation to help policy makers understand the holistic value of technology by considering innovation as a component to be incorporated in the overall value assessment.

Transparency

Declaration of funding

This work was supported by Bristol Myers Squibb (BMS). BMS also provided funding for the development of the manuscript as well as for article processing charges.

Declaration of financial/other relationships

JD is employed by Bristol Myers Squibb. All other authors have indicated that they have no conflicts of interest regarding the content of this article.

Author contributions

SS and NP reviewed literatures, extracted data for the study, synthesis, and conceptualization with input from NC, SN, and JD. SS, NP, SN, JD, and SS wrote the first drafts of the manuscript and all authors made substantial contribution. All authors contributed to the study design, interpretation of findings, and critical revision of the manuscript.

Reviewer disclosures

A reviewer on this manuscript has disclosed that they are friends with three of the senior authors on the paper. The rest of the peer reviewers on this manuscript have no other relevant financial relationships or otherwise to disclose.

Availability of data and material

All data are available from the corresponding author upon request.

Consent for publication

All authors approved the final version of the manuscript for submission.

Acknowledgement

We are grateful to Sajesh K. Veettil for assistance in study selection process.