Health economic evaluation: in need of more analytical rigor or more practical relevance?

Health-technology assessment is the systematic appraisal of the properties and effects of health technologies. It may address the direct, intentional consequences of technologies as well as their indirect, unintentional consequences. Its main purpose is to inform decision-makers in healthcare, and it is conducted by interdisciplinary groups using explicit analytical frameworks drawn from a variety of methods [101]. It should be noted that, broadly defined, health technologies include any health intervention, such as procedures, settings of care and screening programs, and are not solely limited to new medicines or technologically advanced equipment [102]. Health economic evaluation is a dimension of health-technology assessment that aims to inform decision-makers of the most efficient ways to use healthcare resources by weighing the benefits of a technology against its costs [1]. In an era of limited healthcare budgets, this approach can be more useful than other aspects of health-technology assessment, which simply assess the effectiveness of a health technology by only offering information on whether it is clinically beneficial without any consideration of cost. It is not surprising, therefore, that economic evaluation during the last 10 years has become increasingly important in healthcare decision-making [2–5]. This growing influence is reflected clearly by both the rapid expansion of the international literature and by the introduction of national health-technology assessment agencies in a number of countries, which now require formal evidence that technologies represent good value for money before they are approved for use in the healthcare system. Health economic evaluation is undoubtedly a product of the 20th Century welfare state socialism. Its methods have been developed mainly in the UK, reflecting the need of socialized systems to contain costs, prioritize healthcare and make healthcare decision-making more efficient and transparent. It is in sharp contrast to the American approach, which is the result of 18th Century enlightenment; its over-riding concern is the relationship between individual freedom and coercive government power. Here, the government’s powers are limited, with strong emphasis placed on market competition that maximizes consumer choice and less emphasis placed on the value for money of healthcare technologies.

Health economic evaluation methods focus, for the most part, on the appraisal of individual technologies (e.g., drugs, medical devices and surgical procedures) and, to a lesser extent, on complex public health programs [6]. Drummond et al. point out that each method has a different scope and suitability [1]. Cost analyses are the simplest economic evaluation studies and, as their name suggests, they are only concerned with costs; the health-related consequences of treatment alternatives are not considered. Cost-minimization studies take economic analysis to the next level by considering alternative uses of resources, assuming that outcomes are the same in the technology being evaluated. However, the most commonly used type of economic evaluation is the cost–effectiveness analysis, which combines single outcomes (e.g., deaths averted or life-years gained) and costs into a ratio. Sometimes it is necessary for the components of incremental costs and consequences of alternative technologies to be calculated and displayed without aggregating these results. In this case, cost–consequence analyses are conducted, which are essentially a form of cost–effectiveness analysis. When the impact of a technology is needed to be measured and valued in terms of improvements in preference-weighted, health-related quality of life, such as the quality-adjusted life-year (QALY), cost–utility analyses (another form of cost–effectiveness analysis) are performed instead. Cost–utility analyses are usually preferred because they allow comparisons to be made across all areas of health technologies. Finally, cost–benefit analyses value all costs and benefits in the same monetary units and if benefits exceed costs, then the technology in question is deemed to be worthwhile. Cost–benefit analyses are seen as the ideal approach, but conducting them can be problematic because the valuation of outcomes in monetary terms is deemed unethical. In recent years, the research community has witnessed the increasing adoption of sophisticated methods in conducting health economic evaluations. Current state-of-the-art practices include decision-analytic modeling, net-benefit approaches, cost–effectiveness acceptability curves and probabilistic-sensitivity analyses [7]. The growing call for technical expertise demonstrates the attempts of researchers to make economic analyses more consistent by separating the issues that concern society’s health values and those that should be clear, positive scientific questions. The rationale is that once the normative issues are resolved (e.g., which costs to include in the analysis, what to value and whose values to use), estimating the costs and effects of health technologies and handling uncertainty in the results obtained can be seen as positive scientific issues [103].

Research in European countries indicates that, despite its growing importance, the use of economic evaluation in healthcare decision-making still remains rather limited and there is scope for a greater role [8–12]. However, further adoption of health economic evaluation from policy-makers collides on several challenges, at both a conceptual and practical level. The creation of NICE in the UK has triggered an acceleration in the use of health economic analysis in a number of countries. However, the implementation differs in each country. For example, despite the fact that Germany and France’s health-technology assessment agencies follow NICE’s methodological principles for economic evaluation, to date, they have both rejected the use of QALYs to assess health benefits and of thresholds to formulate treatment guidelines and reimbursement recommendations. This choice reflects cultural and philosophical issues that prevail in different countries, and which conflict with the theoretical grounding of methods and instruments used in health economic evaluation [13]. A prominent example is the QALY, which, despite the continuous existence of severe theoretical and methodological limitations, has become increasingly accepted and used [14,15]. Another problem is reaching a consensus regarding an appropriate threshold level. In the UK, it has been suggested that the threshold level must be an empirical question based on explicit and transparent analysis, and that the range of £20,000 to 30,000 per QALY currently used by NICE represents an informed estimate. It has also been suggested that the threshold of £30,000 per QALY and even £20,000 may be too high [16,17]. However, decisions based on an immature evidence base regarding precise threshold estimates may be misleading and can do more harm than good. Therefore, much more work in this area needs to be conducted.

At a more practical level, accessing and collecting evidence on the cost or the cost–effectiveness of competing health technologies is proving a formidable task; potentially usable material is becoming increasingly vast and fragmented in a variety of sources. It is also generally accepted that the quality of health economic evaluations is, in many cases, questionable, thereby limiting the reliability of their results. The evolution of health economic evaluation into an interdisciplinary subject and the departure from its traditional welfare economics foundation means that, increasingly, researchers with different backgrounds are conducting these types of studies. Examination of the international literature reveals that those who carry out studies of health interventions that contain some consideration of costs are increasingly physicians and public-health professionals. An indirect consequence of this is that these studies are published in a variety of sources (e.g., medical, economic and other journals). The diversity and extensive variation in the publishing procedures that these sources practice and the emphasis they put on different aspects (in medical journals, clinical rather than economic) has resulted in studies of differential quality. In addition, the degree to which many of these health professionals fully understand the principles and methods of health economic evaluation is unclear; cost data are often included as an extra rather than as a central component of the analysis [18]. Finally, demographic and epidemiologic reasons, availability of healthcare resources, variations in clinical practice, inconsistent cost measurement and reporting, and population values placed on health states in different countries all limit the transferability of economic evaluation results to other settings [19].

In recent years, a number of specialized databases have been created that identify, index, summarize and critically appraise published health economic evaluation studies. The most prominent database is the National Health Service Economic Evaluation Database in the UK, which provides a summary together with a critical review of the methods used in the study in question in the form of a structured abstract, which highlights the strengths and weaknesses of the study and, in theory, allows readers to conclude whether this study is of use in their own setting [20]. However, there is still no clear evidence regarding the extent to which these databases actually remedy the problems mentioned earlier and what their actual contribution to healthcare decision-making is. The exponential increase in the number of applied studies published worldwide may soon hinder any possible progress made by establishing these databases in the first place. A quality scoring system for published studies would be of vital importance, as it could be used as a filtering mechanism for decision-makers. However, as recent development efforts have demonstrated, there are a number of challenges that still need to be addressed [21]. Last but not least, making more rational decisions using economic analysis also requires decision-makers to understand the principles of economic evaluation methodology and its underlying assumptions. It has been advocated that decision-makers have gradually acquired the expertise that is necessary to appropriately interpret the results obtained from this kind of scientific analysis [103]. However, increasing amounts of evidence suggest that this is not really the case; decision-makers are being called to make decisions using scientific tools that they still do not really comprehend and are thus unable to immediately see the possible caveats of these studies that, in turn, guide their decisions [8–12].

In conclusion, health economics is a relatively young discipline and it is only natural for it to be still plagued with methodological pitfalls. The scope for future research continues to expand and the caveats should not constitute an excuse for not using economic analysis in healthcare decision-making. Clearly, health economic evaluation has an important role to play with regards to the utilization of particular health technologies (based on an evaluation of value for the cost of those interventions), as it can promote rationalized rationing and lead to more consistent decision-making. Nevertheless, microeconomic evaluation at the treatment level constitutes only one piece in the health-policy jigsaw puzzle. Effective cost containment in modern healthcare systems can only be achieved with deep structural reforms that take into consideration the emergence of new challenges, such as chronic and lifestyle conditions, which gradually alter the healthcare needs of populations and require expensive treatments that will always be deemed as cost ineffective. The recent public outcry (and the reaction of vote-sensitive politicians) concerning cancer treatments that NICE refused to approve in the UK should remind us that, ultimately, it is society that dictates at, any given point in time, the ethical framework in which political decisions (including those concerning the use of healthcare resources) take place based on (but not solely and/or always bound to) economic considerations. Thus, fortunately or unfortunately, policy-makers will always have to deal with normative issues that will require value judgments. What also seems to have been long forgotten is the fact that economics is not an exact science, as mathematics is, but an intellectual discipline resting upon empirically observed facts, in which models are simply a tool for understanding. The fact that economics is seen as a social science is exactly because it has some of the attributes of a science; however, the nature of its subject matter is fundamentally ideological.

In light of these issues, perhaps it is time to ask ourselves an important question: does it make sense to merely keep introducing further analytical rigor, or is it time to also focus on the practical relevance of these methods, an aspect largely neglected thus far?

Financial & competing interests disclosure

The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.