The Newest Three-Letter Fad in Health: Can HTA Escape the Fate of NHA, CEA, GBD?

Prioritization in health has always fascinated me,^1,2 so when I saw that the 2016 Prince Mahidol Awards Conference (PMAC) had “Priority Setting” as the theme for the event (http://pmaconference.mahidol.ac.th/), I was really excited. An early draft of the agenda, however, tempered some of the excitement and raised in my mind an issue that the health sector continues to struggle with when it comes to approaches to prioritization: falling in love with technocratic approaches. Global conversations about prioritization always risk being dominated by a highly technocratic agenda that caters more to a donor-focused environment than national needs for low- and middle-income countries. The most recent of these technocratic approaches, on display at PMAC, is health technology assessments (HTAs). Though there is no doubt that HTAs can and should play an important role in prioritization of limited resources for health, there is a long history of overselling technical answers and in some cases causing more harm than good.

The January 2016 issue of Health Systems and Reform, “Special Issue: Prince Mahidol Award Conference 2016: Priority Setting for Universal Health Coverage,” offers some hope. The issue included several commentaries and articles that urge a balanced approach to prioritization,^3-6 and others explore the limitations of empirical tools like cost effectiveness and HTA.^7-9 The main question for me is the following: Will the zeal for a technical answer win over the more pragmatic commentary presented in these articles of Health Systems & Reform?

WHAT CAN WE LEARN FROM THE NATIONAL HEALTH ACCOUNT STORY?

The mid–late 1990s saw the rise of another three-letter tool, national health accounts (NHA). As a health economist working in development, I have been an avid consumer of NHAs, but the challenge was not the tool but how it was sold and used to produce considerable consultancy money with little to no value for or ownership by low- and low-middle income countries (probably upper–middle-income countries as well).

It might be useful to see how NHA went from a useful tool for some countries to a sizable waste of global funding for health. A quick history of the NHA journey may be instructive and could help HTA not go that way:

Start With a Good Thing

The HNA story started with a reality that Organization for Economic Cooperation and Development (OECD) countries have been systematically capturing spending on health. Countries making policy decisions in health need to know how much is being spent (total health spending) and by whom (private, public), how it is organized (insurance funds, public budgets, out-of-pocket), on what levels of care (primary, secondary, tertiary, outreach), on what programs, on what services, and which segments of society are being serviced (benefit incidence).

This level of detail can be highly attractive to policy makers, analysts, the media, civil society, and developing country donors. A question that came back to haunt NHA is whether it is attractive enough to decision makers in developing countries to make it something worth pushing on. As we found out eventually, the answer to that question was an unequivocal no.

Oversell It, Even for Low-Income Countries, but Focus on the Donors

Even for OECD countries, it can be argued that there is a substantial gap between what analysts and health economists think NHA can do and what policy makers use. The impressive thing, however, is the aggressive and successful way NHA was sold to development agencies and funders. A group of hard charging, mostly U.S.-based, professionals made a very strong case for NHA as a prerequisite for health systems work and all of the major donors and agencies jumped in.

At best, a series of NHAs in a country will give a picture over time of how the market and public policy influence decisions by providers and, to a lesser extent, by the population. How these data are used, if used at all, depends entirely on the appetite of the policy makers and an understanding of how data can help with decision making. In other words, for NHA to have any influence it needs to be demanded by the national decision makers. What has been amazingly absent in the 20-year history of NHA in developing countries has been the direct interest by developing country policy makers. It would have made much more sense if the demand for NHA came from Abuja, Dhaka, or Dakar and not from Washington, Geneva, Paris, Seattle, and New York. It would have made far more sense if the institutional housing for NHA work was at a country level in developing countries and not in consulting firms selling a product to the donor world.

How NHA Lost Its Way

A good Ph.D. thesis should be written to capture the many different ways in which NHA has failed to deliver much for low- and middle-income countries. Within this commentary, however, let me just note a few of the ways we can see this failure:

1. Pragmatism lost. It took three years for the OECD, the U.S. Agency for International Development, and the World Bank, among others, to come to an agreement on how to conduct an NHA and to produce a manual. Even when a manual was finally produced, the emphasis was on a level of detail that may work for OECD countries but made little sense to data-poor countries. As a result, many of the low- and lower–middle-income country NHA products relied too much on assumption when detailed data simply did not exist.

2. Country relevance lost. The downside of the “how-to manual” approach is that the NHA work was driven by technocrats interested in cross-country comparisons, which made it less attractive and much less relevant to country policy makers. It is not surprising that it was rarely institutionalized. But even when we turn to use the existing NHA data for international comparisons, the overly detailed requirements needed are rarely available in most low-income countries, which means that most likely the data in existence are dominantly estimated instead of being based on reality.

3. Ownership lost. The mismatch of an OECD instrument and low-income country implementation led to a far more expensive and time-consuming product that was very hard to replicate and not locally owned. In practice, we started seeing one-time efforts where a donor funds an NHA exercise in a country, it takes two to three years for it to be concluded and based on six-year-old data and lots of guesses, and the final product is owned completely by the consultants that conducted it and the donor that funded it but not the country it was conducted in. It is instructive that the most used descriptor of NHA efforts in low-income countries is “lack of institutionalization.”

Perfect Is the Enemy of the Good

Perhaps the saddest outcome of the overselling and mismatch of NHA to low- and middle-income countries is that the limited human and financial resources used for these efforts were not used for a much simpler and much more directly relevant data collection effort on health spending, public expenditure and institutional reviews (PEIRs). PEIRs are a cousin of NHAs but far less ambitious in scope and far more policy friendly. A PEIR only focuses on the part of the health spending that goes through the government budget. This simpler focus makes it far more likely to be done quickly than an NHA. More important, PEIRs have far more direct links to policy:

1. A government budget is the direct empirical representation of the actual policies of the country. In fact, it is far more accurate than policy statements because it represents exactly the priorities a government is funding (versus stating). So the small sub-set of total health spending of a country where government has complete control is far more relevant to decision making than overall spending. That does not mean that overall spending and the breakdowns are not important but that they are relatively far less important, especially if they are very difficult to capture empirically in most developing countries.

2. The “institutional” element of PEIRs looks at the whole process of budget setting relative to priorities and needs, budget transfers (timing and volume), budget execution, and capacity-related issues. All of these elements of a PEIR go to the heart of the ability of a government to tackle priorities and problems and are completely missing in the far more expensive and time-consuming NHA exercise.

With limited resources, human and otherwise, the push for NHAs over the last 20 years has had a direct consequence to alternative, cheaper, and far more relevant health expenditure analysis instruments like PEIRs, depriving countries of more timely and policy friendly data and information. Donor decisions have consequences.

OTHER MISUSED THREE-LETTER TOOLS

During the work on the World Development Report 1993,¹ there were concerns that the new tools being used, cost-effectiveness analysis (CEA) and global burden of disease (GBD), which were fairly crude and included far more assumptions than data, would be eventually abused. These concerns unfortunately came true. What was intended with CEA, for example, was a way to begin to identify extremes in terms of returns to investments in health services. In other words, the tool allowed the identification of interventions that had a fairly high return relative to cost as well as others that have very little return relative to cost (figure 3.2 on page 62 of this report captures the intention of using CEA to identify orders of magnitude instead of point estimates). Given the limited availability of country-level data, the idea of point estimates at the national level made no sense. But even if data existed at the country level, prioritization in a complex sector like health, with multiple outcomes and complex systems, cannot be primarily an empirical exercise.

A few years after the publication of World Development Report 1993,¹ however, saw a whole industry develop around producing country level point estimates of CEA (as well as adapted GBDs) that were then used to develop benefits packages to be financed and delivered. Once again, this was driven by a donor appetite to fund a technocratic approach to prioritization of health budgets in developing countries.

A WAY FORWARD FOR HTA

Like NHA, CEA, and GBD, HTA is an excellent empirical tool that should be very helpful for policy makers in health. Absent some thoughtful interventions, however, HTA can quickly follow the wrong path of the other three-letter technocratic tools. The Prince Mahidol Awards Conference and the Special Issue of Health Systems & Reform presented opportunities for thoughtful conversations on the complex nature of prioritization in health. I offer two thoughts for future conversations on this topic, based on learning from past mistakes:

1. Donor accountability. A common theme from the NHA, CEA, and GBD experiences is the unbridled enthusiasm from donors for technocratic tools that have little country demand. For any tool to be of use, the ultimate users must demand it. Supply-side funding is not enough and in fact may be destructive.

2. Country relevance and adaptation. Another common theme in previous failures is the dominance of an approach that prioritizes international comparisons over country customization and needs. One approach to consider is to think of HTA as a global public good taking advantage of strong existing institutions like the UK National Institute for Health and Care Excellence (NICE) and do adaptive calculations at national levels taking into account local prices and capacity to deliver new technologies.

Though policy-focused conferences, like PMAC, create a space for thoughtful conversations, ultimately the success or failure of HTA as an instrument serving prioritization for the health sector in low- and middle-income countries will depend partly on donor behavior. A global health community that puts country needs first and recognizes and learns from past failures can save considerable resources and advance the greater good.

DISCLOSURE OF POTENTIAL CONFLICTS OF INTEREST

No potential conflicts of interest were disclosed.