How do we design a high-quality evaluation system to ensure that we get value for money from health technology but with a process that does not impose excessive costs? This is the question at the heart of a recent review of the health technology assessment (HTA) process in Australia Citation[1].
HTA in Australia & the review
In Australia, three advisory bodies at the national level are responsible for evaluating evidence relating to the safety, effectiveness and cost–effectiveness of new medical technologies. The Therapeutic Goods Administration (TGA) has assessed the quality, safety and efficacy of medicines and regulated the quality, safety and performance of medical devices at a national level since 1991 Citation[2]. The Pharmaceutical Benefits Advisory Committee (PBAC) was established under section 101 of the National Health Act 1953 to assess the effectiveness of prescription pharmaceuticals prior to subsidy on the Pharmaceutical Benefits Scheme (PBS), and since 1994, it has formally assessed cost–effectiveness Citation[3]. The Medical Services Advisory Committee (MSAC), set up in 1998, assesses health technologies and medical procedures in the private health sector, subsidized under Medicare, the national insurance scheme Citation[4]. The maturity of this two-tier centralized model of HTA assessment for market entry and then for public subsidy contrasts with most other countries where there is a much shorter history of HTA, often only at the market entry level, with perhaps an advisory role in utilization, an indirect relationship to funding, and rarely at the national level. However, the maturation of the system in Australia has not been without controversy. The early years of the PBAC saw a review with particular emphasis on its internal efficiency and fairness Citation[5]. The recent HTA review focused on the MSAC, the Prostheses and Devices Committee (PDC) and the regulatory process by the TGA. A key concern was that sponsors of new medical procedures or devices must navigate at least two separate independent processes (TGA and MSAC or PDC), and in a small number of cases, all three, and if a new diagnostic technology is linked to a medicine, the PBAC might also be involved.
The aim of the 2009 HTA review was to “address the regulatory burden on business that results from HTA processes”, but also to “identify opportunities for reform of the process that may be poorly designed, duplicated or unnecessary” Citation[1].
What did the review find & how can other countries learn from the results?
The development of formal HTAs prior to market entry or prior to public funding has always been accompanied by concerns about the cost that regulation imposes on the health system. In addition to a broad concern that strict criteria for evaluation of effectiveness or cost–effectiveness may preclude approval of technologies that are thought by others to be socially valuable, there have also been concerns regarding process-related delays in approval.
The HTA review highlighted the considerable diversity in approach to the evaluation of health technologies across agencies in Australia, both with respect to the quality of evidence and also the extent of stakeholder, independent expert and community consultation. For example, the review comments that “MSAC has been slower, less flexible and possibly less consistent in its recommendations than other Commonwealth HTA processes” Citation[1]. The lack of consistency of decisions within the MSAC, in part, relates to the comparatively small number of varied evaluations and the lack of standardization of the clinical and economic evidence requirements in detailed guidelines, as well as to conducting evaluations using a set of external evaluators and one-off application-specific specialist committees. Unlike the PBAC, the MSAC has accepted references from government as well as submissions from industry to no strict schedule, consulted widely with experts and published full details of the evaluation on the website. In some of these respects, this is closer to the National Institute for Health and Clinical Excellence (NICE) process in England and Wales. The preparation of assessment reports by independent external groups with formal expert consultation has resulted in variable quality of assessments at a pace that is widely regarded as unacceptable. The MSAC was heavily criticized in the HTA review for its productivity and the time taken to produce reports. The open consultative nature of the process, compared with the more adversarial, less flexible approach taken by authorities such as the PBAC in Australia and the Canadian Common Drug Review (CDR) in Canada (who critique sponsor-prepared submissions with little external input), is widely seen as a major contributor to lower productivity and higher regulatory costs with no obvious gains in outcomes.
One of the strengths of federal HTA processes in Australia is the national consistency in the availability of new technologies through the direct link between assessment and public funding; however, the independence of the medical and pharmaceutical approval processes has raised a number of additional issues. As the PBAC and MSAC operate independently, there is the potential for inconsistency in methodologies, outcome measures and, more broadly, in what thresholds of evidence are used. The compartmentalization of assessments can be awkward, for example, when a biomarker assessed by the MSAC is important in determining the effectiveness of a particular medicine assessed by the PBAC. Coordination problems can also arise in relation to assessments where the technologies are a hybrid of a device and a drug.
In response to these issues, the review made a number of recommendations including:
• Consistency: a common set of principles should apply across all federal HTA activities;
• Procedural performance and fairness: an independent review mechanism and specific milestones and timeframe targets for each HTA process should be established;
• Coordination: applicants should be able to choose to have concurrent assessment for market entry and subsidy for certain products and there should be greater coordination across committees, particularly for complex services that involve a mix of technologies;
• Cost: the rigor of the evidence-based assessment should depend on the (weighted) potential risks and benefits of the technology and apply this across all agencies Citation[1].
The review suggests that as a minimum, there is a need to integrate the processes of the two main federal HTA bodies in Australia with a single entry point for sponsors of technology and a common set of objectives and standards that encourage the production of high-quality evidence. Note that, as NICE illustrates, a single entry point of itself does not necessarily mean a single evaluation stream, nor does this resolve either the issue of differential quality of evidence or the need for a process to evaluate some technologies concurrently. It appears from many of the submissions to the review that the industry concern was not just about a lack of information on processes, but more about an assurance that evidence is properly considered without bias and that there are not unnecessary delays with consequences for patients, government and the health services industry.
A notable feature of the HTA review was the recognition that even in a country with a long history of HTA, such as Australia, there has been little progress in the evaluation of older established technologies (in part because of poor evidence of effectiveness) or of integration of evaluation processes across healthcare streams (pharmaceuticals, medical devices, diagnostics, medical services, and prevention). While there have been some tentative successes in using HTA in other jurisdictions to generate better quality evidence Citation[6], and suggestions for a similar process for prevention Citation[7], for the most part, decisions are limited by uncertainty surrounding the effectiveness of new technology. The review does suggest that discontinuing the use of some older technologies is a priority, but for the most part, this has been left to obsolete clinical processes. In the context of the lack of incentive to replace comparatively ineffective but established techniques, and a typical lack of evidence that a technology is completely ineffective, the review made no new suggestions. Nor was there any discussion regarding mechanisms for ensuring that all new technologies are evaluated prior to funding. Many technologies in Australia are funded either under an existing Medicare item or in public hospitals without MSAC approval. In addition, as some participants in the review pointed out, if HTA processes are to be consistent with broader government health policy, which has a strong emphasis on prevention of illness, then the scope of HTA activities should be widened to include preventative and public healthcare, such as screening programs and genetic testing.
Where to now?
The process of generating and assessing evidence of effectiveness and cost–effectiveness of new health technologies, and then implementing that evidence into changes in practice, is made more difficult by healthcare systems that are fragmented in terms of responsibility for policy and funding for populations and services. Australia has been dealing with these issues for decades with some success, but the development of new kinds of technology that cut across established regulatory and funding divisions have highlighted gaps, inconsistencies and procedural inefficiencies. The review focused on procedural inefficiency from federal agencies, but at least as important are the gaps in HTAs of medical services outside of the national Medicare list, those that are already funded under an existing Medicare subsidy item and non-medical services that are not evaluated at all.
The Australian review recommended that rigorous consideration of evidence be consistently applied across all government HTA processes and that government HTA should have a role in improving the evidence base where necessary. The improvements that are likely to be implemented relatively easily include: a more consistent re-submission and review process across the HTA processes with more consistent guidance on methods (both written guidelines and individualized advice); and allowing concurrent assessment of technologies across agencies. Indeed, parallel evaluation of some drug submissions by the market entry regulatory agency (TGA) and the subsidy regulatory body (PBAC) has already occurred in 2010.
The review also recommended that a risk-based approach to the assessment of medical services, devices and prostheses be taken. This appears to mean that the intensity and cost of assessment would match the risk posed by use of the health technology. There is obvious good sense to this and it is not necessarily inconsistent with the second recommendation of the review, “That the rigorous consideration of evidence be consistently applied across all Commonwealth HTA processes” Citation[1]. The challenge is how to formalize this into a matrix of risks (e.g., adverse events, primary effectiveness and financial) across current and future HTA processes without compromising overall quality.
The review also reveals that successful HTA processes need to generate confidence in the processes if they are to be sustainable. One of the lessons of the MSAC (and to a lesser extent, NICE) experience is that a process, even if it takes pains to seek the views of interested parties, can be both inefficient and seen as unfair if the methods of evaluation and the criteria for decisions are unclear and inconsistent. Confidence in the system lies not just in openness but also in its perceived fairness. Fairness is not just about transparency but a perception that the system is comprehensive and will take account of the best evidence available without bias, and at minimum cost to all parties.
Financial & competing interests disclosure
Anthony Harris has a contract with the Australian Government to provide assessments of submissions to the Pharmaceutical Benefits Advisory Committee. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
No writing assistance was utilized in the production of this manuscript.
References
- Australian Government Department of Health and Ageing. Review of Health Technology Assessment in Australia. Commonwealth of Australia, Canberra, Australia (2009).
- herapeutics Goods Administration. Regulation of Therapeutic Goods in Australia. Australian Goverment Department of Health and Ageing, Canberra, Australia (2005).
- Salked G, Mitchell A, Hill S. Pharmaceuticals. In: Economics and Australian Health Policy. Mooney G, Scotton R (Eds). Allen & Unwin, Sydney, Australia 115–136 (1999).
- Australia Productivity Commission. Impacts of Advances in Medical Technology in Australia: Productivity Commission Research Report. Productivity Commission, Melbourne, Australia (2005).
- The Auditor-General Australian National Audit Office. Audit Report No 12 of 1997–98: Performance Audit, The Pharmaceutical Benefits Scheme. Department of Health and Family Services, Canberra, Australia (1997).
- Goeree R, Chandra K, Tarride JE et al. Conditionally funded field evaluations: paths coverage with evidence development program for Ontario. Value Health13, S8–S11 (2010).
- Harris A, Mortimer D. Funding illness prevention and health promotion in Australia: a way forward. Aust. New Zealand Health Policy, 6, 25 (2009).