Costing bias in economic evaluations

Abstract

Determining the cost-effectiveness of healthcare interventions is key to the decision-making process in healthcare. Cost comparisons are used to demonstrate the economic value of treatment options, to evaluate the impact on the insurer budget, and are often used as a key criterion in treatment comparison and comparative effectiveness; however, little guidance is available to researchers for establishing the costing of clinical events and resource utilization. Different costing methods exist, and the choice of underlying assumptions appears to have a significant impact on the results of the costing analysis. This editorial describes the importance of the choice of the costing technique and it’s potential impact on the relative cost of treatment options. This editorial also calls for a more efficient approach to healthcare intervention costing in order to ensure the use of consistent costing in the decision-making process.

Keywords::

• Costing
• Bias
• Economic evaluations
• Guidelines

As a result of the financial constraints faced by most health systems today, manufacturers of new, expensive healthcare interventions have to demonstrate value for money1. According to Husereau et al.2, economic evaluation (EE) poses a particular challenge for reporting because substantial information must be conveyed to allow scrutiny of study findings; however, EE poses other challenges. To address one of these issues, country-specific guidelines began in the early 1990s3. Australia was the first country to use pharmacoeconomic studies as part of their formal decision-making processes for new drugs in 1993, followed by Canada, New Zealand, Norway, Finland, Sweden, Scotland, and England1. The intent of the guidelines is to provide guidance to both those conducting studies and those using the results by laying out the general ‘state of the art’ regarding methods, and by providing methodological advice on a range of specific matters. The aim is to improve the scientific quality and integrity of studies, and to enhance consistency and comparability across studies3, without restricting investigators to specific methodologies or techniques4.

According to Barnett5, guidelines are not specific enough with respect to costing methods. Very little guidance has been provided from published sources to assess the costing aspect in health economic (HE) trials which could lead to significant biases. Existing guidelines for EE do not provide sufficient detail regarding methods and techniques appropriate for micro-costing analyses5, and few country-specific database exists to assign unbiased unit prices to each resource utilization or gross-costing methods (such as Case-Mix-Grouper (CMG) and Refined-Diagnosis-Related grouper (rDRG)). The ESALUD database in Spain, NHS reference costs, or the IMSS costos unitarios in Mexico are premier examples of unit cost databases, but are often limited in scope and scale.

As HE gained in volume and interest, credible organizations (such as ISPOR) were formed for educational and scientific purposes. ISPOR is a non-profit organization that has built Task Forces and Special Interest Groups, in developing Good Practices for Outcomes Research (including the use of outcomes research in healthcare decision-making). All 48 expert consensus good practice reports and articles are freely available on the ISPOR website. These reports have been downloaded more than 160,000 times over the past 12 months. Available additional tools include6:

• Pharmacoeconomic (PE) Guidelines Around the World7;

• Global HealthCare Systems Road Map;

• ISPOR Guidelines Index for Outcomes Research and use in HealthCare Decision-Making; and

• The CHEERS checklist for reporting2.

Most of the tools are focused on assessing and reporting outcomes and are designed to support researchers and guide better decision-making. One of the goals of the Canadian guidelines revision was to assess their impact on the measurement of outcomes8. Guidelines have helped standardize methods of cost-effectiveness analysis, allowing different interventions to be compared and enhancing the generalizability of study findings5. Even now, most publications—and ISPOR itself—concentrate on guidelines for adequate outcomes reporting. However, few papers and organizations challenge the process of the significant bias when performing the costing of HE research: estimating costs is an important element in any EE9. Clement Nee Shrive et al.10, Barnett5, and Xu et al.9 state that published guidelines on the conduct of EE provide little guidance regarding the use and potential bias induced by different costing methods. Nonetheless, the US Panel on Cost Effectiveness in Health and Medicine has recommended micro-costing as the preferred approach to cost estimation, because the alternative gross-costing estimation could introduce bias9.

Provided that the underlying assumptions appear credible, most journal reviewers and decision-making groups will approve the underlying costing methods. However, this can significantly alter the cut-off between an efficient vs a non-efficient alternative, even if the underlying assumptions are transparent and methodologically sound.

Heerey et al.11 conducted a number of micro-costing studies of acute myocardial infarction, cardiac failure, and HIV from the hospital perspective. The results of these studies were compared with the costing estimates assigned to hospital admissions based on the DRG system: differences ranged from 9–66%. They concluded that the DRG system provides useful cost estimates for patient admissions in the absence of detailed cost-of-illness data, but recommended that supplementary costing studies be performed for specific therapeutic areas, particularly those for which investigation and/or treatment costs are high.

Clement Nee Shrive et al.10, using micro-costing and two gross-costing methods (CMG and rDRG), compared the cost estimates within and between subjects and determined the impact on results of an EE of sirolimus-eluting coronary stents. Their findings indicated a 4-year mean cost estimates of $16,684, $16,232, and $10,474, respectively. Using Monte Carlo simulation, the cost per QALY gained was $41,764, $42,538 and $36,566, respectively. The authors concluded that, within subject, the three costing methods produced markedly different cost estimates. The difference in cost-utility values produced by each method is modest, but of a magnitude that could influence a decision to fund or not a new intervention.

According to Shrestha et al.12, the Division of HIV/AIDS Prevention of the Centers for Disease Control and Prevention (CDC) spends ∼50% of its $325 million annual HIV prevention funds on HIV-testing services. As an accurate estimate of the costs of HIV testing in various settings is essential for efficient allocation of resources for HIV prevention, they assessed the costs of HIV-testing interventions using different costing methods. Median costs per individual notified of a new HIV diagnosis were $12,475, $15,018, $2697, and $20,144, based on micro-costing-direct measurement, micro-costing-staff allocation, gross costing, and program budget methods, respectively. Compared with the micro-costing-direct measurement method, the cost was 78% lower with gross costing, and 20% and 61% higher using the micro-costing-staff allocation and program budget methods, respectively. They concluded that their analysis showed that HIV-testing program cost estimates vary widely by costing methods. However, the choice of a particular costing method may depend on the research question being addressed. Although program budget and gross-costing methods may be attractive because of their simplicity, only the micro-costing-direct measurement method can identify important determinants of program costs and provide guidance to improve efficiency.

The incremental cost-effectiveness ratio (ICER) can be highly sensitive to the costing method; even if sensitivity analyses are performed diligently. For instance, there is conflicting information on administrative cost for a chemotherapy session for a given code, where the price ranged between $319–$553 in Mexico. In another case related to a micro-costing method, the price of medical resources varied significantly depending upon the source; where the most obvious parameter relates to the price of a hospitalization. In an orthopedic study in the UK, the choice was between the price of a LOS ranging between ₤18813 and ₤43314; one figure was as high as ₤117415, since it also included the cost of re-admissions.

Another issue is that, if the average cost of a hospital stay in one or more given hospital is used, these may not be representative of other hospitals in the same or in other regions. Why, if at all, is overhead cost included in the per unit price? For example, in some countries, the cost of nursing is excluded from the LOS price.

On the one hand, to avoid this type of bias, the DRG/HRG option enables the minimization of exhaustive resource utilization data capture and generates the global cost of an episode of inpatient stay. However, more than one DRG code may apply to a population study (for example, because of variations in age, comorbidity, complications etc.), and these codes may have different reimbursement levels. Thus, which one represents the best proxy for cost? This method assumes that the distribution of the relevant variables in the target population is the same as in the wider population of individuals with the condition, or having the intervention and weighting reimbursement accordingly is one solution, but the assumption may not hold true. On the other hand, the DRG/HRG option is often available for centralized modern healthcare systems, but can be difficult to obtain in newer HTA countries, such as Mexico, Russia, Thailand, and South Korea, or is simply unreliable in some countries such as China.

According to Barnett5, questions are raised for future reviews of the quality of costing methods. The analyst must avoid unqualified assumptions given circumstances, especially those that could bias the analysis by excluding costs impacted by the intervention under study. Hence, the transparency of costing guidelines is essential for decision-makers who require information on the efficiency of a healthcare intervention, because effective decision-making depends largely on its applicability to their settings16. Performing the costing prior to analyzing the database could be a wise guideline to avoid any bias by the researcher to ‘adapt’ the costing slightly, within a non-debatable value, in a manner that it could fall within the perimeter of the cost-effectiveness frontier in order to gain acceptance. Another option could have accredited groups, independent and blind to the EE research, that could be mandated for the costing process.

Given what precedes, we believe the time has come to tackle the issue of costing bias. Indeed, as seen, all costing methods and measures are to some extent flawed. However, we do agree with Fukuda and Imanaka16: ‘(…) there is tremendous room for improvement’. To reduce the significant bias process involved with costing (independently of methods), it is suggested that a broadly-based independent body be invested with the mission of producing a ‘book of knowledge’ of standard micro- and/or macro-costing country-specific values that would be available to researchers and decision-makers worldwide. There are some very practical approaches to address these multi-faceted, systemic, and complex challenges, such as this one which consists in bringing transparency by identifying guidelines. Hence, it is possible to bring in together the relevant EE and HE communities or entities to conduct a collective mapping study17^,18. The purpose would be to identify underlying dimensions and concepts, as well as essential standards, to establish costing guideline transparency and criteria according to which they could be gauged, whether at a national level first, and later on beyond. In particular, the information obtained from such an exercise would be an efficient and broadly-based way to measure emerging consensus amongst EE and HE stakeholders regarding guideline implementation aimed at minimizing costing biases. These individual guidelines could be prioritized according to criteria such as importance, credibility, usefulness, value creating, and feasibility, or, else, as identified by the community. This type of information would become part of the process toward gaining an increased credibility in the costing bias minimization process so paramount in the improvement of the scientific quality and integrity of studies. Moreover, this would contribute to an improved consistency and comparability across studies for clearer decision-making criteria regarding the determination of a more credible cost-effectiveness frontier.

Transparency

Declaration of funding

This manuscript has received no funding.

Declaration of financial/other relationships

GT is an employee of and a consultant for Eisai Pharmaceutical. JF, MC, and LMC have disclosed that they have no significant relationships with or financial interests in any commercial companies related to this study or article. JME peer reviewers on this manuscript have no relevant financial or other relationships to disclose.