The PERFECT project: measuring performance of health care episodes

Introduction to the special issue

Performance measurement is an essential element of health care system improvement in terms of public health, medical care, accountability and governance (1). Its role in decision making, planning and resource allocation has increased in many countries. Thus, it is extremely important that various performance indicators are derived using the best possible data and the most appropriate methods.

The measurement of performance and efficiency can proceed at three different levels: system-wide, by disease and by subsector (2). There are pros and cons associated with each but the disease-based approach is the most suitable when health outcomes are the main interest. In subsector analysis (such as in hospital care), meaningful comparative work can be done only if the case mix (patient heterogeneity) can be standardised using the same method.

There are also various approaches to developing information systems for efficiency comparisons. The first approach relies on developing a coherent conceptual framework for information collection, analysis and dissemination. One example of this is National Accounts, in which health care is taken into account as part of the whole economy. Another approach is more opportunistic, seeking merely to assemble readily accessible data, often by-products of existing national data collection, such as hospital discharge registers, as well as work that has been done for other purposes. It is of course helpful to maximise the effectiveness of existing data resources. This bottom-up approach relies on individual experts, provider organisations and countries engaging initiatives to improve quality and efficiency. Micro-level comparative data on clinical actions, costs and outcomes are an essential element of such approaches. In this case the precise definition, collection and scrutiny of the data are left to expert groups to determine (3).

During recent years, many efforts have focused on using health care in national accounts as part of a top-down approach. The current attempts to implement SHA (System of Health Accounts), however, include only expenditure and thus the methodological framework for price and output measurement is still under development (4), and the Atkinson Review (5) have suggested including the quality aspect into output measurements and some illustrative calculations have been done in the UK (6), but there are still many theoretical and practical problems associated with developing outcome-based measures due to the fact that modelling the production of health based on health care consumption involves complex interrelationships. This necessitates the adoption of a broad framework that takes account of the role of social and physical environments as well as the role of genetic determinants (7). The difficulty with a broad framework is that many of its dimensions are not amenable to health policy interventions. One can assume that, at least in the short run, only minor advances can be achieved in developing a measure that can be used to evaluate the impact of the whole health system on health outcomes. Thus at the current stage, the national accounting approach is not suitable for international or national efficiency comparisons.

A microeconomic disease-based strategy offers a supplementary approach that is more suitable to an analysis of health care system performance (8). The disease-oriented approach is based on modelling the natural progress of a disease, with specific interest in the role of health services as a determinant in the disease’s progress (Figure 1). In recent years we have seen a number of international attempts to apply the disease-based approach to analysing the cost and effects of different health system (9–11). However, so far the usefulness of these studies is restricted due to the limitations of the data used. In practice, the usefulness of a disease-based approach is supported by two factors: the availability of comprehensive register data and the possibility of linking different registers and combining information from different sources. The latter requires use of unique personal identification numbers and also the legal authority to perform linkages. Because the Finnish health information system is highly advanced, both of these prerequisites are already fulfilled.

Figure 1. Microeconomic disease-based approach.

The main idea of the approach is that it analyses time trends by using more detailed data pertaining to specific health conditions to illuminate the interconnected aspects (i.e. financing, organisational structures, medical technology choices) responsible for health system performance (i.e. health outcomes and expenditure). The main innovation of the approach is that it will use individual-level data available from registers that allow us to measure the outcome (by following what happens to patients), process (treatment practices) and the use of resources (such as number of hospital days, use of specific procedures and drugs) in selected risk-adjusted and well-defined patient groups. Thus, the interest is in not only a specific treatment or hospital stay (measured e.g. in terms of DRGs) per se, but the outcome and cost related to the whole cycle of care, i.e. episodes that can be defined as series of temporally continuous health service utilisation in response to a specific request by a patient or other relevant entity (12).¹ The idea behind the approach is also stressed in the Tallinn Charter (13), where it is stated that health systems should integrate targeted disease-specific programmes into existing structures and services in order to achieve better and sustainable outcomes.

One of the origins of the disease-based approach is the development of a productivity index for treating specific health problems, as has been done in the USA. The index compares, on an annual level, the value of changes in health status (due to health care) with the costs of producing these health effects (14). For heart attack patients the outcome was evaluated by comparing health gains measured by quality-adjusted life years (QALYs) of similar (risk-adjusted) patients in different years. The approach has also been applied to low-weight infants, depression, cataracts and breast cancer (15) in the USA as well as heart attacks in Canada (16). We will extend the approach to comparing regions and hospitals within a country to reinforce the accuracy and applicability of the results.

The disease-based approach also extends our knowledge on the cost-effectiveness of services. Theoretically, measurement of effectiveness and cost-effectiveness could be based on randomised controlled trials (RCT). However, such studies are expensive, difficult and sometimes unethical. Moreover, RCTs conducted under ‘laboratory conditions’ do not reveal true effectiveness, but only the efficacy of services. The investigation of clinical efficacy might require much shorter follow-up than the investigation of cost-effectiveness. Since the appropriate time horizon for economic evaluations can be longer than the time horizon for an RCT, methods of making projections beyond the trial follow-up period are needed. Moreover, RCTs of health care interventions are often undertaken for a particular target population using strict inclusion criteria and this may make it difficult to generalise the results beyond the trial settings.

Since a single RCT will rarely provide definitive evidence for a policy recommendation, explicit decision-analytic methods are needed to synthesise data from various sources, e.g. RCT data, administrative databases and research literature. To evaluate the performance of the health care system the assessment of cost-effectiveness needs to be extended beyond single procedures and hospital stays to cover the whole service chain, i.e. the episode of care.

As a coordinator of the PERFECT project (PERFormance, Effectiveness and Cost of Treatment episodes, http://www.thl.fi/fi_FI/web/fi/tutkimus/hank keet/perfect) since 2004, CHESS (Centre for Health and Social Economics) has adopted the disease-based approach and has indeed developed protocols for eight health problems (defined by diseases or procedures): AMI – acute myocardial infarction; revascular procedures (PTCA – percutaneous transluminal coronary angioplasty, CABG – coronary artery bypass grafting), hip fracture, breast cancer, hip and knee joint replacements, very low birth weight infants, schizophrenia and stroke. The main aim of the project is to produce comparative performance information for treatment monitoring and development.

The choice of eight health problems was based on the fact that they are all either very common, with a high economic burden, or resource-intensive (such as very low birth weight infants). They also focus on different populations, such as the elderly (AMI, stroke, hip fracture), the middle-aged (breast cancer, schizophrenia), children (infants), males (AMI) and females (hip fracture and breast cancer), are based on different types of expertise (surgery versus internal medicine) and differ with respect to technological change. In addition, the main responsibility for care of all the health problems rests with acute hospital care, which has the best possibilities to apply the adopted approach due to the good coverage of register information.

The development work has been done in seven separate expert groups whose members (some 50) are the country’s leading clinical experts on the previously mentioned diseases. At present, register-based indicators (both on the regional and hospital levels) on the content of care, costs and outcomes between 1998/2000 and 2008 are available for seven health problems. The indicators are freely available on the Internet, and they will be annually updated using more recent information. They have been widely used in local decision making and have also been discussed in the media. In addition, the Ministry of Social Affairs and Health uses the information in its strategic planning: the indicator developed in the project has been used to evaluate the development of regional differences in the effectiveness of specialised care in the National Development Plan for Social Welfare and Health Care 2008–2011.

The main aim of the special issue is to describe the PERFECT project to the international scientific community and others working on the field. Special attention will be focused on both methodological challenges as well as practical solutions for register-based analysis of cost and outcomes. This information is important when the approach is adopted in other countries. We will also discuss the health policy implications of the project. We will concentrate on the five health problems in which the most progress has been made². Each subproject has followed somewhat different directions. For example, the low birth weight subgroup has extended the register-based study by gathering additional survey data, while the hip fracture group has further developed methodology for register-based studies. Thus the chapters on health problems have not been written in an exactly similar manner, and the articles on the subprojects also reflect their own experience.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.

Notes

¹This idea is also recommended to be included in the National Account. For example, Eurostat (4) has defined the output in health care: “Health output is the quantity of care received by patients, adjusted to allow for the qualities of service provided, for each type of health care. The quantities should be weighted together using data on the costs or prices of the health care provided. The quantity of health care received by patients should be measured in terms of complete treatments.”

²In breast cancer the main challenges were related to the poor reliability of data on care given in hospital outpatient departments (radiation, chemotherapy and expensive drugs) as well as problems related to measure outcome (renewal of cancer) whereas in the case of schizophrenia, performance measurement suffers from a lack of data on care given outside the inpatient wards of hospitals (primary outpatient care). In addition, we have excluded analysis considering revascular procedures from this volume since the topic is discussed in the AMI article. However, even for these health problems, performance indicators have been produced and articles published.