Future of quality measurement in the National Health Service

Is the quality of National Health Service (NHS) care better than it used to be? Is quality as high as it could be? Does the NHS offer care of higher quality than the independent sector or health systems in other countries? The straightforward answers are: yes, probably not and it depends what you mean by quality. The latter response points to why things are not so straightforward. Quality is hard to define and harder to quantify, so measuring quality over time, between healthcare providers or across health systems is a challenging task. However, unless effort is directed at meeting this challenge, we cannot be confident that the NHS is delivering the highest possible quality of care.

Confidence is undermined because the stimulus to drive quality improvements is not strong enough. The main thrust stems from the medical and nursing ethic to act in the patient’s best interest, but sometimes this acts more as a safeguard – do no harm – than as a stimulation to provide the highest possible quality of care. And this safeguard sometimes fails: witness the cases of clinical negligence that come before the medical and nursing regulatory bodies [1–3], and the rise in clinical negligence claims [4].

In many sectors of the economy, competition for business is the impetus for manufacturers or service providers to improve the quality of what they offer: if people don’t like the product or service there is a risk that they will go elsewhere. Traditionally, NHS providers have faced limited pressure to compete for patients. Income has not been closely related to how many patients are treated, and patients have rarely had the opportunity to shop around, often because they require urgent care, but also because they lack information regarding their healthcare requirements and whether the quality of service offered by providers differs.

Recent efforts have been made to make the NHS more competitive, in the hope that this will help improve the quality of care. Rather than receiving fixed budgets, hospital income is now related directly to activity, so providers have an incentive to attract more patients because they can invest in additional capacity [5]. Patients are also being given a choice of provider [6]. In the past, NHS patients requiring elective (nonurgent) care simply had to wait until their local hospital admitted them. Now they are offered a choice about where and when they receive treatment, and the options include both NHS (public) and independent sector (private) hospitals. The expectation is that patients will make their decision by weighing up when they will be treated (patients are to be offered a specific date, rather than a vague waiting time), how far they must travel and the quality of treatment.

However, information on quality needs to be improved. At the moment, differential quality is signaled by a star rating system, according to which each provider is awarded from zero stars (poor quality) to three stars (high quality) by the Healthcare Commission, an independent regulator of NHS performance. Similar to Michelin stars for restaurants, providers are rated on the basis of their performance across multiple dimensions, including various measures of waiting time, financial management, cleanliness, booking systems and working conditions [101]. Performance along each dimension is amalgamated into a composite indicator giving a summary score for each provider.

These star ratings have been used mainly for regulatory purposes. Providers that are awarded three stars are able to apply for ‘foundation status’, which releases them from directive management by the Department of Health and gives them more autonomy, particularly over investment decisions. Providers with a low star rating face increased scrutiny and replacement of senior managers.

The star ratings are similar to report cards developed to assess the performance of healthcare providers in the USA [7]. There is some evidence that report cards have stimulated providers to appraise their performance more carefully, but a lack of evidence that many patients refer to report cards when choosing where to be treated [8,9]. Are NHS patients more likely than their more consumer-conscious counterparts in the USA to consider a hospital’s star rating when exercising choice? And if they are, is this only because better information is unavailable?

The problem with star rating is that summarized information regarding the overall performance of a hospital is not sufficiently pertinent to what the patient wants to know. Patients require answers to specific questions about what will happen to them, so that they can decide which treatment option to select and so that they can plan ahead. Such questions include:

•	When will I be admitted?
•	How far away is the hospital?
•	How easy is it to park?
•	How experienced is my consultant?
•	Will I be treated with care and respect?
•	What are my chances of survival?
•	How long will it take to recover?

NHS patients tend to rely on their general practitioners (GPs) for answers. GPs can answer some of these questions quite accurately and the star ratings may provide pointers to others. However, like their patients, GPs lack readily available information to provide much more than fairly general answers to some of the questions. For instance, parents of children referred to the Bristol Royal Infirmary were quoted national survival rates of 80% when they asked about their child’s chances [10]. It is likely they would have chosen to go elsewhere if told that the children treated by the consultant to whom they were being referred had a survival rate of 42%.

The questions asked by patients fall into two broad categories:

•	What is the likely outcome of treatment?
•	What is the process of care like?

Improved efforts are being made in the NHS to shed light on these issues.

Patient-reported outcomes

Health outcome measurement has a long and checkered past. In England, significant progress was made in the 1860s by Florence Nightingale, who encouraged hospitals to publish what happened to all the patients they treated [11]. Outcomes were categorized into three types: relieved, unrelieved and died. For a few years a number of hospitals published this information annually. However, doctors complained that comparisons were inappropriate, as little account was taken of the severity of each patient’s condition. Inadequate risk adjustment continues to bedevil attempts to compare survival rates, but some specialties are willing to tackle the issue [12].

However, survival rates are only part of the story. Less than 3% of NHS patients die within 30 days of their hospital treatment [13]. What type of health outcome do the remaining 97% experience? To what extent, using Nightingale’s words, are they ‘relieved or unrelieved’?

The answer is that we do not really know. No routine data on health outcomes are collected for the majority of patients treated by the NHS. This is not because the technology is unavailable to measure health outcome. Various condition-specific and generic instruments have been developed, but these tend to be applied in the context of clinical trials, which provide financial support to administer and organize data collection. However, fewer than 5% of patients are enrolled in clinical trials [14], and they are not typical of all patients, and insight is usually limited to the relative effectiveness of alternative therapies, not into the quality of NHS treatment in general.

Indications are that this information deficit might be addressed. The British United Provident Association (BUPA), an independent healthcare provider in the UK, has demonstrated the feasibility of routine data collection [15]. Since 1998, BUPA has been administering the short-form 36 (SF36) questionnaire to elective patients prior to their admission and 3 months post intervention. Individual hospitals and consultants are fed back information from BUPA on the changes in SF36 scores for the patients they have treated compared with their (anonymized) peers. The Department of Health is following BUPA’s example by undertaking a similar exercise for five surgical procedures performed by the NHS, independent sector treatment centers and day surgery units [16]. The hope is that this pilot exercise will provide insight into how to roll out the collection of patient-reported outcome measures across the health service in England.

However, it remains to be seen if and how such information will be made available to patients. BUPA has used the exercise to support internal audit, which involves anonymized feedback and only data aggregated by procedure (rather than disaggregated by consultant) being made publicly available [102]. I addition, in the NHS the immediate expectation is that the comparison of outcome data will drive quality improvements through professional critical appraisal, rather than through external pressure exerted by patients armed with consultant-specific success rates.

Patient experience

The other broad dimension that concerns patients is their likely experience of the process of care. Attempts to measure this experience have become more sophisticated over time. In the past, samples of patients or members of the public were asked to rate their satisfaction with the NHS. These measures of satisfaction are problematic. People generally say they are satisfied and, because of this, politicians like to quote such surveys (when they are in power). However, they are not so keen on them when in opposition. And high levels of satisfaction are not necessarily highly correlated with high quality: patients may say that their care was satisfactory even if it was substandard, perhaps because they have no basis for judging the quality of care or because some aspects of the care process outweighed negative experiences. Satisfaction surveys are neither particularly useful for assessing whether care has improved over time – responses may reflect changes in expectations – nor for identifying what action to take – are patients dissatisfied because of the poor bedside manner of the nursing staff or because the hospital food was inedible?

The NHS, like countries such as Australia, Germany, Sweden, Switzerland and the USA, is surveying patients on their experience of care [103]. The survey that has been developed to ask recently hospitalized patients about their inpatient care asks a series of questions that are grouped into eight dimensions (Box 1). These dimensions share some similarity with the set of responsiveness criteria developed by the WHO in their comparative assessment of health system performance (Box 2)[17].

Routine administration of patient experience surveys should foster improvements in the quality of care, provided that:

•	Sufficient numbers of patients are surveyed from each provider and for each condition in order for sound comparative statements to be made;
•	Questions remain consistent over time, to allow assessment of whether things have improved;
•	Providers have incentives to take action in response to the information;

At present, in England, this incentive derives mainly from the contribution that the survey makes to the hospital’s star rating. In future, added stimulus for action may come from making (some of) the information available to patients wishing to choose where to go for treatment.

Conclusion

Thus, is the quality of NHS care better than it used to be? Yes, certainly compared with Nightingale’s day of course, however, there is also evidence of year-on-year improvements. For example, 30-day hospital mortality rates have fallen consistently over the past few years from 3.08% in 1998–1999 to 2.76% in 2003–2004 [13].

Is the quality of NHS care as high as it could be? Probably not, for the simple reason that if inadequate attempts are made to measure at least some of the dimensions of quality, it is difficult to identify and share best practice. Innovation and technical improvement across the health system requires information and incentives for providers to engage in critical appraisal. If the information or incentives are deficient, quality will be pursued and diffused less rapidly.

Does the NHS offer a higher quality of care than elsewhere? Any response to this is hampered by a paucity of comparative data. This was one of the major criticisms of the WHO attempt to compare the performance of health systems – for instance, the WHO relied on ‘expert’ opinion, rather than surveying individual patients to generate data on the dimensions of responsiveness listed in Box 2[17].

Whatever its comparative performance, the NHS has no grounds for complacency. There is too much evidence that people are unhappy about the quality of care they or their families receive from the NHS, all too easily dismissed as anecdotal or as aberrations in the absence of routine data on health outcomes and patient experience of the healthcare process. The NHS has begun to meet the challenge of collecting data on various dimensions of healthcare quality. If the NHS continues to build on these efforts, there is hope for future improvements in the quality of care.

Box 1. Dimension of patient -centered care, National Health Survey in-patient survey.

•	Access, including time spent waiting for admission.
•	Respect for patients’ values, preferences and expressed needs, including involvement in decision making, dignity and autonomy.
•	Coordination and integration of care, including clinical care, ancillary and support services.
•	Information, communication and education.
•	Physical comfort, including pain management and help with activities of daily living.
•	Emotional support and alleviation of fear and anxiety, including impact of illness on self and family.
•	Involvement of family and friends, including social and emotional support.
•	Transition and continuity, including information about medication and discharge planning.

Source: [104].

Box 2. WHO dimensions of responsiveness

Responsiveness: how the health system performs relative to nonhealth aspects, meeting expectations of how people should be treated by service providers.

Respect for persons

•	Respect for the dignity of the person.
•	Confidentiality, or the right to determine who has access to one’s personal health information.
•	Autonomy to participate in choices about one’s health. This includes helping choose what treatment to receive.

Client orientation

•	Prompt attention: immediate attention in emergencies, and reasonable waiting times for nonemergencies.
•	Amenities of adequate quality, such as cleanliness, space and hospital food.
•	Choice of providers, or freedom to select which individual or organization delivers one’s care.

Source: [17].