Rating and Evaluating Health Systems: Reinforcing the Partisan Spirit or Investigating the Public Good?

Abstract

This essay is methodological: whether and in what ways cross-national ranking and rating of public services meet the conditions for effective cross-national learning. Part I makes the argument that such learning requires causal understanding of the mechanisms underlying ranking scores. Part 2 defends the claim that causal understanding requires grappling with the production functions involved in public services. Part 3 suggests that health's dominant features makes effective ranking and cross-national comparison exceedingly difficult. Part 4 describes and criticizes the most widely disseminated attempt to rank and rate the efficiency of different health systems: the World Health Organization's study of 2000. Part 5 explores a contrasting, but more reliable indicator of health status: life expectancy.

Acknowledgements

I thank Emma Orgeret and Christopher Hood for their English translation of the original version of this paper, and Theodore Marmor and Christopher Hood for their for their comments, suggestions and editorial advice. I am deeply grateful to the anonymous reviewers for their valuable comments.

Notes

1. The British figure is 8 for 100,000 inhabitants.

2. In 1985 alone, when the author was the head of the French hospital system at the Ministry of Health, he got 2650 political recommendations drawing his attention to particular cases.

3. France is a country where the suicide rate is high (20 per 100,000 inhabitants per year, compared to only 8 per 100,000 in the United Kingdom or Italy).

4. This laboratory example illustrates that a sector like health does not have a uniform organizational type. Instead, the claim is that organizations in health/healthcare can be categorized in one of these four types. I owe this qualification to one of the anonymous readers.

5. For instance in France, since the reign of King Philip the Fair (1285–1314), every public servant has been considered potentially dishonest. The person who authorizes a payment must therefore not be the same as the person who actually makes that payment, and this rule has become almost universal in public bureaucracies.

6. For example in France 100,000 employees of the health insurance organization spend their lives “controlling” the rights to which every legal resident of the country has been entitled since 1 January 2000. The law changed so that it is now simply residence that counts. Before that date the right to health insurance had been linked to employment (meaning that your entitlements depended on whether you were a salaried employee, a farmer or self-employed). All the former controls derived from this employment relation. They still survive because they remain the raison d'être of different organizations. But the employment part of the checking system no longer has any operational importance – except to explain to survival of these expensive bureaucracies (costing some [euro]8.4 billion a year).

7. “We define the quality of treatment as the level of the characteristics valued by patients and changes in quality are measured as the rate of changes of these characteristics.”

8. A phrase coined by one of my colleagues, Claude Riveline, professor at l'école des Mines de Paris.

9. In Knock ou le triomphe de la médecine, a famous play by Jules Romains (1923), the main character defines health as “a temporary state between two diseases which doesn't bode well”. A critical view of the WHO definition of health is not new; there are many examples in the literature, as there are many examples of disputes about the degree to which medical care interventions improve a population's health status. For a comprehensive review of those issues, see Evans et al. (1994). See also notes 15–19 below. This paper addresses the validity of the approaches to measuring health. It does not seek to explain the reasons various approaches have been initiated, though one recognizes that there are many defensible justifications for innovating here, given the multiple dimensions of health, the lack of critical data, and thus the multiple ways by which health or medical care programs and policies could be evaluated. I am indebted to one of the reviewers for clarifying this point.

10. Loi du 9 août 2004.

11. DREES: “la Direction de la recherche, des études, de l'évaluations et des statistiques”.

12. For instance, “Goal 91: a reduction of 30% by 2008 in the mean of the CAO dental index at the age of 6 (from 1.7 to 1.2), at the age of 12 (from 1.94 to 1.4) and among adults (from 14.6 to 10.2 for those aged between 35 and 44 and 23.3 to 16.3 for those aged between 65 and 74).”

13. Robert Fetter was then a professor in the School of Organization and Management at Yale University. Others have explored at length the context in which DRGs have been used, including the extent to which in the American case, as one reviewer noted, DRGs were employed to replace a “reasonable cost” standard of payment and not to measure the impact of hospital care on the health of elderly Americans.

14. An example of a Type 3 case would be the unknown prevalence of vCJD in the population, a disease for which, in any case, no treatment is known.

15. And life expectancy was 14 years less in 1950.

16. Computation taking into consideration the 2002 purchasing power parities, source OECD.

17. Not exactly Russia but USSR.

18. The population of the United States reached 300 million inhabitants in October 2006.

19. In his article, Jean-Claude Chesnais notes that in the United States, the homicide rate by “colored” people was 37.5 compared to 5.6 for Whites. Violence always hits the weakest.

20. The number of male births went down from 1.25 million in 1987 to 630,000 in 1999.

21. Comparisons can also be used to challenge equally entrenched views, such as the commonly believed proposition that people will consume less health care if they have to pay more from their own pocket. Yet if we examine health spending and its rate of growth in different countries since 1990, we find the most expensive and inflationary countries are those in which private insurance plays an important part (United States, Switzerland), or at least a significant one (Germany, France). International comparisons like the demonstration of the fact that tax-financed health systems cost less than those financed by social contributions, are potentially very useful in knowing at least what should be avoided. Canada seems to have bravely learnt from that lesson, but big reforms such as those undertaken by that country nearly 30 years ago are few and far between (see Evans et al. 1994: Ch.8).

Additional information

Notes on contributors

Jean De Kervasdoué

Jean de Kervasdoué is Professor of Health Economics and Management at the Conservatoire national des arts et métiers in Paris (France). He has experience in academia, government and in the private sector and was for five years (1981–1986) in charge of the French hospital system at the national level. He has published 13 books and is an international expert on health policy and management. He is the co-director of “Pasteur – CNAM” school of public health which was created in 2009.