The market for ‘evidence’ in policy processes: the case of child health policy in Andhra Pradesh, India and Viet Nam

Abstract

Research on policy processes has emerged over the last 30–40 years in Northern contexts. Such research has expanded into Southern contexts. An interest in the use of ‘evidence’ (such as research) in policy processes is a relatively recent phenomenon. There are, to date, relatively few empirical case studies in developing countries. This article seeks to address this gap by providing a comparative case study of two contexts at the opposite ends of the macro-political spectrum: Andhra Pradesh, India – a free participatory democracy with vibrant civil society – and Viet Nam – a society with, historically, more limited political freedom but with some recently introduced participatory processes and a fledgling civil society. We also consider the ‘international’ policy-making context. Senior policy makers and researchers working in child health policy formation were asked about their perceptions of the use of and quality of ‘evidence’ in health policy processes. It has been argued that greater levels of democratic freedoms are associated with greater use of evidence in policy processes. Our research challenges this and explores perceptions of the nature of ‘evidence’ and its use in policy processes.

La manière dont les politiques sociales sont justifiées est un thème de recherche qui est devenu de plus en plus important au cours des 30-40 dernières années dans les sociétés développées. Cette préoccupation commence aussi à s'étendre aux sociétés en voie de développement, où il existe en particulier un intérêt croissant pour les études qui examinent la manière dont différentes « preuves » – telles que la recherche – sont utilisées afin de justifier la mise en oeuvre de certaines politiques de développement. En même temps, il existe peu d'études empiriques sur le sujet, et cet article propose donc une étude comparative de deux cas de pays en voie de développement qui se situent aux deux, extrêmes, politiquement parlant, c'est-à-dire l'Etat indien de l'Andhra Pradesh, qui peut être caractérisé de démocratie ouverte et participative ayant une société civile vibrante, et le Vietnam, une société historiquement moins ouverte, mais dont le système politique a néanmoins récemment commencé à évoluer avec l'introduction de mécanismes de participation politique ainsi que l'émergence d'une société civile indépendante. L'étude prend aussi en compte le contexte international, et se focalise sur la perception qu'ont les décideurs politiques ainsi que les chercheurs, dans ces différents contextes, de l'utilisation, ainsi que de la qualité, des « preuves » utilisées tant dans l'élaboration que la justification de politiques de santé infantile particulières. Bien qu'il soit très commun d'associer de plus grandes libertés politiques avec une meilleure utilisation de « preuves » dans l'élaboration et la mise en œuvre de politiques sociales, l'étude présentée suggère que ce n'est pas nécessairement le cas.

Keywords:

• policy processes
• child health
• India
• Viet Nam

Acknowledgements

For comments on earlier drafts we would like to thank an anonymous referee, Andrew Crabtree, Julius Court, Sharon Huttley, Nicola Jones, Simon Maxwell, Tran Tuan and John Young. This research was funded by a UK HEFCE Promising Researcher Fellowship at London South Bank University.

Notes

1. Indeed, policy processes involve various stages – agenda setting, formulation, implementation and so on – in what is increasingly recognized as an often highly iterative and non-linear process. Though linear or ‘stagist’ models of policy may be discredited, their use is defended by some as a heuristic device (see discussion in Sabatier and Jenkins-Smith 1993). An alternative to policy stages are ‘policy spaces’ (Brock, Cornwall, and Gaventa 2001; Gaventa 2006). This might be seen as an alternative to the stagist approach noted previously. These are spaces in which policy is discussed by some or all actors, depending on the space type (spaces may be closed, invited, claimed/created, visible, hidden, invisible and global, national, local). Such spaces may be conceptual in nature – i.e. conceptual spaces – where new ideas can be introduced into the debate and circulated through various media, or bureaucratic – i.e. bureaucratic spaces – formal and informal policy-making spaces within the government bureaucracy such as poverty reduction strategy papers (PRSPs), or popular in nature – i.e. popular spaces – protests and demonstrations that put pressure on governments.

2. Alternative definitions for comparison can be drawn from the health policy literature – notably Buse, Mays, and Walt (2005, pp. 4, 5, 8, 158).

3. For example, the older rational models noted (e.g. Lasswell 1951), bounded rationality models (e.g. Simon 1957), incrementalism and/or disjointed incrementalism models (e.g. Lindblom 1959), middle ground or mixed scanning models (e.g. Etzioni 1976), garbage can theories (e.g. March and Olsen 1976), argumentative models (e.g. Fischer and Forester 1993), interceptor/receptor models (e.g. Hanney 2005), the three inter-connecting streams model (e.g. Kingdon 1984), the ladder of utilization and receptors receptivity model (e.g. Knott and Wildavsky 1980), the interactive or problem solving/engineering models (e.g. Grindle and Thomas 1991), the structuration model (e.g. Keeley and Scoones 2003), and the Research and Policy In Development (RAPID) research-into-policy model (Crewe and Young 2002). For review of frameworks see Sutton (1999).

4. See for discussion (Martson and Watts 2003, p. 150). Upshur et al. (2001, p. 94) proposed a model of evidence with four distinct but related types of evidence in four quadrants for what kind of research is seen as credible in different disciplines. The vertical axis is methodology – meaning to measurement and the horizontal axis is context – particular to general context. The four were qualitative personal (concrete/historical), qualitative general (concrete/social), quantitative personal (personal/mathematics) and quantitative general (impersonal/mathematics). They argued that each of these dominated in different disciplines. The first in clinical medicine, the second in social sciences, the third in clinical epidemiology and the fourth in economics and political science

5. Davies (2004, p. 7) argues there are six types of evidence (research evidence; systematic reviews; single studies; pilot studies and case studies; expert's evidence; and Internet evidence) and seven types of research evidence (attitudinal evidence – surveys, qualitative; statistical modeling – linear and logistic regression; impact evidence – experimental, quasi-experimental, counter-factual; economic and econometric evidence – cost-benefit, cost effectiveness, cost utility, econometrics; ethical evidence – social ethics and public consultation; implementation evidence – experimental, quasi-experimental qualitative, theories of change; and descriptive analytical evidence – surveys, admin data, comparative and qualitative).

6. The criteria for inclusion in the systematic review were as follows (p. 240): studies that included interviews with health policy decision makers; studies that included health policy formers responsible for decisions on behalf of a large organization or jurisdiction; if others were interviewed – most often researchers – decision makers had to be explicitly defined as a sub-group within the study; studies of clinical decision making for individual patients were excluded; the studies had to address decision makers' use of research evidence in health policy decisions or on a broader range of policy decisions if these included health policy decisions.

7. The notion of ‘different worlds’ between researchers and policy makers is not a new one. Innvear et al. (2002, p. 242) citing Caplan, Morrison, and Stambaugh (1975) sums it up thus, ‘social scientists seem themselves as rational, objective and open to new ideas; they see decision makers as action- and interest-orientated, indifference to evidence and new ideas. Decision makers on the other hand, see themselves as responsible, action-orientated and pragmatic, they see scientists as naïve, jargon driven, and irresponsible in relationship to practical realities’. One might argue with the emergence of the research-into-policy body of literature noted in the text, although different worlds may still exist there is some recasting happening. The exact nature of the change is though unclear.

8. All interviewees received questions before their interview and transcripts after for amendment as well as this paper for comment and feedback. The sampling was as follows: There are 3 contexts and 4 stakeholder groups. A sample frame of ‘key informants’ was constructed in each of the 3 contexts. In the ‘international’ context this was be done by the authors. In AP and Viet Nam this was done by the country partners who work on child health policy research (Save the Children, UK in AP and the Research and Training Center for Community Development, Viet Nam). Response rates in each category were over 60%. Interviews were anonymous. We asked participants for short biographies to cross-check we were interviewing a sample representative of the key decision makers. Although it can be argued politicians are the decision makers we are concerned with policy-formation and thus we targeted senior civil servants.