In search of results: anthropological interrogations of evidence-based global health

Since the 1990s, we have witnessed the rise and prominence of a ‘new regime of truth’ legitimated in the currency of empirically based quantitative evidence (Lambert, 2009, p. 17). This shift has had a profound impact on how we design and evaluate social policy and practice. In the field of global health, in particular, the authority of expert knowledge has ceded to a proliferation of enumerative techniques that dictate the parameters for what constitutes valid and invalid evidence for guiding the architecture of crucial health programs and policies. These technologies of measurement have been realigned to match the logics of economic accountability more than community needs, thus reconfiguring how health is conceptualized and approached around the world. This subsequently affects the interventions designed to respond to current global health problems, sometimes in unanticipated ways. As a result, such techniques not only instantiate new forms of governance and sovereignty (Adams, 2016a; Hacking, 1982), but also shape our view of what counts as an outcome, and guides the way we validate those outcomes.

This special section brings together a collection of papers that interrogate evidence itself as an object of inquiry and examines how different forms of evidence are produced, negotiated, and valued across a range of health contexts. In unpacking evidence and the practices implicated in its production, these papers explore the health interventions mobilized in global health from within the positivist methods that dominate its discourse, attending specifically to the blind spots that sometimes emerge from the limited purview of what is recognized as evidence. In particular, all the papers foreground the business rationalities, measurement technologies, funding structures, and regime of actors and agendas that distinguish global health from its predecessor international health (see Brown, Cueto, & Fee, 2006) and its counterpart public health, both of which are bound by national governance and financing. Our papers thus aim to unravel that ‘obscure object of global health’ (Fassin, 2012) that seems to be less about the spatial and geographic boundaries that determine health and more about the financial and political arrangements through which health practices (and inequalities) are configured. The primary concern is often who is setting the standards for measuring health and how these forms of knowledge circulate in ways that shape the health outcomes themselves.

Our papers aim to interrogate the processes that guide this new practice of global health, thus highlighting the ‘perceptual deficits’ (Biehl & Petryna, 2013, p. 5) of paradigms that tend to occlude the complexities and contradictions of health issues in favor of standardized models that seek proof in the language of metrics. Collectively, these papers suggest that a view beyond the evidence reveals programs that struggle to reach their intended beneficiaries and goals, despite the fact that they find ways to demonstrate success. This is often the unintended consequence of a discourse that privileges a specific iteration of the ‘right’ standards and the ‘right’ outcomes, defined as calculable, replicable, and generalizable evidence. These limited parameters may demonstrate success, but more often than not they elide the realities of the people whose lives are at stake. If, as Janes and Corbett (2009) propose, anthropology’s engagement in global health should focus on the ‘instantiation of global assemblages in local social arenas,’ (p. 169) then these papers reveal the challenge for those working in global health to negotiate with the evidence produced as it intersects with broader structures of knowledge and power.

The papers in this section engage both the politicization and depoliticization of evidence (Adams, 2013), echoing concerns over which types of evidence certain stakeholders privilege and why (Nichter, 2013). In this regard, all the papers illustrate the intentionality behind the production of evidence, in that it is always ‘of or for something’ (Csordas, 2004, p. 475). This engagement comes at an especially critical time, given the growing influence of actors like the Bill and Melinda Gates Foundation; the forging of new public–private partnerships such as the Global Alliance for Vaccines and Immunization (GAVI) and the Global Fund to Fight AIDS, Tuberculosis and Malaria; and the proliferation of new economic arrangements with non-governmental organizations that are reshaping the global health landscape through practices that seemingly prioritize results over people. The increasing authority accorded to evidence, broadly interpreted and problematized, further entrench such ‘audit cultures’ of accountability (Strathern, 2000) in ways that justify certain large-scale programs while excluding others. At the same time, these methods have created new regimes of experts embodied not in the figure of the medical authority but in that of the business professional armed with the language of market logics (see Erikson, 2016). These new forms of expertise are particularly salient in a context that increasingly turns to metrics as an authoritative source of knowledge and scientific validity.

The turn to evidence-based approaches and its ‘trust in numbers’ (Porter, 1995) in global health has emerged, in part, as a reaction to the turn toward evidence-based medicine (EBM) and randomized controlled trials (RCTs) and their production of scientifically valid results as the gold standard for evidence of efficacy (see Bell, in press for a more in-depth analysis of the history of EBM). First introduced by a group of scholars at McMaster University (Lambert, 2006; Mykhalovskiy & Weir, 2004), EBM ascended in its claim to medical authority on the strength of statistical evidence used to inform decisions around the best available treatment (De Vries & Lemmens, 2006). It is an approach that promised to create standards across all phases of the ‘production, distribution, and consumption of evidence’ (Ecks, 2008, p. S81) grounded in ‘legitimate’ forms of statistical results. This turn launched a whole industry of evidence-based programs and research that evaluates program effectiveness, the results of which are often housed in vast repositories like the Cochrane Collaboration’s evidence reviews. Despite the claims found in this body of literature, the statistical strength generated through the methods they rely on, like RCTs, often have little ability to ensure that the same outcomes are replicated from one patient to another, or across different contexts that cannot be controlled experimentally (Goldenberg, 2006; Lakoff, 2007; Lambert, 2013), thus calling into question the meaning and generalizability of the ‘evidence’ in EBM.

The increasing reliance on ‘objective’ statistical evidence signals, in part, a declining public trust in professional experience and authority and a growing need for more transparency and accountability (Lambert, 2006; see also Power, 1997; Strathern, 2000), in conjunction with empirical proof that promises efficacy of a particular intervention. As the papers in this special section demonstrate, it is this demand for accountability that shapes the very kinds of evidence required to demonstrate impact to the international set of actors who now support the practice of global health (Adams, 2010). This model reinforces the use of objective evidence, which has marginalized other forms produced from individual experiences. This bias toward quantifiable results serves as a marker of effectiveness and redefines the boundaries of what gets counted (Mykhalovskiy & Weir, 2004). As our papers foreground, decisions in global health are also now largely driven by the politics of knowledge production and claims to authority over knowledge.

The addition of health to the World Bank’s portfolio foreshadowed a shift toward a new direction in global health that prioritized the economic benefit of interventions. In its World Development Report: Investing in Health (1993), the Bank emphasized the need for more efficient use of public sector spending and called for a reduced role for governments in health provisioning, favoring instead the greater participation of the private sector (see also Brown et al., 2006). The addition of a rational economic model for treating and intervening in the health of resource-poor countries resulted in an approach where counting health in terms of cost often seems to take priority over enhancing health outcomes independent of cost. Using this approach, health has become a matter of an organ or a discrete disease rather than that of an overall person or system. And, as Adams (2010) has shown, health is accounted for as long as that organ is healthy and functioning, regardless of the fact that the person may die. Implicit in this change was the displacement of states as the primary stakeholders responsible for health in favor of global actors such as the Bill and Melinda Gates Foundation that operate across and between borders. These shifts also resulted in the mobilization and circulation of ‘global best practices’ that adhere to universal standards of health interventions over community-motivated approaches that respond to local contexts, as Uretsky (this section) illustrates in the case of HIV prevention among injecting drug users in southern China. Such ‘best practices’ are mobile and transposable across borders and thus considered more efficient. These ‘humanitarian kits,’ as Peter Redfield (2008) describes them, also offer quick technical fixes that have become part and parcel of donor requirements, despite the fact that evidence-based approaches often remove the human element from programs that offer treatment and care (Cronje & Fullan, 2016).

This move toward accountability in development is no better illustrated than with the Millennium Development Goals (MDGs). Sponsored by the United Nations, the MDGs are a set of eight measurable and quantifiable targets intended to address components of extreme poverty. MDG 5 (now included under Sustainable Development Goal 3), for example, targets maternal mortality, which is seen as an indication of a country’s poverty level, given that elevated rates of maternal mortality are often found in the world’s poorest nations. But as Storeng and Béhague (this section) illustrate, most countries targeted by MDG 5 lack the infrastructure for registering births and deaths from which to generate data, but are consistently expected to deliver the necessary indicators that prove they are deserving of the donor commitments that are available to tackle this major development challenge. This leaves us to question the effect of such standardized goals developed with a certain type of evidence in mind.

The MDGs, not unlike EBM, are dependent upon a linear measurement of indicators that elide the structural inequalities from which maternal mortality arises. Consequently, what gets counted is not necessarily what actually counts, or matters, to the women who make up the faces of the global challenge of maternal mortality. Instead, MDGs and their indicators tend to look past those faces and their voices precisely because the metrics used to determine funding needs are often disconnected from local needs. This can result in programs that serve the exigencies of their global donors but neglect those of their intended beneficiaries. Similarly, our thirst for big data to prove the effectiveness of interventions can result in overlooking other factors that contribute to poor health, as has happened with the HIV pandemic. Global health donors continue to prioritize HIV/AIDS in their funding to the neglect of many emerging and endemic diseases (Shiffman, 2008), leaving countries searching for or creating narratives around local HIV epidemics in hopes of attracting international donor funds (see Benton, 2012). Take, for example, the control of sexually transmitted infections (STIs) among pregnant women in China. A total of 10,757 cases of syphilis were reportedly transmitted from mother to child in China in 2009, while there were 57 cases of HIV transmitted from mother to child that same year. The national response to these two public health problems, however, was shaped by a global agenda that is alarmed over mother-to-child transmission of HIV, but neglects mother-to-child transmission of STIs, resulting in little attention to or treatment of the problem in a country striving to find its place on the global stage (Wu & Hawkes, 2015).

That the Gates Foundation is one of the largest funders in global health, supporting institutions ranging from GAVI to the Institutes for Health Metrics and Evaluation, foregrounds the shifting terrain that intertwines evidence and profit (Adams, 2016b; Erikson, 2016). The assumption that underscores many of these initiatives is that ‘moneymaking – and only moneymaking – will motivate people to fix global health problems’ (Erikson, 2016, p. 151). The bottom line, then, is not on health per se, but on generating outcomes that justify the return of profit from interventions and technologies in such programming (Adams, 2016a, p. 42), such as the emphasis on medical technologies and treatment in global health programs that seem to benefit pharmaceutical companies (McCoy, Chand, & Sridhar, 2009). This turn is perhaps best exemplified in the rise of performance incentives that tie funding to the production of measurable targets and indicators to improve health outcomes (Eichler, Levine, & The Performance-based Incentives Working Group, 2009; Low-Beer et al., 2007). Evidence must no longer be simply scientifically legible; it must also be economically legible. This contingency upon the production of evidence and its accountability has tremendous implications for the provision and execution of health services, as Fan (this issue) illustrates in the case of performance incentives used to scale-up HIV testing in China.

The collection of papers in this special section aims to unsettle the presumed objectivity of evidence and the means by which it becomes a measure of accountability. In doing so, we specifically question what gets excluded in the pursuit of a particular kind of evidence. More importantly, we question the way evidence works to render both visible and invisible certain populations (Sangaramoorthy & Benton, 2012). In this regard, how evidence is counted or discounted is as important as what it is intended to do. To be sure, disaggregating the intentions of evidence also allows us to interrogate the kinds of evidence that are needed and, in turn, directs how they are produced (Lambert, 2013, p. 44). This disarticulation is especially salient in global health because of the presumed sovereignty of evidence that threatens to obscure the flexibility with which data is interpreted and how it is transformed into evidence to do things, such as govern populations – as Lorway (this section) demonstrates through HIV mapping in his paper.

All the papers in this section respond to the current moment in global health, which, as we see it is defined more by the demand for evidence than service and aid to the community. Yet the two are not so disparate, as Sullivan (this issue) illustrates in her paper, where documents serve as a form of currency that consolidates the demand for evidence with the ‘needs’ of a community. Of particular significance in this turn is the predominance of profit, not people, which has come to direct the allocation of resources and formation of health policies. Therefore, instead of taking evidence as a given, these papers illustrate the flexibility with which this term is appropriated, adapted, negotiated, and deployed, revealing the very processes of evidence-making as both artificial and an artifice. But it is an artifice that must be taken seriously, precisely because ‘any domain of quantified knowledge ... is in a sense artificial. But reality is constructed from artifice.’ (Porter, 1995, p. 5) In this regard, we demonstrate that what matters is not just what the numbers tell (or don’t tell) us, but also what they can do in terms of legitimizing evidence.

Overall, the papers in this section present a broad picture of the way evidence is being framed in current global health practice, and how that framing affects the implementation of programs and the communities that are the intended beneficiaries of these programs. For example, Fan examines how the utilization of performance-based financing mechanisms to scale-up HIV testing in men who have sex with men (MSM) as a key HIV intervention in China reshapes not only how HIV interventions are carried out, but also what counts in these programs. Evidence, in this context, becomes intertwined with funding such that it directs both what gets counted and how. As a consequence, the production of measurable results often becomes an end in itself at the expense of HIV prevention. Fan’s (2017) paper raises crucial questions about the need for economic legibility and accountability in global health that comes to bear on the lives of the very populations such interventions are intended to save.

It is the question of who counts, or rather who is rendered visible, that is foregrounded in Lorway’s article, which is one of two articles in this section that discusses the effects of the often onerous expectation for standardized reporting. In contributing to this discussion, Lorway examines the deployment of tracking sheets in relation to the contemporary life history of HIV interventions in India. The paper demonstrates how these forms and the evidence they produce have become a type of ‘evidentiary sovereignty’ that creates a juxtaposition and opposition between the monitoring and evaluation experts who impose the forms and the local actors required to fill them out. From the perspective of the global monitoring and evaluation experts who depend on metrics, these forms represent liberation for the local communities, who, on the other hand, see the forms as the oppressors of their freedom.

If Lorway’s (2017) article illustrates how evidence renders population groups visible through HIV mapping exercises, then Sullivan’s (2017) contribution calls forth what is rendered invisible and unknown through data-making practices, and the ways in which evidence both governs and becomes governable. Drawing on the concept of ‘unknown knowns’ (Geissler, 2013), she reveals the processes through which evidence production is actively ‘black-boxed’ in order to assure accountability to multiple audiences. From this process, rendering evidence is less about measuring health outcomes than it is about assuring accountability. What becomes especially salient in Sullivan’s piece is the way in which donor priorities drive the production of evidence that demonstrates success.

This same theme lies at the crux of Storeng and Béhague’s (2017) paper about maternal health and advocacy. Their paper brings us into the worlds of the stakeholders that have become a crucial population in the field of global health connected with producing ‘valid’ indicators of health. This brings into question methods of how data are made and made meaningful. On the one hand, stakeholders recognize the limitations in the production of evidence on maternal mortality ratio (MMR), a key indicator in measuring maternal health; on the other hand, the push towards results-based advocacy demanded by donors has raised the stakes for generating the ‘right’ numbers. And this is where Storeng and Béhague articulate the significance of such debates. These contestations become crucial for what they can do for MMR and what they cannot do for donors. The stakes, then, are about not what the data means, but how activists make them meaningful to the people who matter. And in the end, the people who matter are not the women who benefit from such interventions, but the donors who fund them. As with the other papers in this collection, the authors foreground the disarticulation between evidentiary regimes and lived experiences.

It is this disconnect between the needs of a community and the needs of a donor that Uretsky (2017) discusses in her article on HIV programs in southwest China. Looking at the ‘mobile sovereigns’ of global health who carry a global currency of prevention strategies and treatments grounded in the universal standards of scientific evidence, Uretsky examines the impact of evidence-based science on the effectiveness of global health programming. She uncovers a large disconnect in HIV interventions that elides local needs and ultimately disrupts the mission of the interventions she examines, despite the fact that their designers are still able to make the metric-driven claims demanded by their donors.

Overall, the papers in this special section provide a variety of case studies from different ethnographic contexts that demonstrate the challenge that local communities face in fulfilling the expectations of mobile global health sovereigns who ultimately wield authority and influence over the way they conduct and respond to health. It is a voice that is difficult to resist because it is paired with the power of financing that many communities have come to rely on – funds that are contingent on the authority of science and metrics to provide the evidence needed to justify the provision of such funds, even if at the expense of the communities themselves.

Elsa L. Fan
Department of Anthropology & Sociology, Webster University, St. Louis, MO, USA
Elanah Uretsky
Department of Global Health, George Washington University, Washington, DC, USA
[email protected]