Anthropology and impact evaluation: a critical commentary

Abstract

Quantitative and quasi-experimental methods have become popular in the evaluation of development impact. In response, several commentators have argued for more effective use of ‘mixed methods’. This paper engages with, and builds upon, this current criticism of more quantitatively based impact evaluation from the disciplinary perspective of anthropology. Focusing on one specific evaluation, of an irrigation project in Malawi, it asks what was missed and what was misunderstood in the quantitative focus that was adopted. The paper then reflects on the wider question that is raised of how particular methods and perspectives can take centre stage and produce apparent ‘truths’ even in the face of evidence pointing in opposite directions. The overall argument is that this is a matter of the politics of knowledge production and of how particular disciplinary perspectives may come to dominate.

Keywords:

• impact evaluation
• anthropology
• irrigation
• Malawi

Notes

1. As opposed to the slightly different question: ‘was what was planned done?’ Much development evaluation in the past has focused on this, rather than impacts, understood in terms of overarching goals rather than achievement of planned outcomes.

2. ‘Innovations to Promote Growth Among Small Scale Irrigators’ DFID/ESRC Growth Research Programme ES/J009415/1.

3. For the purposes of the study, youth are regarded as ‘those beneficiaries less than 35 years of age’ (Nkhata, Jumbe, and Mwabumba 2014, 2).

4. Low-income households are defined as those earning less than US$1.25 per day.

5. This is in an ideal world of course. There are plenty of examples of ethnographic fieldwork that is also lacking in transparency or contains biases that are underacknowledged or even considered.

6. There is no space to go into this here. Influential works include Agarwal (1997), Harris (1981) and Whitehead (1981).

7. Again, the literature is extensive. See Chant (2007).

8. And for whom the separation of the professional and the personal is arguably both legitimate and necessary.