Evaluating the long-term impact of anti-poverty interventions in Bangladesh: an overview

Abstract

This paper provides an overview of a research project that assessed the long-term impact of three antipoverty interventions in Bangladesh – the introduction of new agricultural technologies, educational transfers, and microfinance – on monetary and non-monetary measures of well-being. It begins by setting out the conceptual framework, methodology, and empirical methods used for the evaluation of long-term impacts. It discusses the context of the evaluations and the longitudinal data used. Key findings from the papers are then presented, followed by an indicative analysis of the cost-effectiveness of these interventions. The overview concludes with implications for programmes and policy.

Keywords:

• Bangladesh
• long-term impact
• evaluations
• mixed methods

Acknowledgements

This research was funded by the UK Economic and Social Research Council and Department for International Development under their Joint Research Scheme (Award Number RES 167-25-0361), building on a longitudinal dataset whose collection was funded by the Chronic Poverty Research Centre, HarvestPlus, and the University of Waikato. The authors thank DATA Ltd., for their substantive cooperation throughout this project, Dan Gilligan for invaluable advice on methodology, and an anonymous reviewer for helpful comments. All errors and omissions are theirs.

Notes

1. Initial assessments of the impact of microfinance on poverty reduction have probably provided an overly optimistic view of its development impact. Morduch (1999) reviews the cross-country evidence on the microfinance ‘revolution’ and finds that issues of appropriate mechanism design, savings mobilisation, financial sustainability, and scale remain. Roodman and Morduch (2009), using the same data as Pitt and Khandker (1998) and Khandker (2005), find that evidence for impact is weak, mostly because these studies are unable to convincingly remove the effects of endogeneity. Duvendack and Palmer-Jones (2011) apply PSM to the Pitt and Khandker (1998) data, differentiate outcomes by the gender of the borrower, and take into account borrowing from several formal and informal sources, and find that the mainly positive impacts of microfinance observed by previous studies are highly vulnerable to selection on unobservables.

2. While the King and Behrman (2008) paper was written in the context of evaluating social programmes, it is also relevant to programmes disseminating agricultural technologies.

3. See, for example, Todd (2006) on the difficulty of maintaining treatment and control groups over much shorter periods in the PROGRESA evaluation in Mexico.

4. We present details on the precise proceeding used for balancing, matching and trimming the study data in each of the quantitative papers in Appendix 1.

5. Steps involved constructing the matched sample (including estimating the propensity score, trimming the sample to ensure common support for the treatment observations in the comparison group, and testing that the balancing property is satisfied) are discussed in Appendix 1.

6. An alternative estimation approach would have been instrumental variable estimation, but this requires the identification of suitable instrument variables that are highly correlated with the treatment but uncorrelated with the outcome variables. Because of the nature of the interventions and programme eligibility criteria, it was not possible to identify convincing instruments for the agricultural technology, microfinance and education transfer interventions.

7. Since the interviews were conducted by a professional survey company (DATA Ltd) supervised by an international expert (Davis), it is also unlikely that many respondents identified them with the interventions under consideration. (Indeed in one village in Nilphamari, the qualitative interview team were initially mistaken for Christian missionaries!)

8. Households with more than one primary school age child receive 125 taka per month.

9. The term ‘process tracing’ comes from the political science literature, where it refers to peering ‘into the box of causation’ (Geering 2005, 2007, George and Bennett 2005).

10. See http://www.ifpri.org/dataset/chronic-poverty-and-long-term-impact-study-bangladesh

11. For example, the fisheries extension programme in Mymensingh originally provided credit directly to farmers, and only used NGOs as a conduit for microfinance services much later. Both the improved vegetables programme and the food-for-education programme involved the private sector in its activities – using the private sector to set up marketing channels in the former, and relying on private grain dealers to distribute food in the latter.

12. By the time of the re-survey in 2006–2007, the technologies had diffused into all survey villages and had been adopted even by households who were not programme members at baseline.

13. Indeed, qualitative assessments reported in Hallman et al. (2007) showed that women derived social benefits from sharing vegetables with their neighbours, who may have then adopted these varieties. It is quite possible that this sharing also involved the dissemination of technical knowledge, so later adopters would have learned from the experience (and mistakes) of earlier adopters.

14. The dissemination of the polyculture fish technology through group fishponds in the Jessore area resulted in minimal effects on household expenditures in both the short term and the long term. This is due to the costs and benefits of the technology having been diluted over a number of families.

15. The analysis in the Quisumbing and Kumar paper compared changes in husbands’ and wives’ asset holdings across households as well as for husband–wife pairs within the same households. The second comparison also finds that women's assets increase when technologies are dissemination through women's groups, and are therefore robust to time-invariant unobservables at the household level that have time-invariant effects.

16. We use the rural Consumer Price Index published by the Bangladesh Bureau of Statistics to deflate all nominal values to convert them into real terms reflecting 2007 prices.

17. A child is classified as stunted if his or her height-for-age z-score is less than  − 2 using the new World Health Organisation growth charts.

18. It is impossible to evaluate this against a true counterfactual of targeting nutritional advice to men, because men are rarely (if ever) targeted for nutritional advice in Bangladesh. However, it is probably safe to say that untargeted agricultural programmes, such as the individual fishponds programme, end up targeting men by default owing to the perception that women are not involved in the agricultural sector in Bangladesh. Indeed, even among the 30 per cent of individual fishpond farmers who were mandated by donors to be women, it was not rare for husbands to do most of the fish cultivation and sale, with the wife only feeding the fish.

19. Researchers are only beginning to quantify the long-term economic impacts of these investments in early childhood health and nutrition (Hoddinott et al. 2008).