The National Solidarity Programme: Assessing the Effects of Community-Driven Development in Afghanistan

Abstract

Over the past two decades, community-based approaches to project delivery have become a popular means for governments and development agencies to improve the alignment of projects with the needs of rural communities and to increase the participation of villagers in project design and implementation. This article briefly summarizes the results of an impact evaluation of the National Solidarity Programme (NSP), a community-driven development programme in Afghanistan that created democratically-elected community development councils and funded small-scale development projects. Using a randomized controlled trial across 500 villages, the evaluation finds that NSP had a positive effect on access to drinking water and electricity, acceptance of democratic processes, perceptions of economic well-being and attitudes towards women. Effects on perceptions of local and national government performance and material economic outcomes were, however, more limited or short-lived.

ACKNOWLEDGEMENTS

This article directly draws from previous reports and papers.³¹ The analysis and interpretations here summarize particularly those presented in A. Beath, F. Christia and R. Enikolopov, ‘Randomized Impact Evaluation of Afghanistan's National Solidarity Program-Final Report, World Bank Report No. 81107. Washington, DC: World Bank’, 2013 (at: http://documents.worldbank.org/curated/en/2013/07/18273450/randomized-impact-evaluation-afghanistans-national-solidarity-programme). Financial and logistical support for the Randomized Impact Evaluation of the National Solidarity Programme, whose results are summarized in this article, was provided by the World Bank's Trust Fund for Environmental and Socially Sustainable Development (TFESSD), the World Bank's AusAid-SAR Afghanistan Strengthening Community-Level Service Delivery Trust Fund, the World Bank's Development Impact Evaluation (DIME) initiative and the World Bank's Afghanistan Country Management Unit; and by the Food and Agriculture Organization (FAO) of the United Nations. The Ministry of Rural Rehabilitation and Development of the Government of the Islamic Republic of Afghanistan financially supported data collection for the baseline, midline and endline surveys through the monitoring and evaluation budget of Phase-II of the National Solidarity Programme. The Village Benefit Distribution Analysis was supported by the World Food Programme, the US Agency for International Development; the International Growth Centre and the Canadian International Development Agency. The findings, interpretations and conclusions expressed in this summary are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent; the National Solidarity Programme, Ministry of Rural Rehabilitation and Development of the Government of the Islamic Republic of Afghanistan or the Government of the Islamic Republic of Afghanistan; Australian Aid; the United States Agency for International Development, the Canadian International Development Agency; the Food and Agriculture Organization of the United Nations; the World Food Programme; and/or the International Growth Center.

ABOUT THE AUTHORS

Andrew Beath works for the World Bank’s Office of the Chief Economist for East Asia and the Pacific. Since joining the World Bank in 2010, Andrew has led the design and implementation of experimental and quasi-experimental impact evaluations of community-driven development programmes and livelihoods interventions across the Afghanistan, the Philippines, Solomon Islands, Timor-Leste, and Vietnam. Andrew holds a Ph.D. from the Department of Government at Harvard University and a Master of Public Administration in International Development from the Harvard Kennedy School.

Fotini Christia is an associate professor of political science at MIT. She has carried out extensive ethnographic, survey, and experimental research in Afghanistan and Bosnia and is currently working on projects in Yemen and Iraq. Her articles have been published in Science, in the American Political Science Review, and in Comparative Politics among other journals. Her book, Alliance Formation in Civil Wars, published by Cambridge University Press in 2012, was awarded the Luebbert award for best book in comparative politics, the Lepgold prize for best book ininternational relations and the distinguished book award of the ethnicity, nationalism, and migration section of the International Studies Association.

Ruben Enikolopov is an ICREA Research Professor at Barcelona Institute of Political Economy and Governance, UPF, and Associate Professor of economics at the New Economic School, Moscow. His research interests include political economy, development economics, and economics of mass media. Ruben has published his research in leading academic journals such as American Economic Review, Quarterly Journal of Economics, Proceedings of National Academy of Science, American Political Science Review, American Economic Journal: Applied Economics, Journal of Public Economics.

Notes

1. P. Dongier et al., ‘Community-Driven Development', in J. Klugman (ed.), A Sourcebook for Poverty Reduction Strategies, Vol. 1, Washington, DC: The World Bank, 2002, pp.303–31; G. Mansuri and V. Rao, ‘Localizing Development: Does Participation Work?’, World Bank Policy Research Report, Washington, DC: The World Bank, 2012; S. Wong, What Have Been the Impacts of World Bank Community-Driven Development Programmes? CDD Impact Evaluation Review and Operational and Research Implications, Washington, DC: World Bank, 2012. As of 2012, the World Bank supported approximately 400 community-driven development projects in 94 countries (Wong, What Have Been).

2. A. Escobar, Encountering Development: The Making and Unmaking of the Third World, Princeton, NJ: Princeton University Press, 1995; J. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, New Haven, CT: Yale University Press, 1998.

3. M. Cernea (ed.), Putting People First: Sociological Variables in Rural Development, New York: Oxford University Press and World Bank, 1985; A.O. Hirschman, Exit, Voice, and Loyalty: Responses to Decline in Firms, Organizations, and States, Cambridge, MA: Harvard University Press, 1970; A.O. Hirschman, Getting Ahead Collectively: Grassroots Experiences in Latin America, New York: Pergamon Press, 1984; E. Ostrom, Governing the Commons: The Evolution of Institutions for Collective Action, New York: Cambridge University Press, 1990; A.K. Sen, Commodities and Capabilities, Amsterdam: Elsevier, 1985; A.K. Sen, Development as Freedom, New York: Knopf, 1999.

4. Dongier et al. (see n.1 above); Wong (see n.1 above); R. Chambers, Rural Development: Putting the First Last, London: Longman, 1983.

5. D. Narayan (ed.), Empowerment and Poverty Reduction: A Sourcebook, Washington, DC: World Bank, 2002.

6. Macartan Humphreys, Raul Sanchez de la Sierra and Peter van der Windt, ‘Social and Economic Impacts of Tuungane: Final Report on the Effects of a Community Driven Reconstruction Program in Eastern Democratic Republic of Congo’, Mimeo, Columbia University, Jun. 2012; J. Fearon, M. Humphreys and J. Weinstein, ‘Democratic Institutions and Collective Action Capacity’, (forthcoming in American Political Science Review); K. Casey, R. Glennerster and E. Miguel, ‘Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan*’, The Quarterly Journal of Economics, Vol.127, No.4, 2012, pp.1755–812; A. Beath, F. Christia and R. Enikolopov, ‘Randomized Impact Evaluation of Afghanistan's National Solidarity Program-Final Report, World Bank Report No. 81107. Washington, DC: World Bank', 2013 (at: http://documents.worldbank.org/curated/en/2013/07/18273450/randomized-impact-evaluation-afghanistans-national-solidarity-programme).

7. Elisabeth King and Cyrus Samii, ‘Fast Track Institution Building in Conflict Affected Countries? Insights from Recent Field Experiments’, World Development, Vol.64, Dec. 2014, pp.740–54.

8. Bennett Sheree and Alyoscia D’Onofrio, ‘Community-Driven? Concepts, Clarity and Choices for CDD in Conflict-Affected Contexts', [International Rescue Committee Report] New York: International Rescue Committee, 23 Feb. 2015.

9. Villages must have more than 25 households to form a unitary CDC, although smaller villages may form joint CDCs with larger villages.

10. Villages are divided into ‘clusters’ of between 5 and 20 families, with each cluster electing a male and female representative to the CDC. The CDC is headed by an executive council composed of a president, deputy president, secretary and treasurer.

11. NSP features a ‘negative list’, which bans certain types of projects from receiving funding (including mosque construction, land purchases, payment of salaries to CDC members, purchase of weapons and cultivation of illegal crops). Eligible projects are generally approved by NSP provided they are endorsed through a village-wide consultation process; provide for equitable access; are technically and financially sound; include an operation and maintenance plan; and are funded by the community (including labour and material contributions) up to a level exceeding 10 per cent of the total cost.

12. A. Beath, F. Christia and R. Enikolopov, 2013. Randomized Impact Evaluation of Afghanistan's National Solidarity Programme-Final Report, World Bank Report No. 81107, Washington, DC: World Bank, 2013 (at: http://documents.worldbank.org/curated/en/2013/07/18273450/randomized-impact-evaluation-afghanistans-national-solidarity-programme). Seventy-three per cent of NSP funding is allocated to block grants, 18 per cent to facilitation costs and 9 per cent to administration.

13. NSP Phase-III also intends to mobilize the remaining 16,000 villages that have yet to receive the programme.

14. Icon-Institute, ‘National Risk and Vulnerability Assessment 2007/08: A Profile of Afghanistan’, [Main Report] Cologne: Icon-Institute, 2009.

15. T. Barfield, Weak Links on a Rusty Chain: Structural Weaknesses in Afghanistan's Provincial Government Administration, Berkeley, CA: Institute of International Studies, University of California, 1984; N. Nojumi, D. Mazurana and E. Stites, ‘Afghanistan's Systems of Justice: Formal, Traditional, and Customary', Working Paper, Medford: Feinstein International Famine Center, Youth and Community Programme, Tufts University, 2004.

16. A. Rahmani, The Role of Religious Institutions in Community Governance Affairs: How Are Communities Governed beyond the District Level?, Budapest, Hungary: Open Society Institute, Central European University Center for Policy Studies, 2006.

17. P. Kakar, Fine-Tuning the NSP: Discussions of Problems and Solutions with Facilitating Partners, Kabul: Afghanistan Research and Evaluation Unit, 2005.

18. Rahmani (see n.16 above).

19. I. Boesen, From Subjects to Citizens: Local Participation in the National Solidarity Programme, Kabul: Afghanistan Research and Evaluation Unit, 2004.

20. The ten districts were purposively sampled to meet the following three criteria: they had not been mobilized by NSP before; the security situation was conducive to survey implementation; and they had at least 65 villages. For more see A. Beath, F. Christia and R. Enikolopov, Randomized Impact Evaluation of Afghanistan's National Solidarity Programme: Hypotheses and Methodology, Kabul: World Bank, 13 April 2008.

21. Villages assigned to the control group received NSP in 2012.

22. The matched-pair cluster randomization procedure used background village characteristics including village size (based on data collected a few years earlier by Afghanistan's Central Statistics Organization) and a set of geographic variables (distance to river, distance to major road, altitude and average slope). The procedure was successful in statistically balancing treatment and control groups across 19 key variables for which data were collected in the baseline survey. The difference between the means of the two groups is always smaller than 6 per cent of the standard deviation. For more see A. Beath, F. Christia, R. Enikolopov and S. Kabuli, Randomized Impact Evaluation of Phase-II of Afghanistan's National Solidarity Programme (NSP): Estimates of Interim Program Impact from First Follow-Up Survey, Washington, DC: World Bank, 8 July 2010.

23. See here http://www.nsp-ie.org/followups.html for the survey instruments in English, and in the relevant local languages, Dari and Pashto.

24. For the midline male household questionnaire, enumerators sought participation of baseline male household respondents or, in their absence, a relative or cohabitant of the respondent.

25. See here for the full pre-analysis plan that links specific survey questions to indicators and offers detailed specifications of the analysis that were run (at: http://www.nsp-ie.org/toolsanddata/paps/NSP_IE_2FU_PAP_2012_02_07.pdf). This paper present the results of specification from the pre-analysis plan. Appendices IV and V in Beath et al. (n.6 above) present results for an alternative specification with improved statistical power.

26. See Part II, Section VII of Beath et al. (n.6 above) for further discussion of the hypotheses of the study.

27. For the second phase of NSP (2006–10), the PDO was to “lay the foundations for a strengthening of community level governance, and to support community-managed subprojects comprising reconstruction and development that improve access of rural communities to social and productive infrastructure and services”. The key outcome indicators were: (i) to enable “[a]round 21,600 … CDCs across the country [to] avail of basic social and productive infrastructure and other services”; (ii) to achieve “ERRs for community projects [in excess of] 15%”; to ensure that “O&M is in place for the completed projects and that the infrastructure services are use [sic] appropriately by the targeted communities for the purposes intended”; to ensure that “[a]t least 60% of CDCs [are] functioning to address critical development needs as identified by villages”; and (iv) to provide for “an increased level of participation of women in the community decision making [sic]” (World Bank [2006], p. 33). Note that the key outcome indicators identified by the program consist mainly of outputs specific to treatment areas and are thus inappropriate for this type of study, which includes control and treatment villages and seeks to explore impacts on general outcomes.” World Bank (2006). Technical Annex for a Proposed Grant of SDR 81.2 Million (US$120 Million Equivalent) to the Islamic Republic of Afghanistan for a Second Emergency National Solidarity Project (NSPII). Washington, DC: World Bank (November 10).

28. This section summarizes Part III of Beath et al. (see n.6 above).

29. Given the uncertainty over the future schedule of NSP block grant disbursement, villagers are unlikely to expect the implementation of further NSP-funded projects once the village's block grant allotment is completed.

30. A. Beath, F. Christia, G. Egorov, & R. Enikolopov, Electoral Rules and the Quality of Politicians: Theory and Evidence from a Field Experiment, Working Paper: Massachusetts Institute of Technology, 2013; A. Beath, F. Christia and R. Enikolopov, ‘Direct Democracy and Resource Allocation: Experimental Evidence from Afghanistan’ [MIT Political Science Department Research Paper No. 2011-6], SSRN, 26 Apr. 2015; A. Beath, F. Christia and R. Enikolopov, ‘Do Elected Councils Improve Governance?: Experimental Evidence on Local Institutions in Afghanistan’ [MIT Political Science Department Research Paper No. 2013-24], SSRN, 15 Sept. 2013.

31. Beath et al. Electoral Rules and the Quality of Politicians (see n.29 above); A. Beath, F. Christia, & R. Enikolopov, Randomized Impact Evaluation of Afghanistan's National Solidarity Programme – Hypotheses and Methodology, Kabul: World Bank, 13 Apr. 2008a; A. Beath, F. Christia, & R. Enikolopov, Randomized Impact Evaluation of Afghanistan's National Solidarity Programme – Report on Election Monitoring, Kabul: World Bank, 23 July 2008b; A. Beath, F. Christia, & R. Enikolopov, Randomized Impact Evaluation of the National Solidarity Programme: Baseline Survey Report, Kabul: World Bank, 11 Dec. 2008c; A. Beath, F. Christia, & R. Enikolopov, Randomized Impact Evaluation of Afghanistan's National Solidarity Programme – Report on Monitoring of Sub-Project Selection, Kabul: World Bank, 17 Jan. 2009; A. Beath, F. Christia, & R. Enikolopov, ‘Randomized Impact Evaluation of the National Solidarity Programme – Village Benefit Distribution Analysis. Pre-Analysis Plan: Hypotheses, Methodology and Specifications’. EGAP Registered Design, 17 Jan. 2012a; A. Beath, F. Christia, & R. Enikolopov, ‘Randomized Impact Evaluation of the National Solidarity Programme – Final Report. Pre-Analysis Plan: Hypotheses, Methodology and Specifications’. EGAP Registered Design, 14 Feb. 2012b; A. Beath, F. Christia and R. Enikolopov, ‘Empowering Women through Development Aid: Evidence from a Field Experiment in Afghanistan’, American Political Science Review, Vol.107, No.3, 2013, pp.540–57; A. Beath, F. Christia and R. Enikolopov, ‘Winning Hearts and Minds through Development: Experimental Evidence from Afghanistan’ [MIT Political Science Department Research Paper No. 2011-14], SSRN, 13 Apr. 2012; Beath et al. Direct Democracy and Resource Allocation (see n.29 above); Beath et al. Do Elected Councils Improve Governance? (see n.29 above); A. Beath, F. Christia, R. Enikolopov, & S. Kabuli, ‘Randomized Impact Evaluation of the National Solidarity Programme: Sub-Treatment Interventions – Analysis of the Impact of Election Type on Electoral Outcomes’, Kabul: World Bank, 2 July 2009a; A. Beath, F. Christia, R. Enikolopov, & S. Kabuli, ‘Randomized Impact Evaluation of the National Solidarity Programme: Sub-Treatment Interventions – Analysis of the Impact on Project Selection Outcomes of Variation in Selection Procedure and Election Types’, Kabul: World Bank, 2 July 2009b; A. Beath, F. Christia, R. Enikolopov, & S. Kabuli, (2010). ‘Randomized Impact Evaluation of Phase-II of Afghanistan's National Solidarity Programme (NSP) - Estimates of Interim Program Impact from First Follow-Up Survey’, Washington, DC: World Bank, 8 July 2010.