Randomized impact evaluation of education interventions: experiences and lessons from a reading to learn intervention in East Africa

Abstract

This paper presents the experiences and lessons learnt during the design and implementation of the randomized impact evaluation (IE) of a reading to learn (RtL) intervention in early primary grades. The study was to assess the impact of RtL on literacy and numeracy among pupils in low-performing districts in East Africa. The intervention was designed in a way that in each country one district implemented an intervention package that included teacher training, and teaching and learning materials; while the other district implemented this same package with an additional parental involvement component. Baseline data were collected in mid-2009 in Grades 1 and 2, and in 2010 for the incoming Grade 1. The endline data were collected in mid-2011. A total of 119 treatments and 110 control schools participated in the study. The randomized design provided an opportunity to attribute causality and also qualitatively establish pathways through which the intervention impacted on learning. The paper provides lessons learnt and demonstrates how the challenges faced during the evaluation were addressed. Additionally, the paper shows how IE was utilized to provide robust evidence of what works thus enabling policy-makers to make decisions on sustainability and scale-up of education interventions.

Keywords:

• impact evaluation
• early grades
• reading
• learning
• literacy
• numeracy
• Kenya
• Uganda

Acknowledgement

We acknowledge the important contribution of APHRC staff who participated at various stages of the development of this paper, including data collection and processing, as well as giving valuable comments during the internal review process. We are also grateful to our partners, the Aga Khan Foundation and the Ministry of Education for providing us with introductory letters to the District Education Officers and school head teachers. Last but not least, we are very grateful to the school principals, teachers and learners who participated in this study. However, the views presented in this paper are only of the authors and not necessarily shared by those mentioned.

Funding

This work was supported by the William and Flora Hewlett Foundation through the Education Research Program at APHRC [grant number 2009–3230].