An Assessment of the Impacts of Sri Lanka’s Programme for School Improvement and School Report Card Programme on Students’ Academic Progress

Abstract

This paper examines two education programmes in Sri Lanka: the Programme for School Improvement (PSI), which decentralises decision-making power, and the School Report Card Programme (SRCP), which was designed to provide parents and other community members with information on the characteristics and performance of their local schools. Using a difference in differences identification strategy, it finds the following results. First, the PSI programme significantly increased Math and English reading test scores among Grade 4 students, but not first language (Sinhalese or Tamil) test scores. However, PSI has had no effect on any test scores of Grade 8 students. In contrast, the SRCP had no significant impacts on any test scores in either grade, and further inquiries revealed that the SRCP was never really implemented. Second, the paper examined the impact of both programmes on teacher and school principal variables. Overall, few effects were found, and in some cases effects were found that one would associate with reduced school quality. On a more positive note, the PSI programme does appear to have led schools to form School Development Committees (SDCs), as the programme stipulates, to establish a list of school priorities and to implement projects funded through local fundraising.

Acknowledgements

We would like to thank two anonymous referees for very helpful comments on an earlier draft of this paper.

Notes

1. School Development Committees (SDC) are headed by the principal and contain members of the teaching staff and other stakeholders, such as parents. SDC are dominated by the principal, while School Management Teams (SMT) spread power more evenly between the principal, teachers, and the local community.

2. One referee suggested that further confirmation could be found by examining parents’ reports of their school participation. Yet the available data, which may be noisy, show no significant effects of the PSI on parents’ participation in school meetings or in meetings with teachers; these results are shown in Table A1 in the Online Appendix.

3. Sri Lanka’s school year runs from January to November.

4. The education zones chosen for implementing the PSI in 2007 were in different districts from the education zones chosen for PSI in 2006, so none of the control zones chosen in 2006 were selected for the PSI in 2007. Similarly, 16 additional zones implemented the PSI in 2008, and another eight did so in 2009, but none of these 24 zones are the control zones selected in 2006.

5. All Sri Lankan primary and secondary schools are divided into four types: 1AB, 1C, 2, and 3. Type 1AB schools teach grades 1–13 and offer all three curriculum streams (arts, commerce, and science). Type 1C schools teach grades 1–13 but offer only two streams (arts and commerce). Type 2 schools offer only grades 1–11, and small Type 3 schools offer only grades 1–5 or 1–8.

6. In addition to the school types described in Note 5, there are five types of school ‘levels’: very congenial, congenial, uncongenial, difficult, and very difficult. There are also three race categories: Sinhalese, Tamil, and Muslim.

7. All ‘national’ and ‘Divisional Secretariat Division’ (DSD) schools were excluded from the sample because the Ministry of Education decided to implement the PSI programme in all national and DSD schools, so there is no control group for those schools. National schools, which make up about 3 per cent of Sri Lanka’s schools, are operated directly by the national government, while other schools are operated by provincial governments. DSD schools are another 3 per cent of schools that have recently been designated by the central government for a major improvement in their physical facilities.

8. In Sri Lanka, about half of the schools are, in effect, combinations of primary and secondary schools, and so have grades 1–13. These schools have one principal, but are also divided into primary and secondary ‘sections’, each of which has a ‘section head’. Education zones also have in-service advisors (school inspectors), who visit schools to supervise, and provide support for, teachers.

9. About 3.9 per cent of students drop out between Grades 8 and 9 (3.0% of girls and 4.8% of boys). This low attrition is unlikely to have a large effect on the estimates presented below.

10. See Angrist and Pischke (2009) for a detailed discussion of difference in differences estimation.

11. ATT (at T = 1) ≡ E[Y₁ – Y₀| P_s = 1, T = 1] = E[α + βT + γP_s + δ_ATT + ε_1isT – (α + βT + γP_s + ε_0isT)| P_s = 1, T = 1] = E[α + β + γ + δ_ATT + ε_1isT - (α + β + γ + ε_0isT)| P_s = 1, T = 1] = δ_ATT + E[ε_1isT| P_s = 1, T = 1] - E[ε_0isT| P_s = 1, T = 1] = δ_ATT.

12. More sophisticated methods allow ATT and ATE to be functions of observed student and school variables, which can be denoted by X. While those methods allow for ATE ≠ ATT, they still require that ATT = ATT for two schools, or two students, that have the same values for the X variables.

13. More formally, the required assumption is that E[Y₁ – Y₀| P_s = 1, T = 1] = E[Y₁ – Y₀| P_s = 0, T = 1]. Using Equations (6) and (7) yields a useful assumption for ε_1isT: E[α + βT + γP_s + δ_ATT + ε_1isT – (α + βT + γP_s + ε_0isT)| P_s = 1, T = 1] = E[α + βT + γP_s + δ_ATT + ε_1isT - (α + βT + γP_s + ε_0isT)| P_s = 0, T = 1], which can be written as δ_ATT + E[ε_1isT| P_s = 1, T = 1] - E[ε_0isT| P_s = 1, T = 1] = δ_ATT + E[ε_1isT| P_s = 0, T = 1] - E[ε_0isT| P_s = 0, T = 1], so that 0 + 0 = E[ε_1isT| P_s = 0, T = 1] + 0, which implies that E[ε_1isT| P_s = 0, T = 1] = 0.

14. In many double difference estimates the same students are observed over time, which allows the regression equation to use the change in the students’ test scores over time as the dependent variables, which automatically differences out school fixed effects, and indeed student fixed effects. Yet our test score data from Sri Lanka are from different students in different years, although from the same schools, so we need to explicitly include school fixed effects in the regression equation (or, equivalently, express all variables as deviations from school means).

15. These results for the PSI are similar, though less precisely estimated for the Math score, when the household variables are dropped from the regression.

16. In an earlier version of this paper, before we were informed that the SRCP had not been implemented, we estimated separate impacts for the 50 schools that were to have PSI only, the 50 schools that were to have the SRCP only, and the 50 schools that were to have both programmes. For first language, we found significant negative impacts for the 50 SRCP-only schools and weakly significant positive impacts for the 50 schools with both programmes. However, these two effects in opposite directions were not jointly significant.

17. One possible reason for the differences across Grades 4 and 8 is that the samples differ: 39 schools have Grade 4 but not Grade 8. Yet this is not the case; when the sample is limited to schools with both Grades 4 and 8, the estimated impacts on Math and reading of Grade 4 students are 0.189 (10% significance) and 0.250 (1% significance), respectively. One could also argue that two years is too short a time period to find a significant impact. Yet several studies in the past 10–15 years have found statistically significant impacts of fairly similar programmes that had been operating for only one or two years. Examples include Jimenez and Sawada (1999), Galiani et al. (2008), and Banerjee et al. (2010).

18. School development societies are not the same as SDC, which were initiated by the PSI programme. School development societies are an older association that can receive local donations, but (unlike schools development committees) does not actively solicit donations from the local community.