Contracting out schools at scale: evidence from Pakistan

Figures & data

Figure 1. Treatment (Phase 1) and comparison (Phase 3) Schools.

Note: This figure demonstrates that there is no geographical clustering of schools by phase. We refer here to Phase 1 schools as ‘treatment’ and Phase 3 as ‘comparison’.

Table 1. PSSP selection criteria.

		Actual		Estimated
Criteria	Definition	Schools selected Phase 1–3	Eligible Schools	Phase 1	Phase 2	Phase 3	Phase 1–3
1. Multi grade & over crowded	1 teacher, needs more classrooms, & above 80 students	103	943	24	25	52	101
2. Under utilised	2 or more teachers & below 30 students	430	1,464	24	197	145	366
3. Low enrolment	Fewer than 21 students in Grade 1–5	1,504	5,320	70	486	424	980
4. Poor learning outcomes	0% PEC passing rate (Phase 1 handover) or Less than 25% passing rate (Phase 2 handover)	872	1,317		394	1,264	1,658
5. Non-functional & closed	Non-operational / Merged	520	1,620	10	8		18
Total		3,429	10,664	128	1,110	1,885	3,266

Note: Schools were eligible to be recommended for inclusion in the PSSP if they fulfilled at least one of the selection criteria. Aggregate numbers of schools selected according to each criteria are reported in columns 1 and 2. Our school-level data does not associate individual schools with specific criteria, so we estimate each school’s selection criteria based on their enrolment, teachers, and test scores in 2015–16. Our estimate performs well for criteria 1–3, but over-estimates schools selected on criteria 4 and underestimates schools selected on criteria 5. Criteria 4 (poor learning outcomes) was 0% for Phase 1 and then increased to 25% for Phase 2 and 3 due to low numbers of schools with a 0% pass rate.

Table 2. Data and programme timing.

Month	PSSP Programme	Enrolment Data			Learning Data
		EMIS	MEA	ASER	LND	QAT
2012
- Oct		2012–13
2013
- Oct		2013–14
2014
- Oct		2014–15
2015
- Oct		2015–16		Y
- Nov				Y
2016
- Apr	Phase 1
- Sep				Y
- Oct		2016–17		Y
- Nov				Y
2017
- Jan			Y		Y
- Feb			Y		Y
- Mar			Y		Y	Phase 1 Endline
- Apr	Phase 3		Y		Y	Phase 3 Baseline
- May			Y		Y
- Aug			Y		Y
- Sep			Y	Y	Y
- Oct		2017–18

Note: This table presents shows the timing of the PSSP programme and of the key data used.

Table 3. Baseline descriptive statistics (2015–16).

	Phase 1	Phase 3	Non-PSSP Primary Schools	Diff (Phase 1–3)	P-value
N (Schools)	835	1,634	34,163	.	.
Enrolment (Students)	78.9 (35.3)	71.6 (36.7)	115.0 (77.6)	7.3	0.000
Enrolment Pre-Trend (Change 2012–2016)	−3.76	−3.83	2.48	0.07	0.96
	(38.6)	(31.7)	(46.5)
Teachers	2.3 (0.9)	2.2 (0.8)	3.2 (1.8)	0.1	0.000
PEC Pass Rate	0.66 (0.33)	0.64 (0.36)	.	0.02	0.30
PEC Average Marks	54.1 (10.7)	53.4 (12.3)	.	0.7	0.18
PEC Candidates	5.9 (4.2)	5.4 (4.3)	.	0.5	0.008
Years in operation	37.4 (16.2)	35.6 (15.6)	38.4 (17.7)	1.8	0.009
Classrooms	2.8 (1.3)	2.5 (1.1)	3.2 (1.8)	0.3	0.000
Classes	5.9 (0.6)	5.8 (0.7)	5.8 (0.7)	0.1	0.002

Note: This table presents characteristics of PSSP schools prior to conversion. Standard deviations are shown in parentheses. We don’t have data for PEC exams for non PSSP schools. Although our identification does not rest on baseline balance in covariates between schools, we nonetheless present baseline descriptive statistics for the groups of schools. Differences between treatment and comparison schools are statistically significant but small for prior enrolment, years in operation, number of classrooms, and classes. Differences are not statistically significant for the number of books. PSSP schools are all primary schools. PSSP schools are smaller than average. Phase 1 schools had 79 students and Phase 3 schools 72 students, compared with 114 students for all other public primary schools.

Table 4. Effect of treatment on enrolment by grade.

	All	K	G1	G2	G3	G4	G5	G5 (Exam)
Treatment x Post	50.69*	38.19*	7.12*	4.00*	4.07*	1.85*	1.09*	1.42*
	(1.61)	(1.33)	(0.47)	(0.34)	(0.28)	(0.24)	(0.20)	(0.19)
Controls	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Year FE	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
School FE	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Baseline Comparison Group Mean	82.9	27.0	13.1	10.6	8.1	7.0	5.4	5.5
N	13,614	4,548	4,548	4,548	4,548	4,548	4,548	6,566
N (Schools)	2,274	2,274	2,274	2,274	2,274	2,274	2,274	2,262
N (Years)	6	2	2	2	2	2	2	3
R-squared	0.704	0.809	0.799	0.834	0.825	0.845	0.835	0.767

Note: The dependent variable in each regression is the number of students in that grade. ‘Katchi’ is equivalent to Kindergarten. Coefficients on the treatment and post dummies are omitted. Standard errors are clustered at the school level. Omitted control variables are pre-treatment school characteristics (enrolment, teachers, test scores, years of operation, and classes), interacted with the post-treatment dummy. *p < 0.01.

Figure 2. Enrolment trends in treatment & comparison schools. Panel A: PSSP Schools. Panel B: Neighbouring Schools.

Note: The left panel A presents trends in average student enrolment numbers for treatment (Phase 1) schools and comparison (Phase 3) schools. The right panel A presents estimated treatment effects by year (comparing Phase 1 and Phase 3 schools). Data for 2012–2016 is self-reported by head teachers in annual school census carried out in October. Data for 2017 is collected by independent monitors from the school register in August. Panel B repeats this analysis for the closest neighbouring public school of each PSSP school.

Table 5. Tests for spillovers.

	PSSP Schools		Closest non-PSSP Public School
	Enrolment	Teachers	Enrolment	Teachers
Treat X Post	52.617***	2.717***	3.285***	0.146***
	(4.274)	(0.230)	(0.945)	(0.020)
Treat X Post X Other Schools in UC	−1.855	0.365
	(4.612)	(0.249)
Treat X Post X Under 1 km			4.426**	0.032
			(2.241)	(0.046)
School FE	Yes	Yes	Yes	Yes
Year FE	Yes	Yes	Yes	Yes
Baseline Comparison Mean	83	2	214	7
Years	6	3	6	3
Schools	2,469	2,469	2,469	2,469
N	14,776	4,954	146,897	72,819
R-squared	0.703	0.793	0.979	0.992

Note: Column (1) shows heterogeneity in effect size by the number of schools with the local village (Union Council or UC). If enrolment growth came primarily from neighbouring schools, then we would expect to see larger growth for schools in villages with other schools. Column (2) shows the same for the number of teachers in the school. Columns (3) and (4) shows the effect of PSSP conversion on student enrolment and the number of teachers at the nearest non-PSSP public school, and heterogeneity in this effect by distance between the PSSP and non-PSSP school. Here we see small positive effects on students at teachers at neighbouring schools, with a bigger positive effect on neighbouring schools within 1 km. Coefficients on the treatment and post dummies are omitted. Standard errors are clustered at the school level. *p < 0.1, **p < 0.05, ***p < 0.01.

Table 6. Heterogeneity and mechanisms.

	(1)	(2)	(3)	(4)
	Student Enrolment	Student Enrolment	Number of Teachers	Pupil-Teacher Ratio
Treatment x Post	56.01***	51.89***	3.20***	−5.32***
	(2.325)	(2.48)	(0.12)	(1.65)
Treat x Post x Male Only		17.27***
		(5.18)
Treat x Post x Female Only		11.76**
		(5.90)
Controls	Yes	Yes	Yes	Yes
Year FE	Yes	Yes	Yes	Yes
School FE	Yes	Yes	Yes	Yes
Baseline Comparison Mean	58.2	58.2	1.8	30.2
Years	6	6	2	2
Schools	1,151	1,151	1,151	1,151
N	6,889	6,889	2,302	2,302
R-squared	0.721	0.722	0.802	0.780

Note: The dependent variable in columns 1–2 is the number of students enrolled. The dependent variable in column 3 is the number of teachers, and in column 4 is the pupil-teacher ratio. Controls include year and school fixed effects. Coefficients on the treatment and post dummies are omitted. Standard errors are clustered at the school level. *p < 0.1, **p < 0.05, ***p < 0.01.

Table 7. Treatment effects by operator.

	Enrolment	Teachers	Enrolment	Teachers
All NGOs	33.467***	2.815***
	(1.812)	(0.126)
Other PSSP	62.091***	3.174***
	(2.130)	(0.118)
CARE			36.622***	4.075***
			(3.563)	(0.212)
DIL			16.232***	1.208***
			(3.749)	(0.216)
ITA			41.643***	2.835***
			(4.819)	(0.285)
Muslim Hands			51.225***	1.371***
			(6.150)	(0.296)
NRSP			23.477***	1.149***
			(2.976)	(0.121)
TCF			41.321***	4.952***
			(3.897)	(0.234)
Other PSSP			62.091***	3.174***
			(2.131)	(0.118)
Year FE	Yes	Yes	Yes	Yes
School FE	Yes	Yes	Yes	Yes
Baseline Comparison Mean	82.9	2.2	82.9	2.2
N (Years)	6	3	6	3
N (Schools)	2,469	2,469	2,469	2,469
N	14,776	4,954	14,776	4,954
R-squared	0.706	0.794	0.707	0.822

Note: Coefficients on the treatment and post dummies are omitted. CARE is the ‘CARE Foundation’, DIL is ‘Developments in Literacy’, ITA is ‘Idara-e-Taleem-o-Aagahi’, NRSP is the ‘National Rural Support Programme’, TCF is ‘The Citizen’s Foundation’. Standard errors are clustered at the school level. *p < 0.1, **p < 0.05, ***p < 0.01.

Table 8. Effects on test scores.

	(1)	(2)	(3)
Treat X Post	−0.263***	−0.243***	−0.278***
	(0.013)	(0.016)	(0.018)
Grade 3		0.210***
		(0.006)
Treat X Post X Grade 3		−0.029*
		(0.015)
Math			−0.259***
			(0.008)
Urdu			−0.168***
			(0.008)
Treat X Post X Math			0.033
			(0.020)
Treat X Post X Urdu			0.003
			(0.019)
Lee Bounds – Lower	−0.607***
	(0.015)
- Upper	0.119***
	(0.014)
School FE	Yes	Yes	Yes
Month FE	Yes	Yes	Yes
MEA FE	Yes	Yes	Yes
N	298,419	298,419	298,419
N (Schools)	2,469	2,469	2,469
R-squared	0.085	0.094	0.091

Note: The dependent variable is the standardised test score at the individual item level. We include fixed effects for schools, months, and test administrator (MEA), and cluster standard errors at the school level. Coefficients on the treatment and post dummies are omitted. *p < 0.1, **p < 0.05, ***p < 0.01.

Figure 3. Trends in test scores.

Note: The left panel presents trends in average student test scores for PSSP and all schools. Phase 1 schools were converted in the prior year, and Phase 3 schools in April. The right panel presents estimated treatment effects by year, in this case treating the Phase 3 schools (newly converting) as the treatment group and Phase 1 schools (already converted) as the comparison group. Data is from the MEAs for grades 2 and 3.

Figure 4. Quality assurance tests (QAT) (2017).

Note: This figure presents the frequency of schools reporting each student pass rate on the QAT test. ‘Endline’ pass rates on the right refer to the end of the school year for schools in Phase 1, conducted in March 2017, and ‘Baseline’ pass rates on the left refer to the start of the school year for schools in Phase 3, conducted in April 2017.