A Comparison of Student Growth Percentile and Value-Added Models of Teacher Performance

Figures & data

Table 1 Rank correlations and misclassification measures across estimators with simulated data generated with Normal(0,1) errors. Results from 100 replications. Row 1: average rank correlation. Row 2: percentage of teachers above bottom 25% in true effect misclassified in bottom 25%. Row 3: percentage of teachers in bottom 25% in true effect misclassified in top 75%

					Corr DOLS/
Estimator	DOLS	AR	SGP-median	SGP-mean	SGP-median
Assign mech
	λ = 1
RG-RA	0.88	0.88	0.82	0.87	0.93
	8%	8%	10%	8%
	24%	24%	29%	25%
DG-RA	0.87	0.87	0.83	0.87	0.93
	7%	8%	9%	8%
	22%	23%	28%	23%
DG-PA	0.88	0.78	0.71	0.76	0.87
	7%	11%	12%	11%
	23%	33%	37%	33%
DG-NA	0.87	0.77	0.71	0.76	0.87
	8%	10%	12%	10%
	22%	31%	37%	31%
	λ = 0.5
RG-RA	0.88	0.88	0.82	0.87	0.93
	8%	8%	10%	8%
	24%	24%	29%	25%
DG-RA	0.87	0.87	0.83	0.87	0.93
	7%	8%	9%	8%
	22%	23%	28%	23%
DG-PA	0.88	0.78	0.71	0.76	0.87
	8%	11%	12%	11%
	23%	33%	37%	33%
DG-NA	0.87	0.77	0.71	0.76	0.87
	7%	10%	12%	10%
	22%	30%	36%	30%

Figure 1 Plots of true teacher effect percentiles on DOLS, AR, SGP-Median, and SGP-Mean percentiles using simulated data—RG-RA scenario.

Figure 2 Plots of true teacher effect percentiles on DOLS, AR, SGP-Median, and SGP-Mean percentiles using simulated data—DG-RA scenario.

Figure 3 Plots of true teacher effect percentiles on DOLS, AR, SGP-Median, and SGP-Mean percentiles using simulated data—DG-PA scenario.

Figure 4 Plots of true teacher effect percentiles on DOLS, AR, SGP-Median, and SGP-Mean percentiles using simulated data—DG-NA scenario.

Table 2 Rank correlations and misclassification measures across estimators with simulated data generated with (3) errors. Results from 100 replications. Row 1: average rank correlation. Row 2: percentage of teachers above bottom 25% in true effect misclassified in bottom 25%. Row 3: percentage of teachers in bottom 25% in true effect misclassified in top 75%

					Corr DOLS/
Estimator	DOLS	AR	SGP-median	SGP-mean	SGP-median
Assign mech
	λ = 1
RG-RA	0.71	0.71	0.72	0.79	0.85
	13%	13%	13%	12%
	40%	40%	39%	35%
DG-RA	0.72	0.72	0.71	0.78	0.84
	12%	12%	13%	12%
	36%	36%	38%	36%
DG-PA	0.70	0.58	0.57	0.66	0.80
	13%	15%	16%	15%
	39%	46%	48%	44%
DG-NA	0.71	0.59	0.59	0.66	0.80
	13%	14%	15%	14%
	38%	47%	45%	43%
	λ = 0.5
RG-RA	0.71	0.71	0.71	0.79	0.85
	13%	13%	13%	12%
	40%	40%	39%	35%
DG-RA	0.72	0.72	0.71	0.78	0.84
	12%	12%	13%	12%
	36%	36%	38%	35%
DG-PA	0.70	0.58	0.57	0.66	0.80
	13%	15%	16%	15%
	39%	46%	48%	44%
DG-NA	0.71	0.59	0.59	0.66	0.80
	13%	16%	15%	14%
	38%	47%	45%	43%

Table 3 Spearman rank correlations across estimators for nonrandom grouping and random grouping schools in administrative dataset. Grades 5 and 6. Years 2002–2007

Variables	DOLS	EB-Lag	SGP-Median	SGP-Mean
Panel A: Random grouping schools
DOLS	1.000
EB-Lag	0.990	1.000
SGP-Median	0.912	0.901	1.000
SGP-Mean	0.929	0.914	0.981	1.000
Teacher/Year Obs	457	457	457	457
Panel B: Nonrandom grouping schools
DOLS	1.000
EB-Lag	0.985	1.000
SGP-Median	0.891	0.875	1.000
SGP-Mean	0.912	0.895	0.982	1.000
Teacher/Year Obs	1061	1061	1061	1061

Table 4 Fraction of teachers rated in bottom 25% in the initial estimator who are not rated in bottom 25% in another estimator for nonrandom grouping and random grouping schools

Variables	DOLS	EB-Lag	SGP-Median	SGP-Mean
Panel A: Random grouping schools
DOLS	0	0.043	0.174	0.165
EB-Lag	0.043	0	0.157	0.157
SGP-Median	0.174	0.157	0	0.087
SGP-Mean	0.165	0.157	0.087	0
Teacher/Year Obs	457	457	457	457
Panel B: Nonrandom grouping schools
DOLS	0	0.09	0.222	0.18
EB-Lag	0.09	0	0.218	0.203
SGP-Median	0.222	0.218	0	0.09
SGP-Mean	0.18	0.203	0.09	0
Teacher/Year Obs	1061	1061	1061	1061

Table 5 Fraction of teachers rated in top 25% in the initial estimator who are not rated in top 25% in another estimator for nonrandom grouping and random grouping schools

Variables	DOLS	EB-Lag	SGP-Median	SGP-Mean
Panel A: Random grouping schools
DOLS	0	0.079	0.202	0.175
EB-Lag	0.079	0	0.193	0.184
SGP-Median	0.202	0.193	0	0.096
SGP-Mean	0.175	0.184	0.096	0
Teacher/Year Obs	457	457	457	457
Panel B: Nonrandom grouping schools
DOLS	0	0.079	0.211	0.189
EB-Lag	0.079	0	0.249	0.211
SGP-Median	0.211	0.249	0	0.083
SGP-Mean	0.189	0.211	0.083	0
Teacher/Year Obs	1061	1061	1061	1061

Appendix A Summary statistics for administrative data

Grade 5
Student-level characteristics
Variable	Mean	Std. Dev.	Min.	Max.
Math scale score	1630.033	239.368	569	2456
Reading scale score	1552.276	321.701	474	2713
Black	0.281	0.45	0	1
Hispanic	0.597	0.491	0	1
Free and reduced price lunch	0.703	0.457	0	1
Limited English proficiency	0.507	0.5	0	1
N	110,970
Teacher-level characteristics
Avg. lag Math score	1456.094	152.644	806.769	1986.808
Prop. FRL	0.718	0.249	0	1
Prop. LEP	0.508	0.259	0	1
Prop. Hispanic	0.584	0.322	0	1
Prop. Black	0.3	0.341	0	1
Number of students per year	24.019	7.929	12	145
Teacher experience	9.374	10.101	0	47
N	4620
Grade 6
Student-level characteristics
Math scale score	1641.693	247.982	770	2492
Reading scale score	1618.179	311.402	539	2758
Math scale standardized score	0.14	0.971	3.707	3.354
Reading scale standardized score	0.192	0.969	4.049	3.526
Black	0.288	0.453	0	1
Hispanic	0.6	0.49	0	1
Free and reduced price lunch	0.705	0.456	0	1
Limited English proficiency	0.511	0.5	0	1
N	104,441
Teacher-level characteristics
Avg. lag Math score	1608.182	143.225	903.733	2053.576
Prop. FRL	0.727	0.218	0	1
Prop. LEP	0.515	0.238	0	1
Prop. Hispanic	0.589	0.31	0	1
Prop. Black	0.307	0.33	0	1
Number of students per year	65.113	42.807	12	216
Teacher experience	7.668	8.978	0	40
N	1604