Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels

Figures & data

Table 1. Teacher-level summary statistics.

Variables	Mean	Standard deviation	Minimum	Maximum
Elementary School
Math posttest score	0.02	0.57	−1.69	1.26
Math pretest score	0.02	0.55	−1.34	1.36
Reading pretest score	0.04	0.49	−1.21	1.25
Free lunch eligible	0.69	0.27	0.00	1.00
Reduced-price lunch eligible	0.06	0.06	0.00	0.23
Limited English proficiency	0.08	0.14	0.00	0.68
Special education	0.10	0.09	0.00	0.46
Grade 4	0.50	0.49	0.00	1.00
Grade 5	0.50	0.49	0.00	1.00
Student-equivalents	23.73	12.03	5.03	83.40
Preshrinkage value-added estimates	0.01	0.30	−0.90	0.96
Estimated variance of preshrinkage value-added estimates	0.01	0.01	0.00	0.07
Shrinkage value-added estimates	0.00	0.25	−0.77	0.71
Estimated variance of shrinkage value-added estimates	0.01	0.01	0.00	0.04
Number of teachers	297
Middle School
Math posttest score	−0.04	0.49	−1.27	1.17
Math pretest score	−0.05	0.50	−1.12	1.16
Reading pretest score	−0.04	0.44	−1.28	1.06
Free lunch eligible	0.72	0.21	0.00	0.98
Reduced-price lunch eligible	0.07	0.05	0.00	0.33
Limited English proficiency	0.08	0.13	0.00	0.58
Special education	0.13	0.10	0.00	0.92
Grade 6	0.39	0.45	0.00	1.00
Grade 7	0.30	0.38	0.00	1.00
Grade 8	0.31	0.40	0.00	1.00
Student-equivalents	54.07	25.54	7.99	138.75
Preshrinkage value-added estimates	−0.01	0.20	−0.58	0.63
Estimated variance of preshrinkage value-added estimates	0.01	0.01	0.00	0.04
Shrinkage value-added estimates	0.00	0.17	−0.40	0.58
Estimated variance of shrinkage value-added estimates	0.01	0.00	0.00	0.02
Number of teachers	182

Source: Administrative data from OSSE and DCPS.

NOTE: Rows with student characteristics are based on teacher-level averages. Value added is estimated at the teacher level.

DCPS = District of Columbia Public Schools; OSSE = Office of the State Superintendent of Education of the District of Columbia.

Table 2. Effects of student characteristics on teacher value-added estimates.

		Log variance of individual teacher value-added estimates			Absolute value of individual teacher value-added estimates
Characteristic	Squared	Preshrinkage	Postshrinkage	Difference	Preshrinkage	Postshrinkage	Difference
	residuals (1)	(2)	(3)	(4)	(5)	(6)	(7)
Elementary
Math pretest score	−0.08^*	−0.20^*	−0.17^*	0.03^*	−0.07^*	−0.05^*	0.02^*
	(0.01)	(0.07)	(0.05)	(0.01)	(0.02)	(0.02)	(0.01)
Reading pretest score	−0.08^*	−0.14^*	−0.12^*	0.02	−0.06^*	−0.05^*	0.01^*
	(0.01)	(0.07)	(0.06)	(0.01)	(0.02)	(0.02)	(0.01)
Free lunch eligible	0.05^*	0.34^*	0.29^*	−0.05^*	0.15^*	0.12^*	−0.03^*
	(0.01)	(0.12)	(0.10)	(0.02)	(0.03)	(0.03)	(0.01)
Reduced-price lunch eligible	0.05	−0.80	−0.70	0.11	−0.10	−0.05	0.05
	(0.04)	(0.63)	(0.52)	(0.12)	(0.19)	(0.15)	(0.05)
Limited English proficiency	0.05	−0.91^*	−0.79^*	0.12^*	−0.16^*	−0.11	0.05^*
	(0.03)	(0.21)	(0.18)	(0.02)	(0.07)	(0.06)	(0.01)
Special education	0.22^*	0.22	0.18	−0.05	0.03	0.01	−0.01
	(0.03)	(0.41)	(0.34)	(0.07)	(0.11)	(0.09)	(0.03)
Log student equivalents		−0.63^*	−0.54^*	0.09^*
		(0.07)	(0.06)	(0.01)
Student-equivalents /100					−0.09	−0.02	0.07^*
					(0.08)	(0.06)	(0.02)
Number of observations (students)	7546	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.
Number of Observations (teachers)	n.a.	297	297	297	297	297	297
Middle School
Math pretest score	−0.18^*	−0.64^*	−0.54^*	0.10^*	−0.01	0.01	0.02^*
	(0.01)	(0.08)	(0.07)	(0.01)	(0.02)	(0.01)	(0.00)
Reading pretest score	−0.15^*	−0.69^*	−0.58^*	0.11^*	−0.02	0.00	0.02^*
	(0.01)	(0.08)	(0.07)	(0.02)	(0.02)	(0.02)	(0.01)
Free lunch eligible	0.08^*	0.67^*	0.57^*	−0.10^*	0.05	0.03	−0.02^*
	(0.02)	(0.18)	(0.16)	(0.03)	(0.04)	(0.03)	(0.01)
Reduced-price lunch eligible	−0.02	−1.61	−1.38	0.23	−0.24	−0.15	0.09^*
	(0.03)	(0.83)	(0.71)	(0.13)	(0.15)	(0.13)	(0.03)
Limited English proficiency	0.08^*	−0.26	−0.19	0.07	−0.03	−0.01	0.02
	(0.03)	(0.30)	(0.25)	(0.05)	(0.08)	(0.07)	(0.01)
Special education	0.32^*	1.72^*	1.42^*	−0.30^*	−0.03	−0.08	−0.05
	(0.03)	(0.35)	(0.29)	(0.07)	(0.07)	(0.05)	(0.03)
Log student equivalents		−0.66^*	−0.56^*	0.10^*
		(0.07)	(0.06)	(0.01)
Student-equivalents/100					0.01	0.04	0.02^*
					(0.03)	(0.03)	(0.01)
Number of observations (students)	9930	n.a.	n.a.	n.a.	n.a.	n.a.	n.a.
Number of observations (teachers)	n.a.	182	182	182	182	182	182

Source: Administrative data from OSSE and DCPS.

NOTE: Each result is from a separate regression except for those for eligibility for free or reduced-price lunch, which were entered simultaneously into a regression. In column 1, the results in the first four rows are from a student-level regression of the squared residuals from the value-added model on the student characteristics. The remaining columns are teacher-level regressions. In columns 2 and 3, the outcome is the natural log of the value-added estimates' variance. Column 4 tests the statistical significance of the difference in the relationship between student characteristics and the natural log of the value-added estimates' variance. In columns 5 and 6, the outcome is the absolute value of the value-added estimates. Column 7 tests the statistical significance of the difference in the relationship between student characteristics and the absolute value of the value-added estimates. Robust standard errors are in parentheses.

^* Significantly different from zero at the 0.05 level, two-tailed test.

DCPS = District of Columbia Public Schools; OSSE = Office of the State Superintendent of Education of the District of Columbia.

n.a. = not applicable.

Table 3. Effects of student characteristics on distribution of teacher value-added estimates.

	Probability in either the bottom or top decile			Probability in either the bottom or top quintile
	Pre-shrinkage	Post-shrinkage	Difference	Pre-shrinkage	Post-shrinkage	Difference
Characteristic	(1)	(2)	(3)	(4)	(5)	(6)
Elementary School
Math pretest score	−0.14^*	−0.14^*	0.00	−0.16^*	−0.15^*	0.01
	(0.04)	(0.04)	(0.01)	(0.05)	(0.05)	(0.02)
Reading pretest score	−0.12^*	−0.13^*	−0.01	−0.15^*	−0.14^*	0.01
	(0.05)	(0.05)	(0.01)	(0.05)	(0.05)	(0.03)
Free lunch eligible	0.25^*	0.24^*	−0.01	0.42^*	0.36^*	−0.06
	(0.07)	(0.07)	(0.01)	(0.09)	(0.09)	(0.04)
Reduced-price lunch eligible	−0.35	−0.23	0.12	−0.08	0.13	0.21
	(0.39)	(0.40)	(0.14)	(0.46)	(0.47)	(0.21)
Limited English proficiency	−0.17	−0.15	0.02	−0.22	−0.22	0.00
	(0.13)	(0.13)	(0.03)	(0.18)	(0.18)	(0.04)
Special education	−0.23	−0.12	0.11	0.20	−0.04	−0.24
	(0.24)	(0.25)	(0.09)	(0.33)	(0.32)	(0.20)
Student-equivalents/100	−0.35^*	−0.32^*	0.04	−0.19	0.06	0.25
	(0.16)	(0.16)	(0.04)	(0.25)	(0.25)	(0.14)
Number of teachers	297	297	297	297	297	297
Middle School
Math pretest score	−0.01	0.02	0.03	0.04	0.10	0.05
	(0.06)	(0.06)	(0.05)	(0.07)	(0.07)	(0.03)
Reading pretest score	−0.05	−0.02	0.04	0.05	0.09	0.03
	(0.07)	(0.06)	(0.07)	(0.08)	(0.08)	(0.02)
Free lunch eligible	0.12	0.19	0.07	−0.05	−0.12	−0.07
	(0.14)	(0.13)	(0.12)	(0.18)	(0.18)	(0.06)
Reduced-price lunch eligible	−0.51	−0.27	0.24	−0.69	−0.44	0.25
	(0.56)	(0.55)	(0.32)	(0.66)	(0.67)	(0.22)
Limited English proficiency	−0.14	−0.14	0.00	−0.27	−0.25	0.02
	(0.19)	(0.19)	(0.07)	(0.28)	(0.28)	(0.06)
Special education	0.14	−0.13	−0.27	−0.28	−0.56	0.25
	(0.30)	(0.22)	(0.26)	(0.34)	(0.33)	(0.22)
Student-equivalents /100	−0.05	0.06	0.12	0.13	0.22	0.09
	(0.11)	(0.11)	(0.10)	(0.14)	(0.14)	(0.06)
Number of teachers	182	182	182	182	182	182

Source: Administrative data from OSSE and DCPS.

NOTE: Each result is from a separate teacher-level regression except for those for eligibility for free or reduced-price lunch, which were entered simultaneously into a regression. In columns 1 and 2, the outcome is the probability the teacher value-added estimate is below the 10th or above the 90th percentile of estimates. Column 3 tests the statistical significance of the difference in the relationship between student characteristics and the probability of being below the 10th or above the 90th percentile of estimates. In columns 4 and 5, the outcome is the probability the teacher value-added estimate is below the 20th or above the 80th percentile of estimates. Column 6 tests the statistical significance of the difference in the relationship between student characteristics and the probability the teacher value-added estimate is below the 20th or above the 80th percentile of estimates. Robust standard errors are in parentheses.

^* Significantly different from zero at the 0.05 level, two-tailed test.

DCPS = District of Columbia Public Schools; OSSE = Office of the State Superintendent of Education of the District of Columbia.