Does the Model Matter? Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments

Figures & data

Table 1. Descriptive statistics for restricted and unrestricted samples

	Unrestricted	Restricted
	sample means	sample means
Female	0.93  (0.26)	0.93  (0.26)
White	0.84  (0.36)	0.85  (0.36)
Black	0.14  (0.35)	0.14  (0.35)
Hispanic	0.00  (0.06)	0.00  (0.05)
Other non-white	0.01  (0.11)	0.01  (0.10)
Master's or higher	0.28  (0.45)	0.29  (0.45)
Experience	12.46  (9.78)	14.0  (9.6)
Self-contained classroom	0.90  (0.30)	1.00  (0.00)
N (teacher-years)	169,041	76,518

NOTE: standard deviations in parentheses.

Table 2. Descriptions of teacher effectiveness measures

Model name	Control variables	Teacher effectiveness modeling comparisons
MGP	Prior year same-subject score	Teacher-level median of the student-level SGPs
Student Background VAM	Prior year math and reading score, FRL status, disability status, parental education level, ELL status, race/ethnicity, gender	Variation in student achievement across all teachers
Classroom Characteristics VAM	Same as Student Background VAM, but also includes classroom-level variables: class size, average prior year math and reading scores, %FRL, %parents with bachelors or higher, %disability, %minority	Variation in student achievement across all teachers
School Fixed Effects (FE) VAM	Same as Student Background VAM, but also includes school fixed effects	Variation in student achievement across teachers in each school

Table 3. Estimated effect sizes for VAMs

	Unadjusted effect	Unadjusted effect
	size in math/	size in reading/
	adjusted effect	adjusted effect
	size in math	size in reading
Student Background VAM	0.22/.20	0.15/0.13
Classroom Characteristics VAM	0.22/.20	0.16/0.13
School FE VAM	0.25/0.17	0.20/0.03

NOTES: Effect sizes represent the impact, in terms of student achievement, of a one standard deviation change in teacher quality. Adjusted effect sizes are calculated by subtracting out the estimated error variance (weighted average of the standard errors of teacher effects) from the total variance and taking the square root of the adjusted variance.

Table 4. Pairwise correlation matrix of teacher effectiveness estimates

		Student	Classroom	School
	MGP	Background VAM	Characteristics VAM	FE VAM
Panel A. Math
MGP	1.00
Student Background VAM	0.93	1.00
Classroom Characteristics VAM	0.92	0.99	1.00
School FE VAM	0.61	0.65	0.64	1.00
Panel B. reading
MGP	1.00
Student Background VAM	0.83	1.00
Classroom Characteristics VAM	0.84	0.99	1.00
School FE VAM	0.48	0.58	0.58	1.00

Figure 1. Predicted percentile ranks of teacher effectiveness vs. classroom characteristics. (A) Teacher effectiveness (math) vs. average prior achievement. (B) Teacher effectiveness (reading) vs. average prior achievement. (C) Teacher effectiveness (math) vs. percent FRL. (D) Teacher effectiveness (reading) vs. percent FRL. (E) Teacher effectiveness (math) vs. percent nonwhite. (F) Teacher effectiveness (reading) vs. percent nonwhite.

Table 5. Average percentile ranks for stylized classrooms

	Advantaged	Average	Disadvantaged
Panel A. Math
MGP	61.9	49.1	41.4
Student Background VAM	57.9	48.7	48.5
Classroom Characteristics VAM	60.5	48.4	47.0
School FE VAM	52.9	50.3	49.6
Panel B. Reading
MGP	68.4	48.8	33.0
Student Background VAM	57.5	50.1	43.8
Classroom Characteristics VAM	59.7	49.8	42.6
School FE VAM	51.1	51.0	50.3

NOTE: Advantaged is defined as Q5 for average prior achievement and Q1 for % FRL (7,672 classrooms); average is defined as Q3 for average prior achievement and Q3 for % FRL (3,820 classrooms); disadvantaged is defined as Q1 for average prior achievement and Q5 for % FRL (8,002 classrooms).

Table 6. Intertemporal stability: correlations of teacher effectiveness estimates over time

	Adjacent time periods	1-Year gap	2-Year gap	3-Year gap	4-Year gap	5-Year gap
Panel A. Math
MGP	0.59	0.53	0.49	0.46	0.43	0.41
Student Background VAM	0.63	0.57	0.53	0.50	0.47	0.43
Classroom Characteristics VAM	0.64	0.58	0.54	0.51	0.48	0.44
School FE VAM	0.32	0.28	0.26	0.25	0.24	0.26
Panel B. Reading
MGP	0.43	0.39	0.36	0.33	0.30	0.28
Student Background VAM	0.47	0.42	0.38	0.35	0.32	0.29
Classroom Characteristics VAM	0.48	0.43	0.40	0.36	0.33	0.30
School FE VAM	0.18	0.16	0.13	0.14	0.13	0.13