Median Growth Percentiles (MGPs): Assessment of Intertemporal Stability and Correlations with Observational Scores

ABSTRACT

While states are no longer required to set up teacher evaluation systems based in significant part on student test scores, quite a few continue to use value-added (VAMs) or student growth percentile (SGP) models for that purpose. In this study, we analyzed three years of teacher data to illustrate the performance of teachers’ median growth percentiles (MGPs)). We found MGP’s consistency over time to be comparable with the existing estimates from the value-added models (VAMs). Additionally, we found that MGPs do not substantively agree with another measure of teacher quality – teachers’ observational scores. These findings suggest that caution should be exercised when teacher’s MGPs, as well as VAMs, are used in teacher evaluation system to make high-stakes decisions such as merit pay, tenure, or teacher contract termination. Our findings about the correlation of MGPs with observational scores support the idea of the multidimensional nature of teacher effectiveness construct.

Notes

1. Thus, thirty states require measures of student academic growth to be at least a significant factor within teacher evaluations; another 10 states require some student growth, and 11 states do not require any objective measures of student growth (Walsh, Joseph, Lakis, & Lubell, 2017).

2. Lockwood and Castellano (2015) and Monroe and Cai (2015) propose to use multidimensional item response theory (MIRT) to estimate SGPs and demonstrate the benefits of using the MIRT framework for estimation and inference in the SGPs context.

3. There is only one master teacher in the final sample. For that reason, we abstain from making comparison of the performance between master, mentor, and career teachers and focus only on comparisons between mentor and career teachers.

4. As per Merrigan and Huston (2004), a representative interpretive framework to help interpret simple correlation coefficients in the social sciences (see also Bloom, Hill, Black, & Lipsey, 2008; Cohen, 1988) is as follows: = a very strong correlation; = a strong correlation; a moderate correlation; = a weak correlation; = a very weak correlation, if no at all.

5. This matches what McCaffrey et al. (2009) estimated, with the share of teachers staying in the top quartile to be 20% on average, with estimates varying over years from 22% to 47%; what Aaronson, Barrow, and Sander (2007) found, that 57% of highly ranked teachers remained in the same top quartile in Chicago public high schools, and 36% of the lowest ranked teachers stayed in the bottom quartile from one year to another; and what Koedel and Betts (2007) found when they examined the rank-persistence in teacher value-added estimates in San Diego, estimating that 35% of teachers in the top quintile were ranked in the same quintile the following year, and for teachers in the lowest quintile, this share was 30%.