Does the Model Matter? Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments

REFERENCES

• Aaronson, D., Barrow, L., and Sander, W. (2007), “Teachers and Student Achievement in the Chicago Public High Schools,” Journal of Labor Economics, 25, 95–135. Available at http://www.journals. uchicago.edu/doi/abs/10.1086/508733.
   Web of Science ®Google Scholar
• Ashenfelter, O., and Krueger, A. (1994), “Estimates of the Economic Return to Schooling From a New Sample of Twins,” The American Economic Review, 84, 1157–1173.
   Web of Science ®Google Scholar
• Atteberry, A. (2011), Defining School Value-Added: Do Schools that Appear Strong on One Measure Appear Strong on Another? Evanston, IL. . Available at http://www.eric.ed.gov/ERICWebPortal/search/detailmini. jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED518222&ERIC ExtSearch_SearchType_0=no&accno=ED518222.
   Google Scholar
• Baker, B.D. (2012), “The Toxic Trifecta, Bad Measurement & Evolving Teacher Evaluation Policies.” . Available at http://nepc.colorado.edu/ blog/toxic-trifecta-bad-measurement-evolving-teacher-evaluation-policies.
   Google Scholar
• Ballou, D., Mokher, C.G., and Cavalluzzo, L. (2012), “Using Value Added Assessment for Personnel Decisions: How Omitted Variables and Model Specification Influence Teachers’ Outcomes,” . Available at http://www.aefpweb.org/sites/default/files/webform/AEFP-Using VAM for personnel decisions_02-29-12.pdf.
   Google Scholar
• Ballou, D., Sanders, W.L., and Wright, P.S. (2004), “Controlling for Student Background in Value-Added Assessment of Teachers,” Journal of Education and Behavioral Statistics, 29, 37–65. . Available at http://jeb.sagepub.com/cgi/content/abstract/29/1/37.
   Web of Science ®Google Scholar
• Barlevy, G., and Neal, D. (2012), “Pay for Percentile,” American Economic Review, 102, 1805–1831.
   Web of Science ®Google Scholar
• Betebenner, D. (2007), Estimation of Student Growth Percentiles for the Colorado Student Assessment Program, Dover, NH: National Center for the Improvement of Educational Assessment. . Available at http://www.cde.state. co.us/ dedocs/Research/PDF/technicalsgppaper_betebenner.pdf.
   Google Scholar
• ——— (2009), “Norm- and Criterion-Referenced Student Growth,” Educational Measurement: Issues and Practice, 28, 42–51. . Available at http://doi.wiley.com/10.1111/j.1745-3992.2009.00161.x.
   Google Scholar
• Boyd, D.J., Grossman, P., Lankford, H., Loeb, S., and Wyckoff, J. (2008), Measuring Effect Sizes: The Effect of Measurement Error. National Conference on Value-Added Modeling, Madison, WI: University of Wisconsin.
   Google Scholar
• Boyd, D.J., Lankford, H., Loeb, S., and Wyckoff, J.H. (2010). Teacher Layoffs: An Empirical Illustration of Seniority vs. Measures of Effectiveness, Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. . Available at http://www. caldercenter.org/UploadedPDF/1001421-teacher-layoffs.pdf.
   Google Scholar
• Briggs, D., and Domingue, B. (2011), Due Diligence and the Evaluation Of Teachers A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by The Los Angeles Times, Boulder, CO: National Education Policy Center. . Available at http://greatlakescenter.org/docs/Policy_Briefs/Briggs_LATimes.pdf.
   Google Scholar
• Buddin, R. (2010), How Effective are Los Angeles Elementary Teachers and Schools? . Available at http://www.latimes.com/media/acrobat/2010-08/55538493.pdf.
   Google Scholar
• Burke, M.A., and Sass, T.R. (2008), “Classroom Peer Effects and Student Achievement,” CALDER working paper #18. . Available at http://www.urban.org/UploadedPDF/1001190_peer_effects.pdf.
   Google Scholar
• Caldas, S.J., and Bankston, C. (1997), “Effect of School Population Socioeconomic Status on Individual Academic Achievement,” The Journal of Educational Research, 90, 269–277. . Available at http://apps.webofknowledge. com.offcampus.lib.washington.edu/full_record.do?product=WOS&search_ mode=GeneralSearch&qid=17&SID=2FC8FA7Hp8kFmOCeOOF&page= 1&doc=2.
   Web of Science ®Google Scholar
• Castellano, K. (2011). Unpacking Student Growth Percentiles: Statistical Properties of Regression-Based Approaches With Implications for Student and School Classifications, Doctoral Dissertation, University of Iowa. . Available at http:/ir.uiowa.edu/etd/931.
   Google Scholar
• Chetty, R., Friedman, J.N., and Rockoff, J.E. (2011), “The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood,” . NBER Working Paper no. #17699.
   Google Scholar
• Clotfelter, C., Ladd, H., and Vigdor, J. (2006), “Teacher–Student Matching and the Assessment of Teacher Effectiveness,” The Journal of Human Resources, 41, 778–820.
   Web of Science ®Google Scholar
• Cornelissen, T. (2008), “The Stata Command Felsdvreg to Fit a Linear Model With Two High-Dimensional Fixed Effects,” Stata Journal, 8, 170–189.
   Web of Science ®Google Scholar
• Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., and Rothstein, J. (2012), “Evaluating Teacher Evaluation: Popular Modes of Evaluating Teachers Are Fraught With Inaccuracies and Inconsistencies, But the Field has Identified Better Approaches,” Phi Delta Kappan, 93, 8–15. . Available at http://uwashington.worldcat.org.offcampus.lib.washington.edu/title/evalua ting-teacher-evaluation/oclc/779377670&referer=brief_results.
   Web of Science ®Google Scholar
• Ehlert, M., Koedel, C., Parsons, E., and Podgursky, M. (2012), “Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?” . Working Paper no. #1210, University of Missouri.
   Google Scholar
• Gabriel, T. (2010, September 2), “A Celebratory Road Trip for Education Secretary,” New York Times, p. A24. . Available at http://www.nytimes.com/2010/09/02/education/02duncan.html.
   Google Scholar
• Glazerman, S., Loeb, S., Goldhaber, D., Staiger, D., Raudenbush, S., and Whitehurst, G. (2010), Evaluating Teachers: The Important Role of Value-Added. . Available at http://www.brookings. edu/∼/media/Files/rc/reports/2010/1117_evaluating_teachers/1117_evalua ting_teachers.pdf.
   Google Scholar
• Goldhaber, D. (2007), “Everyone's Doing It, But What Does Teacher Testing Tell Us About Teacher Effectiveness?” Journal of Human Resources, 42, 765–794.
   Web of Science ®Google Scholar
• ——— (2010), When the Stakes Are High, Can We Rely on Value-Added? Exploring the Use of Value-Added Models to Inform Teacher Workforce Decisions, Washington, DC: Center for American Progress. . Available at http://www.americanprogress.org/issues/2010/12/pdf/vam.pdf.
   Google Scholar
• Goldhaber, D., and Anthony, E. (2007), “Can Teacher Quality be Effectively Assessed? National Board Certification as a Signal of Effective Teaching,” Review of Economics and Statistics, 89, 134–150.
   Web of Science ®Google Scholar
• Goldhaber, D., Brewer, D., and Anderson, D. (1999), “A Three-Way Error Components Analysis of Educational Productivity,” Education Economics, 7, 199–208.
   Google Scholar
• Goldhaber, D., and Chaplin, D. (2012), “Assessing the “Rothstein Falsification Test.” Does it Really Show Teacher Value-added Models are Biased?” . Mathematica Working Paper. Available at http://www. mathematica-mpr.com/publications/pdfs/education/rothstein_wp.pdf.
   Google Scholar
• Goldhaber, D., Cowan, J., and Walch, J. (2012), “Is a Good Elementary Teacher Always Good? Assessing Teacher Performance Estimates Across Contexts and Comparison Groups,” . CEDR Working Paper no. #2012-7.
   Google Scholar
• Goldhaber, D., Goldschmidt, P., Sylling, P., and Tseng, F. (2011), “Assessing Value-Added Model Estimates of Teacher Contributions to Student Learning at the High School Level,” . CEDR Working Paper no. #2011-4.
   Google Scholar
• Goldhaber, D., and Hansen, M. (2010), “Using Performance on the Job to Inform Teacher Tenure Decisions,” American Economic Review, 100, 250–255.
   Web of Science ®Google Scholar
• ——— (2013), “Is it Just a Bad Class? Assessing the Long-Term Stability of Estimated Teacher Performance,” Economica, 80, 589–612.
   Web of Science ®Google Scholar
• Goldhaber, D., and Theobald, R. (2011), “Managing the Teacher Workforce in Austere Times: The Implications of Teacher Layoffs,” . CEDR Working Paper no. #2011-1.2. Available at http://www.cedr.us/papers/working/CEDR WP 2011-1.2 Teacher Layoffs (6-15-2011).pdf.
   Google Scholar
• Goldhaber, D., and Walch, J. (2012), “Strategic Pay Reform: A Student Outcomes-Based Evaluation of Denver's ProComp Teacher Pay Initiative,” Economics of Education Review, 31, 1067–1083.
   Web of Science ®Google Scholar
• Goldhaber, D., Walch, J., and Gabele, B. (2012), “Does the Model Matter: Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments,” . CEDR Working Paper no. #2012-6. Available at http://www.cedr.us/papers/working/CEDR%20WP%202012-6_Does%20the%20Model%20Matter.pdf.
   Google Scholar
• Goldschmidt, P., Choi, K., and Beaudoin, J.P. (2012), Growth Model Comparrison Study: A Summary of Results, Washington, DC. . Available at http://www. ccsso.org/Documents/2012/Summary_of_Growth_Model_Comparison_ Study_2012.pdf.
   Google Scholar
• Gordon, R.J., Kane, T.J., and Staiger, D.O. (2006), Identifying Effective Teachers Using Performance on the Job. Hamilton Project White Paper, Washington, DC: Brookings Institution. . Available at http://www.brook.edu/views/papers/200604hamilton_1.htm.
   Google Scholar
• Hanushek, E. (1979), “Conceptual and Empirical Issues in the Estimation of Education Production Functions,” Journal of Human Resources, 14, 351–388.
   Web of Science ®Google Scholar
• ——— (1986), “The Economics of Schooling—Production and Efficiency in Public-Schools,” Journal of Economic Literature, 24, 1141–1177.
   Web of Science ®Google Scholar
• Hanushek, E.A., and Rivkin, S.G. (2010), “Generalizations About Using Value-added Measures of Teacher Quality,” American Economic Review, 100, 267–271.
   Web of Science ®Google Scholar
• Harris, D., and Sass, T.R. (2006), “Value-Added Models and the Measurement of Teacher Quality,” . unpublished manuscript.
   Google Scholar
• Harris, D., Sass, T., and Semykina, A. (2010), Value-Added Models and the Measurement of Teacher Productivity, Washington, DC, unpublished manuscript.
   Google Scholar
• Jackson, C., and Bruegmann, E. (2009), “Teaching Students and Teaching Each Other: The Importance of Peer Learning for Teachers,” American Economic Journal: Applied Economics, 1, 85–108.
   Web of Science ®Google Scholar
• Jacob, B., and Lefgren, L. (2008), “Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education,” Journal of Labor Economics, 26, 101–136.
   Web of Science ®Google Scholar
• Kane, T., and Staiger, D. (2008), “Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation,” . NBER Working Paper no. #14607. Available at http://www.nber.org/papers/w14607.pdf?new_window=1.
   Google Scholar
• Koedel, C., and Betts, J. (2007), Re-Examining the Role of Teacher Quality in the Educational Production Function, San Diego, CA: University of Missouri. . Available at http://economics.missouri. edu/working-papers/2007/WP0708_koedel.pdf.
   Google Scholar
• ——— (2011), “Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique,” Education Finance and Policy, 6, 18–42.
   Web of Science ®Google Scholar
• Koenker, R. (2005), Quantile Regression (Econometric Society Monographs) (p. 366), Cambridge: Cambridge University Press. . Available at http://www.amazon.com/Quantile-Regression-Econometric-Society-Monog raphs/dp/0521608279.
   Google Scholar
• LA Times (2010), “Study Critical of Teacher Layoffs (2010, December 24),” Los Angeles Times. . Available at http://www.latimes. com/news/local/teachers-investigation/la-me-teachers-study-critical-ap,0, 6451518.story.
   Google Scholar
• Lavy, V., Silva, O., and Weinhardt, F. (2012), “The Good, the Bad, and the Average: Evidence on Ability Peer Effects in Schools,” Journal of Labor Economics, 30, 367–414.
   Web of Science ®Google Scholar
• Lockwood, J.R., and McCaffrey, D.F. (2012), “Reducing Bias in Observational Analyses of Education Data by Accounting for Test Measurement Error,” . unpublished manuscript, RAND Corporation.
   Google Scholar
• Lockwood, J.R., McCaffrey, D.F., Hamilton, L., Stecher, B., Le, V.-N., and Martinez, J.F. (2007), “The Sensitivity of Value-Added Teacher Effect Estimates to Different Mathematics Achievement Measures,” Journal of Education Measurement, 44, 47–67.
   Web of Science ®Google Scholar
• McCaffrey, D., Koretz, D., Lockwood, J.R., Louis, T.A., and Hamilton, L.S. (2004), “Models for Value-Added Modeling of Teacher Effects,” Journal of Educational and Behavioral Statistics, 29, 67–101.
   PubMed Web of Science ®Google Scholar
• McCaffrey, D., Sass, T.R., Lockwood, J.R., and Mihaly, K. (2009), “The Intertemporal Variability of Teacher Effect Estimates,” Education Finance and Policy, 4, 572–606.
   Web of Science ®Google Scholar
• MET (2010), Learning About Teaching: Initial Findings from the Measures of Effective Teaching Project, Bill & Melinda Gates Foundation. . Available at http://www.metproject.org/downloads/Preliminary_Findings-Research_ Paper.pdf.
   Google Scholar
• Nichols, A. (2008), “FESE: Stata Module to Calculate Standard Errors for Fixed Effects,” Boston College Department of Economics. . Available at http://econpapers.repec.org/software/bocbocode/s456914.htm.
   Google Scholar
• Papay, J. (2010), “Different Tests, Different Answers: The Stability of Teacher Value-Added Estimates Across Outcome Measures,” American Educational Research Journal, 48, 163–193.
   Web of Science ®Google Scholar
• Podgursky, M., and Springer, M. (2007), “Teacher Performance Pay: A Review,” Journal of Policy Analysis and Management, 26, 909–949.
   Web of Science ®Google Scholar
• R Development Core Team (2008), R: A Language and Environment for Statistical Computing, Vienna, Austria: R Foundation for Statistical Computing. . Available at http://www.r-project.org.
   Google Scholar
• Racine, J. (2011), A Primer on Regression Splines. . Available at http://cran.r-project.org/web/packages/crs/vignettes/spline_primer.pdf.
   Google Scholar
• Rivkin, S., Hanushek, E., and Kain, J. (2005), “Teachers, Schools, and Academic Achievement,” Econometrica, 73, 417–458.
   Web of Science ®Google Scholar
• Rockoff, J., and Speroni, C. (2011), “Subjective and Objective Evaluations of Teacher Effectiveness: Evidence From New York City,” Labour Economics, 18, 687–696.
   Web of Science ®Google Scholar
• Rothstein, J. (2010), “Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement,” Quarterly Journal of Economics, 125, 175–214.
   Web of Science ®Google Scholar
• Schochet, P., and Chiang, H. (2010), Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains. NCEE 2010-4004. National Center for Education Evaluation and Regional Assistance. . Available at http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED511026&ERICExtSearch_SearchType_0=no&accno=ED511026.
   Google Scholar
• Toch, T., and Rothman, R. (2008). Rush to Judgment: Teacher Evaluation in Public Education, Washington, DC: Education Sector. . Available at http://www.educationsector.org/usr_doc/RushToJudgment_ES_Jan08.pdf.
   Google Scholar
• Todd, P., and Wolpin, K. (2003), “On the Specification and Estimation of the Production Function for Cognitive Achievement,” Economic Journal, 113(485), F3–F33.
   Web of Science ®Google Scholar
• Tucker, P. (1997), “Lake Wobegon: Where All the Teachers are Competent (Or, Have We Come to Terms With the Problem of Incompetent Teachers?),” Journal of Personnel Evaluation in Education, 11, 103–26.
   Google Scholar
• Wei, Y., and He, X. (2006), “Conditional Growth Charts,” The Annals of Statistics, 34, 2069–2097.
   Web of Science ®Google Scholar
• Weisburg, D., Sexton, S., Mulhern, J., and Keeling, D. (2009), The Widget Effect: Our National Failure to Acknowledge and Act of Differences in Teacher Effectiveness, New York: The New Teacher Project.
   Google Scholar
• Wright, P. (2010), An Investigation of Two Nonparametric Regression Models for Value-Added Assessment in Education, SAS. . Available at http://www.sas.com/resources/whitepaper/wp_16975.pdf.
   Google Scholar
• Wright, P., White, J., Sanders, W., and Rivers, J. (2010), “SAS EVAAS Statistical Models,” . SAS White Paper. Available at http://www. sas.com/resources/asset/SAS-EVAAS-Statistical-Models.pdf.
   Google Scholar