The stability of kindergarten teachers’ effectiveness: A generalizability study comparing the Framework For Teaching and the Classroom Assessment Scoring System

References

• American Educational Research Association. (2015). AERA statement on use of value-added models (VAM) for the evaluation of educations and educator preparation programs. Educational Researcher, 44, 448–452.
   Web of Science ®Google Scholar
• American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
   Google Scholar
• Atilgan, H. (2013). Sample size for estimation of G and Phi coefficients in generalizability theory. Eurasian Journal of Educational Research, 51, 215–228.
   Google Scholar
• Barnett, W. S. (1993). Benefit-cost analysis of preschool education: Findings from a 25-year follow-up. American Journal of Orthopsychiatry, 64, 500–508.
   Google Scholar
• Bell, C. A., Qi, Y., Croft, A. J., Leusner, D., McCaffrey, D. F., Gitomer, D. H., & Pianta, R. C. (2014). Improving observational score quality: Challenges in observer thinking. In T. J. Kane, K. A. Kerr, & R. C. Pianta (Eds.), Designing teacher evaluation systems (pp. 50–97). San Francisco, CA: Jossey Bass.
   Google Scholar
• Berliner, D. (2014). Exogenous variables and value added assumptions: A fatal flaw. Teachers College Record, 116, 1–31.
   PubMed Web of Science ®Google Scholar
• Berrueta-Clement, J. R., Schweinhart, L. J., Barnett, W. S., Epstein, A. S., & Weikart, D. P. (1985). Changed lives: The effects of the Perry Preschool Program on youths through age 19. Ypsilianti, MI: High Scope.
   Google Scholar
• Bill & Melinda Gates Foundation. (2013). Measures of effective teaching. Retrieved from http://www.metproject.org/
   Google Scholar
• Brennan, R. L. (2001). Generalizability theory. New York, USA: Springer-Verlag.
   Google Scholar
• Brennan, R. L. (2011). Using generalizability theory to address reliability issues for PARCC assessments: A white paper. Center for Advanced Studies in Measurement and Assessment (CASMA). University of Iowa. Iowa City, IA, USA.
   Google Scholar
• Brophy, J. (1988). Educating teachers about managing classrooms and students. Teaching and Teacher Education, 4, 1–18.
   Web of Science ®Google Scholar
• Brophy, J. (2006). Observational research on generic aspects of classroom teaching. In P. A. Alexander & P. H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 755–780). Mahwah, NJ: Erlbaum.
   Google Scholar
• Brophy, J., Coulter, C. L., Crawford, J., Evertson, C. M., & King, C. E. (1975). Classroom observation scales: Stability across time and context and relationships with student learning gains. Journal of Educational Psychology, 67, 873–881.
   Web of Science ®Google Scholar
• Calkins, D., Borich, G. D., Pascone, M., Kluge, S., & Marston, P. T. (1997). Generalizability of teacher behaviors across classroom observation systems. Journal of Classroom Interaction, 13, 9–22.
   Google Scholar
• Campbell, F. A., & Ramey, C. T. (1994). Effects of early intervention on intellectual and academic achievement: A follow-up study of childen from low-income families. Child Development, 65, 684–698.
   PubMed Web of Science ®Google Scholar
• Center for Advanced Study of Teaching and Learning. (2015). Measuring and improving teacher-student interactions in PK-12 settings to enhance students’ learning. Retrieved from http://curry.virginia.edu/uploads/resourceLibrary/CLASS-MTP_PK-12_brief.pdf
   Google Scholar
• Center on Great Teachers & Leaders. (2013). Database on state teacher and principal evaluation policies. Retrieved from http://resource.tqsource.org/stateevaldb/
   Google Scholar
• Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Philadelphia, PA, USA: Harcourt.
   Google Scholar
• Cronbach, L. J., Gleser, G., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY, USA: Wiley.
   Google Scholar
• Curby, T. W., Grimm, K. J., & Pianta, R. (2010). Stability and change in early childhood classroom interactions during the first two hours of a day. Early Childhood Research Quarterly, 25, 373–384.
   Web of Science ®Google Scholar
• Curby, T. W., Stuhlman, M., Grimm, K., Mashburn, A., Chomat-Mooney, L., Downer, J., … Pianta, R. C. (2011). Within-day variability in the quality of classroom interactions during third and fifth grade: Implications for children’s experiences and conducting classroom observations. Elementary School Journal, 112, 16–37.
   Web of Science ®Google Scholar
• Danielson, C. (2007). Enhancing professional practice: A framework for teaching (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
   Google Scholar
• Danielson, C. (2013). The framework for teaching evaluation instrument (2013 ed.). Retrieved from http://www.teachscape.com/frameworkforteaching/home
   Google Scholar
• Danielson, C., & Dwyer, C. (1995). How Praxis III supports beginning teachers. Educational Leadership, 52, 66–67.
   Web of Science ®Google Scholar
• Darling-Hammond, L. (2009). Recognizing and enhancing teacher effectiveness. International Journal of Educational and Psychological Assessment, 8, 1–24.
   Google Scholar
• Dee, T., & Wyckoff, J. (2015). Incentives, selection, and teacher performance: Evidence from impact. Journal of Policy Analysis and Management, 34, 267–297.
   Web of Science ®Google Scholar
• Dwyer, C. A. (1994). Criteria for performance-based teacher assessments: Validity, standards, and issues. Journal of Personnel Evaluation in Education, 8, 135–150.
   Google Scholar
• Emmer, E. T., & Peck, R. F. (1973). Dimensions of classroom behavior. Journal of Educational Psychology, 64, 223–240.
   Web of Science ®Google Scholar
• Evertson, C. M., & Emmer, E. T. (1982). Effective management at the beginning of the school year in junior high classes. Journal of Educational Psychology, 74, 485–498.
   Web of Science ®Google Scholar
• Garrett, R., & Steinberg, M. P. (2015). Examining teacher effectiveness using classroom observation scores: Evidence from the randomization of teachers to students. Educational Evaluation and Policy Analysis, 37, 224–242.
   Web of Science ®Google Scholar
• Good, T. L. (1979). Teacher effectiveness in the elementary school. Journal of Teacher Education, 30, 52–64.
   Web of Science ®Google Scholar
• Good, T. L. (1983). Classroom research: A decade of progress. Educational Psychologist, 18, 127–144.
   Google Scholar
• Good, T. L., & Lavigne, A. L. (2014). Issues of teacher performance stability are not new: Limitations and possibilities. Education Policy Analysis Archives, 23(2). doi:10.14507/epaa.v23.1916
   Google Scholar
• Grimm, K. J., Curby, T. W., Pianta, R. C., Mashburn, A. J., Downer, J., Chomat-Mooney, L., & Hamre, B. (2008, March). Partitioning variance associated with classroom observations. Paper presented at the annual meeting of the American Educational Research Association, New York.
   Google Scholar
• Hamre, B. K. (2009, April). Using classroom observation to gauge teacher effectiveness: Classroom Assessment Scoring System (CLASS). Presentation for the National Comprehensive Center for Teacher Quality’s Evaluating Teacher Effectiveness. Retrieved from http:/www.gse.harvard.edu/cepr-resources/files/news-events/ncte-conference-class-hamre.pdf
   Google Scholar
• Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56–64.
   Web of Science ®Google Scholar
• Ho, A. D., & Kane, T. J. (2013). The reliability of classroom observations by school personnel. Retrieved from http://www.metproject.org/downloads/MET_Reliability%20of%20Classroom%20Observations_Research%20Paper.pdf
   Google Scholar
• Hull, J. (2013). Trends in teacher evaluation. Retrieved from http://www.centerforpubliceducation.org/Main-Menu/Evaluating-performance/Trends-in-Teacher-Evaluation-At-A-Glance/Trends-in-Teacher-Evaluation-Full-Report-PDF.pdf
   Google Scholar
• Hurlihy, C., Karger, E., Pollard, C., Hill, H. C., Kraft, M. A., Williams, M., & Howard, S. (2014). State and local efforts to investigate the validity and reliability of scores from teacher evaluation systems. Teachers College Record, 116, 1–28.
   Web of Science ®Google Scholar
• Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment. MET Project Research Paper, Bill & Melinda Gates Foundation.
   Google Scholar
• Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching. Retrieved from http://www.metproject.org/downloads/MET_Gathering_Feedback_Research_Paper.pdf
   Google Scholar
• Kimball, S. M., & Milanowski, A. (2009). Examining teacher evaluation validity and leadership decision-making within a standards-based evaluation system. Educational Administration Quarterly, 45, 34–70.
   Web of Science ®Google Scholar
• Kolen, M. J. (2006). Scaling and norming. Educational Measurement, 4, 156–186.
   Google Scholar
• Lakes, D. K., & Hoyt, W. T. (2010). Applications of generalizability theory to clinical child and adolescent psychology research. Journal of Clinical & Adolescent Psychology, 38, 144–165.
   Web of Science ®Google Scholar
• Lazar, I., & Darlington, R. (1982). Lasting effects of early education. Monographs of the Society for Research in Child Development, 47(2–3, Serial No. 195).
   Web of Science ®Google Scholar
• Malmberg, L., Hagger, H., Burn, K., Mutton, T., & Colls, H. (2010). Observed classroom quality during teacher education and two years of professional practice. Journal of Educational Psychology, 102, 916–932.
   Web of Science ®Google Scholar
• Mashburn, A. J., Meyer, J. P., Allen, J. P., & Pianta, R. C. (2014). The effect of observation length and presentation order on the reliability and validity of an observational measure of teaching quality. Educational and Psychological Measurement, 74, 400–422.
   Web of Science ®Google Scholar
• McCaffery, D. F., Sass, T. R., Lockwood, J. R., & Mihaly, K. (2009). The intertemporal variability of teacher effect estimates. Education Finance and Policy, 4, 572–606.
   Web of Science ®Google Scholar
• Mihaly, K., & McCaffrey, D. F. (2014). Grade-level variation in observational measures of teacher effectiveness. In T. J. Kane, K. A. Kerr, & R. C. Pianta (Eds.), Designing teacher evaluation systems: New guidance from the measures of effective teaching project (pp. 9–49). San Francisco, CA: Jossey-Bass.
   Google Scholar
• Milanowski, A. T., Heneman, III, H. G., & Kimball, S. M. (2011). Teaching assessment for teacher human capital management: Learning from the Current State of the Art (WCER Working Paper No. 2011-2). Retrieved from http://www.wcer.wisc.edu/publications/workingpapers/Working_Paper_No_2011_02.pdf
   Google Scholar
• National Council on Teacher Quality. (2013). State of the states 2013. Connect the dots: Using evaluations of teacher effectiveness to inform policy and practice. Retrieved from http://www.nctq.org/dmsView/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report
   Google Scholar
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY, USA: McGraw-Hill.
   Google Scholar
• Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237–257.
   Web of Science ®Google Scholar
• Office of Head Start. (2014). Use of Classroom Assessment Scoring System (CLASS) in head start. Retrieved from http://eclkc.ohs.acf.hhs.gov/hslc/sr/quality/class/docs/use-of-class.pdf
   Google Scholar
• Patrick, H., & Mantzicopoulos, P. (2014). Engaging young children with informational books. Thousand Oaks, CA: Corwin Press.
   Google Scholar
• Patrick, H., & Mantzicopoulos, P. (2016). Is effective teaching stable? Journal of Experimental Education, 84, 23–47.
   Web of Science ®Google Scholar
• Patrick, H., Mantzicopoulos, P., & Sears, D. (2012). Effective classrooms. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook. Volume 2: Individual differences and cultural and contextual factors (pp. 443–469). Washington, DC: American Psychological Association.
   Google Scholar
• Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom assessment scoring system manual K-3. Baltimore, MA: Brookes Publishing.
   Google Scholar
• Plank, S. B., & Condliffe, B. F. (2013). Pressures of the season: An examination of classroom quality and high-stakes accountability. American Educational Research Journal, 50, 1152–1182.
   Web of Science ®Google Scholar
• Praetorius, A., Pauli, C., Reuseer, K., Rakoczy, K., & Klieme, E. (2014). One lesson is all you need? Stability of instructional quality across lessons. Learning and Instruction, 31, 2–12.
   Web of Science ®Google Scholar
• Schweinhart, L., Weikart, D., & Larner, M. (1986). Consequences of three preschool curriculum models through age 15. Early Childhood Research Quarterly, 1, 15–45.
   Google Scholar
• Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. London, England: Sage.
   Google Scholar
• Shavelson, R. J., Webb, N. M., & Burstein, L. (1986). Measurement of teaching. In M. Wittrock (Ed.), Handbook of research on teaching (pp. 50–91). New York, NY: McMillan.
   Google Scholar
• Shinn, Y., & Raudenbush, S. W. (2012). Confidence bounds and power for the reliability of observational measures on the quality of a social setting. Psychometrika, 7, 543–560.
   Google Scholar
• Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–22.
   Web of Science ®Google Scholar
• Swiss Society for Research in Education Working Group. (2006). EDUG user guide. Neuchatel, Switzerland: IRDP.
   Google Scholar
• Sullivan, J. P. (2012). A collaborative effort: Peer review and the history of teacher evaluations in Montgomery County, Maryland. Harvard Educational Review, 1, 142–152.
   Google Scholar
• Thissen, D. (2016). Commentray on the assessment of fairness of comparisons under divergent measurement conditions. In N. J. Dorans & L. L. Cook (Eds.), Fairness in educational assessment and measurement (pp. 203–214). New York: Routledge. NY, USA.
   Google Scholar
• U.S. Department of Education. (2011). Fact sheet: Bringing flexibility and focus to education law. Retrieved from http://www.whitehouse.gov/sites/default/files/fact_sheet_bringing_flexibility_and_focus_to_education_law_0.pdf
   Google Scholar
• U.S. Department of Education. (2012). ESEA flexibility: Flexibility to improve student academic achievement and increase the quality of instruction. Retrieved from http://www2.ed.gov/policy/elsec/guid/esea-flexibility/index.html
   Google Scholar
• Van Horn, L. M., Karlin, E. O., Ramey, S. L., Aldridge, J., & Snyder, S. W. (2005). Effects of developmentally appropriate practices on children’s development: A review of research and discussion of methodological and analytic issues. Elementary School Journal, 105, 325–351.
   Web of Science ®Google Scholar
• Weinstein, C. S., Romano, M. E., & Mignano, Jr., A. J. (2011). Elementary classroom management: Lessons from research and practice (5th ed.). New York, NY, USA: McGraw Hill.
   Google Scholar
• Whitehurst, G. J., Chingos, M. M., & Lindquist, K. (2014). Evaluating teachers with classroom observations. Lessons learned in four districts. Retrieved from http://www.brookings.edu/~/media/research/files/reports/2014/05/13-teacher-evaluation/evaluating-teachers-with-classroom-observations.pdf
   Google Scholar