Leveraging the Power of Observations: Locating the Sources of Error in the Individualized Classroom Assessment Scoring System

References

• Acar, I. H., Rudasill, K. M., Molfese, V., Torquati, J., & Prokasky, A. (2015). Temperament and preschool children’s peer interactions. Early Education and Development, 26, 479–495. doi:10.1080/10409289.2015.1000718
   Web of Science ®Google Scholar
• Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi:10.1146/annurev.psych.59.103006.093639
   PubMed Web of Science ®Google Scholar
• Bodrova, E., & Leong, D. J. (2006). Self-regulation as a key to school readiness: How early childhood teachers can promote this critical competency. In M. Zaslow & I. Martinez-Beck (Eds.), Critical issues in early childhood professional development (pp. 203–224). Baltimore, MD: Brookes.
   Google Scholar
• Bohlmann, N., Downer, J., Booren, L., Maier, M., & Williford, A. (2012). Using the individualized classroom assessment scoring system (inclass) to measure preschool children’s engagement with teachers, peers and tasks:examining measurement invariance across gender, ethnicity, and poverty status in three samples. Poster presented at the Head Start National Research Conference, Washington, DC.
   Google Scholar
• Bohlmann, N. L., & Downer, J. T. (2016). Self-regulation and task engagement as predictors of emergent language and literacy skills. Early Education and Development, 27, 18–37. doi:10.1080/10409289.2015
   Web of Science ®Google Scholar
• Booren, L. M., Downer, J. T., & Vitiello, V. E. (2012). Observations of children’s interactions with teachers, peers, and tasks across preschool classroom activity settings. Early Education and Development, 23, 517–538. doi:10.1080/10409289.2010.5487671046784
   PubMed Web of Science ®Google Scholar
• Buell, M., Han, M., & Vukelich, C. (2017). Factors affecting variance in classroom assessment scoring system scores: Season, context, and classroom composition. Early Child Development and Care, 187, 1635–1648. doi:10.1080/03004430.2016.1178245
   Web of Science ®Google Scholar
• Burts, D. C., Hart, C. H., Charlesworth, R., Fleege, P. O., Mosley, J., & Thomasson, R. H. (1992). Observed activities and stress behaviors of children in developmentally appropriate and inappropriate kindergarten classrooms. Early Childhood Research Quarterly, 7, 297–318. doi:10.1016/0885-2006(92)90010-V
   Google Scholar
• Carter, L. M., Williford, A. P., & LoCasale-Crouch, J. (2014). Reliability and validity of a measure of preschool teachers’ attributions for disruptive behavior. Early Education and Development, 25, 949–972. doi:10.1080/10409289.2014.898358
   Web of Science ®Google Scholar
• Cash, A. H., Hamre, B. K., Pianta, R. C., & Myers, S. S. (2012). Rater calibration when observational assessment occurs at large scale: Degree of calibration and characteristics of raters associated with calibration. Early Childhood Research Quarterly, 27, 529–542. doi:10.1016/j.ecresq.2011.12.006
   Web of Science ®Google Scholar
• Center for Advanced Study of Teaching and Learning. (2018). InCLASS observation: Individualized classroom assessment scoring system (InCLASS) pre-K technical manual. Charlottesville: University of Virginia. Retrieved from http://inclassobservation.com/research/inCLASSTechnicalManual.6.25.12.pdf
   Google Scholar
• Clopet, T. C., & Bulotsky-Shearer, R. (2016). A comprehensive look at peer social competence in preschool. NHSA Dialog, 19(1), 72–77. Retrieved from https://journals.uncc.edu/dialog/article/view/500/
   Google Scholar
• Copple, C., Bredekamp, S., Koralek, D. G., & Charner, K. (Eds.). (2013). Developmentally appropriate practice: Focus on preschoolers. Washington, D.C.: National Association for the Education of Young Children.
   Google Scholar
• Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY: Wiley.
   Google Scholar
• Curby, T. W., Downer, J. T., & Booren, L. M. (2014). Behavioral exchanges between teachers and children over the course of a typical preschool day: Testing bidirectional associations. Early Childhood Research Quarterly, 29, 193–204. doi:10.1016/j.ecresq.2014.01.002
   PubMed Web of Science ®Google Scholar
• Dev, D. A., Speirs, K. E., McBride, B. A., Donovan, S. M., & Chapman-Novakofski, K. (2014). Head start and child care providers’ motivators, barriers and facilitators to practicing family-style meal service. Early Childhood Research Quarterly, 29(4), 649–659. doi:10.1016/j.ecresq.2014.07.004
   Web of Science ®Google Scholar
• Downer, J. T., Booren, L. M., Lima, O. K., Luckner, A. E., & Pianta, R. C. (2010). The individualized classroom assessment scoring system (inCLASS): Preliminary reliability and validity of a system for observing preschoolers’ competence in classroom interactions. Early Childhood Research Quarterly, 25, 1–16. doi:10.1016/j.ecresq.2009.08.004
   PubMed Web of Science ®Google Scholar
• Gerson, S. A., & Woodward, A. L. (2012). A claw is like my hand: Comparison supports goal analysis in infants. Cognition, 122, 181–192. doi:10.1016/j.cognition.2011.10.014
   PubMed Web of Science ®Google Scholar
• Glenberg, A. M., Witt, J. K., & Metcalfe, J. (2013). From the revolution to embodiment: 25 years of cognitive psychology. Perspectives on Psychological Science, 8, 573–585. doi:10.1177/1745691613498098
   Web of Science ®Google Scholar
• Gordon, R. A., Hofer, K. G., Fujimoto, K. A., Risk, N., Kaestner, R., & Korenman, S. (2015). Identifying high-quality preschool programs: New evidence on the validity of the early childhood environment rating scale–revised (ECERS-R) in relation to school readiness goals. Early Education and Development, 26, 1086–1110. doi:10.1080/10409289.2015.1036348
   Web of Science ®Google Scholar
• Halle, T., Vick Whittaker, J. E., & Anderson, R. (2010). Quality in early childhood care and education settings: A compendium of measures. Washington, DC: Child Trends. Prepared by Child Trends for the Office of Planning, Research and Evaluation, Administration for Children and Families, US Department of Health and Human Services. Retrieved from https://www.acf.hhs.gov/sites/default/files/opre/complete_compendium_full.pdf
   Google Scholar
• Hamre, B. K. (2014). Teachers’ daily interactions with children: An essential ingredient in effective early childhood programs. Child Development Perspectives, 8, 223–230. doi:10.1111/cdep.12090
   Web of Science ®Google Scholar
• Harms, T., Clifford, R. M., & Cryer, D. (1998). Early childhood environment rating scale-revised. New York, NY: Teachers College Press.
   Google Scholar
• Hartz, K., Williford, A. P., & Koomen, H. M. Y. (2017). Teachers’ perceptions of teacher–Child relationships: Links with children’s observed interactions. Early Education and Development, 28, 441–456. doi:10.1080/10409289.2016.1246288
   Web of Science ®Google Scholar
• Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41, 56–64. doi:10.3102/0013189X12437203
   Web of Science ®Google Scholar
• Hofer, K. G. (2010). How measurement characteristics can affect ECERS-R scores and program funding. Contemporary Issues in Early Childhood, 11, 175–191. doi:10.2304/ciec.2010.11.2.175
   Google Scholar
• Iverson, J. M., & Thelen, E. (1999). Hand, mouth and brain. The dynamic emergence of speech and gesture. Journal of Consciousness Studies, 6(11–12), 19–44.
   Google Scholar
• Lawrence, S. M. (2018). Preschool children and iPads: Observations of social interactions during digital play. Early Education and Development, 29, 207–228. doi:10.1080/10409289.2017.1379303
   Web of Science ®Google Scholar
• Lemberger, M., Carbonneau, K.J., Atencio, D., Zieher, A., & Palacios, A. (2018). Growth effects for children participating in a mindfulness and social-emotional intervention. Journal of Counseling & Development. 96, 289–302. doi: 10.1002/jcad.12203
   Google Scholar
• Lemberger-Truelove, M., Carbonneau, K.J., Zieher, A., & Atencio, A. (in press). Support for the development and use of the child observation of mindfulness measure. Mindfulness. https://doi.org/10.1007/s12671-019-1094–5.
   Google Scholar
• Mantzicopoulos, P., French, B. F., Patrick, H., Watson, J. S., & Ahn, I. (2018). The stability of kindergarten teachers’ effectiveness: A generalizability study comparing the framework for teaching and the classroom assessment scoring system. Educational Assessment, 23, 24–46. doi:10.1080/10627197.2017.1408407
   Web of Science ®Google Scholar
• Morris, P., Millenky, M., Raver, C. C., & Jones, S. M. (2013). Does a preschool social and emotional learning intervention pay off for classroom instruction and children’s behavior and academic skills? Evidence from the foundations of learning project. Early Education and Development, 24, 1020–1042. doi:10.1080/10409289.2013.825187
   PubMed Web of Science ®Google Scholar
• National Association for the Education of Young Children. (2009). Developmentally appropriate practice in early childhood programs serving children from birth through age 8. Retrieved from https://www.naeyc.org/sites/default/files/globally-shared/downloads/PDFs/resources/position-statements/PSDAP.pdf
   Google Scholar
• National Research Council. (2008). Early childhood assessment: Why, what and how. Washington, D.C.: The National Academies Press.
   Google Scholar
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
   Google Scholar
• Peets, K., & Kikas, E. (2017). Teachers’ promotion or inhibition of children’s aggression depends on peer-group characteristics. Journal of Clinical Child & Adolescent Psychology, 46, 848–857. doi:10.1080/15374416.2015.1079778
   PubMed Web of Science ®Google Scholar
• Phillips, K., & Downer, J. (2017). Classroom context and years of teaching experience as predictors of misalignment on ratings of preschoolers’ classroom engagement. Early Education and Development, 28, 343–367. doi:10.1080/10409289.2016.1218730
   Web of Science ®Google Scholar
• Pianta, R. C., LaParo, K. M., & Hamre, B. K. (2008). Classroom assessment scoring system [CLASS] manual: Pre-K. Baltimore, MD: Brookes.
   Google Scholar
• Sabol, T. J., & Pianta, R. C. (2012). Patterns of school readiness forecast achievement and socioemotional development at the end of elementary school. Child Development, 83, 282–299. doi:10.1111/j.1467-8624.2011.01678.x
   PubMed Web of Science ®Google Scholar
• Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. London, England: Sage.
   Google Scholar
• Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44, 22–932. doi:10.1037/0003-066X.44.6.922
   Web of Science ®Google Scholar
• Slot, P. L., & Bleses, D. (2018). Individual children’s interactions with teachers, peers, and tasks: The applicability of the inCLASS pre-K in Danish preschools. Learning and Individual Differences, 61, 68–76. doi:10.1016/j.lindif.2017.11.003
   Web of Science ®Google Scholar
• Vitiello, V. E., Bassok, D., Hamre, B. K., Player, D., & Williford, A. P. (2018). Measuring the quality of teacher–Child interactions at scale: Comparing research-based and state observation approaches. Early Childhood Research Quarterly, 44, 161–169. doi:10.1016/j.ecresq.2018.03.003
   Web of Science ®Google Scholar
• Vitiello, V. E., Booren, L. M., Downer, J. T., & Williford, A. P. (2012). Variation in children’s classroom engagement throughout a day in preschool: Relations to classroom and child factors. Early Childhood Research Quarterly, 27, 210–220. doi:10.1016/j.ecresq.2011.08.005
   PubMed Web of Science ®Google Scholar
• Vitiello, V. E., Moas, O., Henderson, H. A., Greenfield, D. B., & Munis, P. M. (2012). Goodness of fit between children and classrooms: Effects of child temperament and preschool classroom quality on achievement trajectories. Early Education and Development, 23, 302–322. doi:10.1080/10409289.2011.526415
   Web of Science ®Google Scholar
• von Suchodoletz, A., Gunzenhauser, C., & Larsen, R. A. A. (2015). Die beobachtung von interaktionen im kindergartenalltag: Das individualized assessment scoring system (inclass) [Observing children’s interactions in kindergarten classrooms: The individualized assessment scoring system (inCLASS)]. Frühe Bildung [Early Education], 4, 211–217. doi:10.1026/2191-9186/a000207
   Google Scholar
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
   Google Scholar
• Williford, A. P., Maier, M. F., Downer, J. T., Pianta, R. C., & Howes, C. (2013). Understanding how children’s engagement and teachers’ interactions combine to predict school readiness. Journal of Applied Developmental Psychology, 34, 299–309. doi:10.1016/j.appdev.2013.05.002
   PubMed Web of Science ®Google Scholar
• Williford, A. P., Whittaker, J. E. V., Vitiello, V. E., & Downer, J. T. (2013). Children’s engagement within the preschool classroom and their development of self-regulation. Early Education and Development, 24, 162–187. doi:10.1080/10409289.2011.628270
   PubMed Web of Science ®Google Scholar