An ecological analysis of the Classroom Assessment Scoring System in K-1 Mexican classrooms

ABSTRACT

Measures of teaching are used internationally to understand and improve quality in early education with little consideration for ecological validity. In this study, we analyze videos gathered in 58 K-1 classrooms in Central Mexico to evaluate the validity and reliability of scores from an observational tool developed in the U.S. and used internationally: the Classroom Assessment Scoring System (CLASS). We use mixed methods to examine ecological validity to propose a revised, better-fitting three-factor solution of the CLASS in Mexico: Emotional Support (Positive Climate, Negative Climate), Social Relationships for Teaching (Teacher Sensitivity, Instructional Learning Formats, Behavior Management, Productivity), and Instructional Interactions (Regard for Student Perspectives, Concept Development, Quality of Feedback, Language Modeling). Generalizability study findings demonstrate moderate reliability across three sources of measurement error: raters, days, and video segments. These findings support the need for a ‘unified approach to validity’ to develop, adapt, and refine measures of teaching as a basis for enhancing teacher development and teaching quality in early education in Mexico and throughout Latin America.

KEYWORDS:

• Teaching quality
• mexico
• classroom observation
• generalizability theory
• validity

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. ‘Teaching quality’ refers to the processes and structures that are associated with delivering a given curriculum. Because several aspects of teaching quality are outside the immediate purview of the teacher – e.g., classroom resources, curricular content, class size – it is important to distinguish ‘teaching quality’ from ‘teacher quality.’ We argue for stronger measures of teaching quality to support rather than to penalize schools and teachers – for formative rather than summative or evaluative uses.

2. There are significant differences between early education in Mexico and in the US. For one, 3 years of preschool in Mexico (i.e., from ages 3- to 5-years old) are compulsory, though enforcement of this relatively recent policy differs across Mexico’s 32 states. Enrollments are lower in poorer states. Primary school begins in first grade (age 6) in Mexico, versus in kindergarten (age 5) in the US. Policy, curriculum, and teacher hiring are highly centralized in Mexico. In the US, policy is largely managed at the state level, and local school districts have significant latitude to choose their own curriculum, in response to state learning standards. Even private school programs in Mexico are required by law to implement national curricular materials. Nearly 9 in 10 children in Mexico attend public (rather than private) school, though the quality of public programs varies with widely by urbanicity. Schools in urban communities in Mexico have longer school hours, more resources, and teachers with more preparation than schools in rural or semi-rural communities. Mexican classrooms tend to be more communal, in general, than US classrooms (Jensen In Press). Partnership work and group solidarity are normative in public school classrooms in Mexico. So too are rote, teacher-centered instructional practices that limit teacher-child discourse and emphasize accuracy over analysis (Gallo In Press; Teague, Smith, and Jímenez 2010).

3. Research on the predictive validity of the CLASS for Hispanic (or Latino) students in the U.S. is mixed. Some studies (e.g., Downer et al. 2012) find significant and meaningful prediction, though others (e.g., López 2011) do not.

Additional information

Funding

This work was supported by the BYU McKay School of Education; University of California Institute for Mexico and the United States [2014 UC MEXUS-CONACYT].