Measuring Cultural Dimensions of Classroom Interactions

ABSTRACT

We trace the development and analyze the generalizability of the Classroom Assessment of Sociocultural Interactions (CASI), an observation system designed to measure cultural dimensions of classroom interactions. We establish CASI measurement properties by analyzing panoramic videos of 4th and 5th grade classrooms from the Measures of Effective Teaching project, and argue for its significance in terms of achievement opportunity for minoritized students and needed evidence regarding equitable teaching. We frame ten dimensions of sociocultural interactions within three domains: Life Applications (i.e., connections with what students know and do outside of school); Self in Group (i.e., interdependence to motivate learning and foster social identities); and Agency (i.e., how freedom and choice are managed).

We demonstrate how measurement error is associated with raters, lessons, and lesson segments, and discuss implications for CASI refinement, as well as appropriate instrument uses to enrich learning opportunities for minoritized students across a variety of classroom settings.

Acknowledgments

This work has benefitted from collaborations and discussions with several colleagues. In particular, we would like to acknowledge and thank Margaret Bridges, Stephanie Curenton, Jason Downer, Tonia Durden, Peggy Estrada, Bruce Fuller, Ronald Gallimore, Eugene García, Claude Goldenberg, Kris Gutiérrez, Bridget Hamre, Marisha Humphries, Iheoma Iruka, Robert Jiménez, Heather HyunJung Kim, Carol Lee, Rebeca Mejía-Arauz, Leslie Reese, and Robert Rueda. Financial support for this work came from an early career research grant from the National Academy of Education as well as from the BYU McKay School of Education.

Notes

^1. Scholars (e.g., Flores & Rosa, 2015; González, 2005) use the term minoritized instead of minority to emphasize that children of color often do not have access to equitable opportunity even when they comprise the numerical majority in many communities. They remain excluded and relegated to an underprivileged status. The term, however, does not strictly refer to racial or ethnic groups. It can also refer to any group of students who are “devalued in society” (Sensoy & DiAngelo, 2012, p. 5), due to differences in social class, gender, religion, native language, etc.

^2. We use the terms protocol, instrument, and measure interchangeably. We use the term “rubrics” to describe CASI scoring guides, composed of descriptive statements characterizing points along a scoring continuum. CASI rubrics are used to score classroom interactions at the indicator level. Markers are within indicators, indicators within dimensions, and dimensions within domains. We use “construct” generically—to reference markers, indicators, dimensions, and domains.

^3. Competition is scored in the opposite direction of other Self in Group dimensions. More competitive classrooms are associated with more “independent” interactions—less competitive classrooms indicate more “interdependent” interactions. Competition and collaboration are related, but not opposite ends of the same continuum. Classroom interactions, in other words, can simultaneously be collaborative and competitive, like when small groups of students compete against each other in an exam-review game.

^4. Originally we planned to include Responsibility as a fourth dimension to Agency, but we could not identify enough markers that met our inclusion criteria. Instead we omitted Responsibility and added relevant indicators and markers to the Autonomy dimension.

^5. Treating segment as crossed with lesson allowed us to determine whether, across teachers and lessons, earlier (first) versus later (second) segments differ systematically in some way. Instead we could have treated segment as nested within lesson, but that design would have provided less information. With the segment-crossed design we chose to use, we were able to obtain separate estimates for segment, for the segment by teacher interaction, and for the segment by lesson-within-teacher interaction. If we had treated segment as nested within lesson, these three variance components would have been confounded. Similarly, the rater by segment interaction, the teacher by rater by segment interaction, and the lesson-within-teacher by rater by segment interaction would have been confounded. Reliability estimates were not affected by the choice of one of these models versus the other.

^6. For readers familiar with the standard terminology for generalizability studies (e.g., Brennan, 2001), the full univariate model for each Domain or Dimension was t x r x s x l:t. The three submodels were t x r x l:t (run separately for segment 1 versus segment 2); t x r x s (run separately for the three lessons); and t x r x l*:t, where l* was a six-level facet formed by collapsing segment and lesson to form a single facet.

^7. The Spearman-Brown formula is as follows: ρ*_xx’ = (N * ρ_xx’) / (1 + (N – 1) * ρ_xx’) where ρ*_xx’ is the predicted reliability, ρ_xx’ is the estimated inter-rater reliability, and N is the number of raters.