Is validation always valid? Cross-cultural complexities of standard-based instruments migrating out of their context

ABSTRACT

The international application of standard-based measures to assess ECEC quality raises crucial questions concerning the cultural complexities and the problematic validity of instruments migrating out of their cultural cradle; nevertheless the topic has received only marginal attention in the literature. This paper, which aims to address this gap, presents the research design and first findings from the Italian data collected within the European study CARE and a national extension, developing a critical-cultural approach to the Classroom Assessment Scoring System (CLASS) – a standard-based instrument to evaluate teacher–child interactions in U.S. classrooms. ECEC experts and teachers have been involved in focused discussions of CLASS recognizing elements of continuity, differences and disagreements and key-features of the teacher–child relationship not captured by the tool. Results offer interesting insights into a methodological reflection on the international use of standardized evaluation tools and into a theoretical reflection on ‘universal vs. culture-related’ views on education and quality.

KEYWORDS:

• Cross-cultural research
• early childhood education
• quality assessment
• standardized instruments
• validation process
• professional reflexivity

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. 7th Framework Programme SSH-2013, Project coordinator Paul Leseman (Utrecht University). The Italian research team, supervised by Professor Susanna Mantovani, consists of researchers from University of Milan-Bicocca – Department of Human Sciences (G. Pastori, C. Bove, P. Braga, F. Zaninelli, S. Cescato, V. Pagani, G. Banzi e T. Morgandi) and from Reggio Children (C. Giudici, C. Rinaldi, P. Cagliari, M. Castagnetti, S. Bonilauri, L. Colla, M. Ruozzi, M. Nicolosi – till November 2014). Besides Italy, Belgium, Denmark, Finland, Germany, Greece, the Netherlands, Norway, Poland, Portugal and England are also taking part in the CARE project.

2. England, Finland, Germany, Italy, the Netherlands, Poland, and Portugal.

3. M.K. Lekkanen (Jyvaskyla University) is the international leader of the WP2 (Curriculum, Pedagogy, and Classrooom quality: promoting effectiveness of ECEC, Task 2.3); P. Slot (Utrecht University), J. Cadima (Instituto Universitário de Lisboa) and J. Salminen (Jyvaskyla University) are responsible for the quantitative analysis. J. Salminen is also conducting a qualitative discourse analysis on educational dialogues.

4. Our greatest gratitude to professor Susanna Mantovani, who inspired the research and gave her support and insights.

5. The same methodology will be applied also in Portugal and the Netherlands, furthering the discussion to other countries and enabling broader cross-cultural comparison.

6. The national extension (Pagani, doctoral thesis, in progress/forthcoming), besides extending the qualitative critical-cultural analysis started within the CARE project, is complementing/has complemented the qualitative exploration with a quantitative analysis of the tool adopting a mixed-methods research design (Creswell and Plano Clark 2011). Quantitative data (i.e. preschool and infant–toddler centres classroom observations coded using, respectively, the CLASS Pre-K and Toddler) were used to test even at statistical level the applicability and generalizability of the CLASS framework to the Italian ECEC context.