Language Shift and the Inclusion of Indigenous Populations in Large-Scale Assessment Programs

Abstract

Indicators of academic achievement for bilingual students can be inaccurate due to linguistic heterogeneity. For indigenous populations, language shift (the gradual replacement of one language by another) is a factor that can increase this heterogeneity and poses an additional challenge for valid testing. We investigated whether and how indigenous populations can be validly included in a large-scale assessment program. We gave Mexican preschool Mayan students aged 5 to 6 years the same set of mathematics items in three versions: (1) original in Spanish, (2) Mayan translation, and (3) content equivalent, developed from scratch in Mayan. Also, we collected information on the students’ and the teachers’ use of Spanish and Mayan, and on the communities’ support of the two languages. Students performed poorly on the three versions. Generalizability theory-based analyses revealed considerable performance inconsistency across items and language versions and low generalizability and dependability coefficients. This performance instability appears to stem from a dwindling support of the Mayan language in Mayan schools and communities. Fair, valid assessment of indigenous populations in either their languages or their countries’ dominant languages appears to be difficult to accomplish with current testing models and policies.

Keywords:

• fairness
• indigenous populations
• language shift
• large-scale assessment
• linguistic minorities
• validity

Notes

Academic language has been defined as the language used in “professional books, characterized by specific linguistic features associated with academic disciplines” (Scarcella, 2003, p. 9). It includes both the register of a discipline—which is defined in terms of the meanings underlying the grammar and words that are specific to a given discipline within a specific context (Halliday, 1978)—and the ways in which language is used to socialize within the context of a discipline.

Generalizability studies were performed with GENOVA, originally designed by R. L. Brennan and coded by J. E. Crick. This software is publicly available at http://www.uiowa.edu/∼casma/computer_programs.htm.

Version is a random facet because any item belonging to the same version can be worded and illustrated in multiple possible ways.