A Framework for Developing Comparable Multilingual Assessments for Minority Populations: Why Context Matters

Abstract

The assessment of linguistic minorities often involves using multiple language versions of assessments. In these assessments, comparability of scores across language groups is central to valid comparative interpretations. Various frameworks and guidelines describe factors that need to be considered when developing comparable assessments. These frameworks provide limited information in relation to the development of multiple language versions of assessments for assessing linguistic minorities within countries. To this end, we make various suggestions for the types of factors that should be considered when assessing linguistic minorities. Our recommendations are tailored to the particular constraints potentially faced by various jurisdictions tasked with developing multiple language versions of assessments for linguistic minorities. These challenges include having limited financial and staffing resources to develop comparable assessments and having insufficient sample sizes to perform psychometric analyses (e.g., item response theory) to examine comparability. Although we contextualize our study by focusing on linguistic minorities within Canada due to its bilingual status, our findings may also apply to other bilingual and multilingual countries with similar minority/majority contexts.

Keywords:

• comparability
• linguistic minorities
• validity

Notes

In some jurisdictions, a percentage of the assessment scores are used for final marks in the assessed subject; in some occasions (e.g., Alberta, Quebec), that percentage is up to 50%, thus raising the stakes of the assessment for individual students (Simon, van Barneveld, King, & Nadon, 2011).

Even if results of student assessments do not lead to penalties or rewards for teachers and schools, educators teaching grades assessed by these tests often feel responsible for their schools’ results. There are instances when teachers have requested to not teach these grades based on their perceived stakes of these assessments (Klinger & Rogers, 2011). One reason for the perceived higher stakes is publicized school rankings which lead to inappropriate interpretations of test scores as indicators of school quality and effectiveness (Ercikan & Barclay-McKeown, 2007).

The survey is available upon request from the corresponding author.

Having high assimilation rates presents difficulties for (Francophone) linguistic minority groups to retain their cultural and linguistic identity given high rates of assimilation to the linguistic majority (Anglophone) populations.