Designing a Normed Receptive Vocabulary Test for Bilingual Populations: A Model from Welsh

Abstract

The purpose of this paper is to propose an applied model for the assessment of bilingual children's language abilities with standardised tests. We discuss the purposes of such tests, especially in relation to vocabulary knowledge, and potential applications of test results for each of those purposes. The specific case to be examined here is that of Welsh vocabulary, but the principles are relevant to any other bilingual population and any other aspect of language. Our proposal is that all standardised language tests for bilinguals should take into account children's exposure to the language in question and should report both a general score comparing a given child to all children – bilingual or monolingual – and a score that indicates the child's placement relative to (bilingual) children who have similar language exposure profiles.

Keywords:

• Language Testing
• Bilingual Acquisition
• Language Achievement

Notes

1. This is to be contrasted with criterion-referenced testing or curriculum related assessment, in which a given individual's performance is measured against some instructional objective, usually identified by some experts (Brown & Hudson, 2002; Popham, 1975). Criterion-referenced testing is ‘closely linked to a curriculum’ and could reflect ‘differences in pedagogical decision-making’ (Lee, 2003). A criterion-referenced test, for example, might be used in relation to second-language teaching, in which explicit stated goals are set at the outset of the course structure (see Bond, 1996). A norm-referenced language test assumes, in contrast, that children generally develop language naturally and appropriately, so the norms capture what the natural stages in development are observed to be. Such a normed test for vocabulary or other aspect of language can of course be used in conjunction with a criterion-referenced test. The choice will depend on the goals and purposes of the use of the test, as well as on the linguistic and cultural diversity of the groups to be assessed (e.g. Frederickson & Cline, 1996).

2. Note, however, that this does not necessarily go in both directions: a child can have difficulties with both languages for other reasons as well – e.g. limited input in both.

3. We also considered classifying participants according to the language of instruction in the school. However, reliable data concerning the language of instruction in each school was not available. If such information is available for other children in particular bilingual settings, such information can be taken into consideration in the norming of a test for that group of bilinguals. It is worth noting that in our previous work, home language has been much more predictive of performance on Welsh than school language (see, e.g. Gathercole & Thomas, 2005; Gathercole et al., 2005).

4. This information was supplemented in some cases by a further detailed questionnaire handed out to parents/guardians. A few of those questionnaires (145) were returned. In three cases, the parents did not specify home language information. For the remaining 142 cases, 83.1% (i.e. 118) of the responses matched exactly. And if we allow that a difference in response of one degree (e.g. a parent answers OWH, the child WEH, or a parent answers WEH and the child OWH) is closer than one of two degrees (one answers OWH, the other OEH), correlations between children's responses and those of their parents/guardians were high: r=0.882. (In fact there was no case of a discrepancy of two degrees.) The discrepancies were about equal in whether the child reported more or less Welsh than the parent. However, interestingly, where the child and the adult differed in their classification, the child was more likely to include both languages (WEH) when the parent answered only one (OWH or OEH) – 75% of the cases – than vice versa.We do not wish to suggest that we underestimate the difficulty of measuring the level of input a given child has in each of his/her languages and recognise that the input can vary according to social and cultural variables as well (e.g. Ortiz & Yates, 2001). However, we were seeking a tool that could be used fairly easily as an initial screening measure by teachers and language professionals, but that would take into account potential language background differences across children. For these reasons, all children were assigned to their language group based on their own responses on the initial sheet. It is clear, however, that when a given child's language background or level of ability with Welsh is in question, the researcher or professional should seek further confirmation and exploration of the child's case before drawing any hard and fast conclusions.

5. When there is doubt about the proper categorisation by Home Language, the most conservative choice should be made. That is, if it is not clear whether the child should be assigned to the OWH or the WEH group, s/he should be assigned to the WEH group; if it is unclear whether a child should be assigned to the WEH or the OEH group, s/he should be assigned to the OEH group.Also, in a few cases, a given child may be learning Welsh alongside a language other than English (e.g. Welsh and French, Welsh and Urdu, etc.). If such a child is growing up with just two languages, the standardisation norms that should be consulted should be those of the WEH or OEH group, the former if the two languages are both used in the home, the latter if the non-Welsh language is the language of the home.In some other cases, a child may be learning more than two languages (e.g. Welsh, English and Urdu). In such a case, the standardised norms can only be taken as suggestive. Either the WEH or OEH norms should be consulted, depending on the amount of Welsh used in the home: WEH if Welsh is used at all in the home; OEH if Welsh is not used at all in the home.