LITMUS-CLT: A new way to assess bilingual lexicons

All words are learned. The only source of words for children acquiring a language is the surrounding social environment. How many and which words children learn depend crucially on how many and which words they encounter. Bilinguals are prone to have smaller vocabularies (in one language) compared to monolinguals since their immersion in a language is split between the two systems. This may lead to an incorrect diagnosis of language disorder in bilingual children, since children with Specific Language Impairment (SLI) are usually late talkers and lag behind typically developing children with respect to vocabulary size during preschool. On the other hand, bilingual children with a language disorder also run the risk of not being diagnosed, since a smaller vocabulary is expected in a bilingual child. Thus, an accurate assessment of vocabulary size of bilingual children in both of their languages is needed to differentiate between children with typical, balanced language development (similar vocabulary size in both languages, smaller than in monolingual children but typical of bilingual ones), unbalanced language development (significantly smaller vocabulary in one language only, but typical of bilinguals in the other one) and suppressed vocabulary in both languages (suggesting a risk for SLI).

Most normed vocabulary tests are designed for monolingual populations, and bilingual children are typically tested in only one language, often in their L2 – the majority language. In research settings, testing in both languages is more widespread, but still only a limited set of language pairs have been studied. Since bilingualism and multilingualism have become the norm in today’s societies, there is a substantial need for standardised and normed tools enabling impartial assessment of vocabulary in children acquiring more than one language.

In the COST Action IS0804 (2010–2013) “Language Impairment in a Multilingual Society: Linguistic Patterns and the Road to Assessment” (www.bi-sli.org), the problem of disentangling bi/multilingualism from language impairment was addressed, and a battery of tools under the umbrella name of LITMUS (Language Impairment Testing in Multilingual Settings) was developed (Armon-Lotem, de Jong, & Meir, 2015).

One of these tools is the Cross-linguistic Lexical Tasks (LITMUS-CLT), which assesses both receptive and expressive knowledge of nouns and verbs through a picture-based test. The tool represents an innovative method for constructing picture lexical tasks for preschool children (in the age range 3–6 years) that are comparable across a wide range of languages (Haman, Łuniewska, & Pomiechowska, 2015). Instead of translating one test between languages, CLT is developed for a multitude of languages simultaneously, based on the same underlying criteria of cultural and linguistic equivalence for target word selection. Thus, every language version is based on the same general principles and has the same overall structure, but has its own, unique composition of lexical items that are not identical, but selected according to the exactly same criteria of word’s complexity and age of acquisition (AoA). This way, any combination of CLT language versions can be used for a variety of bilingual populations. So far, the CLT has been developed for 25 languages, thus yielding a high number of possible combinations for assessment of multilingual children (see http://psychologia.pl/clts/for updated information about the new developments in CLT).

A thorough description of the CLT, its construction and the rationale behind it can be found in Haman et al. (2015). It is also presented in the article opening this special issue (Haman et al., this issue). Here, we will give a short overview of how the tool was constructed and briefly comment the results obtained so far (mostly presented in this volume) in the context of the ultimate aims of COST Action IS0804 and the use of the LITMUS battery.

As a starting point, a large set of more than 1000 pictures (for objects and actions), accessed from a variety of previous studies and differing in style (from black and white line pictures to colour photos), was named and rated by competent judges in 34 languages. This process resulted in a set of 299 words that reliably evoked a single word with the same English equivalent across the 34 languages, and new pictures unified in style (coloured line drawings) were drawn for these words in accordance with ratings of drawing style.

To further ensure equivalence across languages for target word selection, the 299 words were assessed for subjective AoA through asking at least 20 L1 speakers of each language to rate how old they thought they were when they acquired each of these words. For details of this study, see Łuniewska, Haman, Armon-Lotem et al. (2016). AoA has been found to be a robust measure for lexical assessment in studies of children as well as adults (see, e.g. Juhasz, 2005), and it has been found to correlate with objective measures of acquisition age like MacArthur-Bates Communicative Development Inventories (CDI) norms across several languages (Łuniewska et al., 2016; Lind, Simonsen, Hansen, Holm, & Mevik, 2015). Based on the AoA assessment, nouns and verbs were divided into two equally large groups, early and late, which were further used in matching target words for comprehension and production tasks.

Finally, to make the target words comparable in linguistic complexity, a composite measure called the complexity index (CI) was created. It has four components related to phonological, morphological and etymological features as well as exposure rated for each word by L1 speakers of each language with a background in linguistics or a related field. Details about the rationale behind the CI and the calculations of the CI score are found in Haman et al. (2015). Based on the CI score, the nouns and the verbs were (like for the AoA explained above) divided into two equally large groups: high and low. As with the AoA ratings, these CI categories were further used for matching target words across comprehension and production tasks.

For each language, a total of 128 target words and 128 distractors (for the comprehension tasks) were then selected from among the pool of 299 candidate words with the use of AoA and CI values. The tool consists of four subtasks: noun comprehension, verb comprehension, noun production and verb production, with 30 target words (plus two training items) in each task. For the comprehension tasks, the child is presented with four pictures (one target and three distractors) and should identify the picture matching the target word. For the production tasks, the child should name the picture shown – an object or an action – with the corresponding noun or verb. The final CLT comes in two versions: one traditional printed booklet version and one electronic version where the instructions are prerecorded in each language. One of the advantages with the electronic version is that it can also record reaction times. In the papers presented in this volume, both versions have been used.

As already mentioned, CLT includes comprehension and production tasks for both nouns and verbs. This makes it possible to make a fair assessment of the vocabulary in both languages, but in addition specific profiles may be obtained including language modes (production/comprehension) and word classes (nouns/verbs) for the child. This can be crucial for SLI diagnosis since SLI children are often claimed to have word finding difficulties (production is more severely suppressed than comprehension) and to have more problems with processing of verbs than nouns.

So far, the CLTs are not normed for any population in any language, but the research presented in the papers below will show how important it is to study not only different populations of children but also to make methodological analyses of the tools used before norming. Thanks to this series of individual studies where CLT was used as an experimental method rather than a psychometric tool in a proper sense, we are in a better position to go ahead with norming the CLTs for clinical use.

This special issue of Clinical Linguistics & Phonetics presents Cross-linguistic Lexical Tasks and some of its applications. The papers build on a symposium on the CLTs held at the 13th International Congress for the Study of Child Language in Amsterdam in July 2014 and show the application of CLT in a variety of linguistic, social and cultural contexts to monolingual and bilingual children, typically developing (TD) as well as SLI groups. The papers illuminate a varied set of aspects connected to this assessment tool: its comparability across languages; its relevance for assessing language dominance in bilingual children; the effects of the psycholinguistic properties underlying the tool; its psychometric properties compared to other assessment tools; its sensitivity to socioeconomic factors; and not least, its clinical value in distinguishing between bilingual or monolingual children with and without language impairment.

The first paper presenting CLT results (Potgieter & Southwood, 2016) was published in Clinical Linguistics & Phonetics before this special issue came out, although initially it was intended to be included here as well. In this paper, Potgieter and Southwood used three language versions of CLT (Afrikaans, South African English and isi-Xhosa ones) with low-socioeconomic status (low-SES) 4-year-old children being either trilingual or monolingual. They found that the vocabulary size in the dominant language of trilinguals (isi-Xhosa) was comparable to that of monolinguals, but in the two other languages (South African English and Afrikaans) trilinguals were outperformed by monolinguals. This effect may be explained by the wide differences in the cumulative exposure to each of the languages. Importantly, when trilinguals were tested in their three languages (during three sessions, taking place at 1 week intervals each), researchers did not observe any adverse effects of repeated testing. Thus, they assume that CLT may be potentially used for trilinguals as well as monolinguals or bilinguals. Furthermore, Potgieter and Southwood compared the results of CLT-Afrikaans in monolingual children from different socioeconomic backgrounds. They found that overall, low-SES children scored lower than mid-SES children. However, looking at nouns and verbs separately, the low-SES children scored significantly lower than the mid-SES children on the noun measures, but not on the verb measures. This indicates that the differences between these children may be greater in terms of material possessions (denoted by nouns) than in actions (denoted by verbs). The authors conclude that at least for verbs, the CLT-Afrikaans is suitable for use across SES boundaries (unlike commercially available assessment tools for Afrikaans).

The first paper in the present issue by Ewa Haman et al. – Noun and verb knowledge in monolingual preschool children across 17 languages: Data from Cross-linguistic Lexical Tasks (LITMUS-CLT) – constitutes a kind of background for the other papers in the special issue. Although CLT was intended primarily for bilingual populations, this paper compares the results of monolingual children (aged 3–6 years) acquiring 17 different languages. It provides evidence on the cross-linguistic comparability of the CLT in terms of stable effects of word class (nouns proved to be easier than verbs) and language mode (comprehension proved to be easier than production). Additionally, it shows participants’ CLT accuracy scores to be positively correlated with their age (indicating vocabulary size growth with age). These effects are well known and largely not disputed in the literature (Clark, 2016). When obtained with the use of a new tool, they indicate the tool validity. Furthermore, significant negative correlations of CLT items’ accuracy with its AoA (the earlier AoA, the higher item accuracy) indicate CLT validity as well. This paper also includes a discussion of the issue to what extent the differences or similarities observed across languages reflect real variability across languages in the timing and scope of developmental processes.

The second paper by Daniela Gatt, Donna Attard, Magdalena Łuniewska and Ewa Haman – The effects of bilingual status on lexical comprehension and production in Maltese five-year-old children: A LITMUS-CLT study – presents CLT results for bilingual Maltese-English 5-year-old children. It describes the very specific societal status of bilingualism in Malta where both languages are considered to be majority languages. Nonetheless, it identifies different patterns of language dominance in the linguistic environment of children: from a Maltese dominant, via a Maltese-English balanced pattern to an English dominant one. These patterns are reflected in direct measurements of children’s vocabulary size with the use of Maltese and British English versions of the CLT. The paper also analyses language mixing as a common strategy of language use in Malta which is also reflected in the verbal production of the participants. Overall, the results underline the importance of customising bilingual norms to reflect the varying degrees of bilingual exposure for children as well as the varying practice of language mixing in child-directed speech and children’s own speech.

The third paper by Christel Khoury Aouad Saliby, Christophe dos Santos, Edith Kouba Hreich and Camille Messarra – Assessing Lebanese bilingual children: The use of cross-linguistic lexical tasks in Lebanese Arabic – reports a study comparing bilingual children with TD and bilingual children with SLI tested on a variety of language measures. It describes the social context of language acquisition in a highly multilingual Lebanon and underlines the lack of standardised tools for bilingual children assessment there. In particular, tools for Lebanese Arabic are usually adaptations with no norms, while tools for other languages (French, English) have monolingual norms only. The CLT for Lebanese Arabic used in the study was the first tool designed for bilingual populations in Lebanon. In the comparison of bilingual children with TD to bilingual children with SLI, the CLT seems to clearly differentiate between the two groups. The Bi-SLI children scored significantly lower than the Bi-TD children on all expressive and receptive tasks, and in particular on the production of verbs – this indicates that naming of verbs is a predictive marker distinguishing between Bi-TD and Bi-SLI Lebanese children. CLT results were also compared to analogous data obtained from other adapted tools. In some cases, significant correlations were obtained, but only when conceptual scoring (scoring the meaning of a response regardless of the language in which it is produced) was used. The authors offer a thorough discussion of methodological issues concerning tool design in a multilingual context such as Lebanon and underline the importance of assessing both languages, and taking conceptual scoring into account.

Paper number four by Svetlana Kapalková and Daniela Slančová – The vocabulary profile of Slovak children with primary language impairment compared to typically developing Slovak children measured with LITMUS-CLT – analyses data of monolingual children. In contrast to the first paper, it focuses on one language only (Slovak), but most importantly it compares the performance of TD and SLI (called PLI in this paper) children. CLT results show that the performance of SLI children was significantly lower than that of age-matched TD children (TDAM), indicating that the CLT is a useful tool for identifying SLI children. Compared to younger TD children matched on receptive vocabulary size (TDLM), SLI children did not differ significantly on lexical production. However, the CLT results revealed different response patterns, in that the SLI children showed higher rigidity in their answers compared to the TDLM children, as well as different naming errors: TDLM children showed more semantic errors, while SLI children had more associative errors. This suggests that not only accuracy scores but also error types in naming should be considered when the performance of SLI children is addressed.

Paper number five by Pernille Hansen, Hanne Gram Simonsen, Magdalena Łuniewska and Ewa Haman – Validating the psycholinguistic aspects of LITMUS-CLT: Evidence from Polish and Norwegian – takes a closer look at some of the properties underlying the CLT construction. As mentioned above, the CLT is constructed on the basis of a set of underlying variables, among them subjective AoA and a composite CI. Based on the data from Polish-Norwegian children and their monolingual (Polish and Norwegian) peers, the paper investigates whether the underlying language-specific properties are robust predictors of children’s performance. Importantly, the study found no significant effect of language on the performance of the Polish and Norwegian monolinguals, indicating that the CLT construction has succeeded in cross-linguistic comparability for these two languages. However, while AoA was found to be a robust predictor of performance, the CI was not. The paper discusses different reasons for these findings, and different solutions related to the CLT construction.

The final paper by Carmit Altman, Tamara Goldstein and Sharon Armon-Lotem – Quantitative and qualitative differences in the lexical knowledge of monolingual and bilingual children on the LITMUS-CLT task – uses the Hebrew CLT to assess lexical performance in Russian–Hebrew bilinguals compared to their monolingual Hebrew peers. The monolingual children outperformed the bilingual children in both comprehension and production, with both groups performing better on nouns than verbs. Like the previous paper, the authors found a strong correlation between lexical performance and AoA for both monolingual and bilingual children, while the CI did not correlate with performance in neither groups. The authors suggest that this may stem from the fact that the measures used for the CI were very general and did not address language-specific features, such as the Hebrew non-concatenative morphology. Through an item-by-item error analysis, the authors further seek to identify which aspects of words’ characteristics could explain variability in the results across the groups. They suggest that items of lower frequency in the input or of higher semantic complexity may pose a bigger problem for bilinguals than monolinguals.

For many or most of the languages that are presented in this special issue, no tools for child language assessment were available so far. To the extent that such tools were available in single languages, they were not normed for bilingual populations. CLT enables lexical assessment in a comparable way across a wide range of languages, and for the first time such an assessment can be made in both languages of a bilingual child acquiring any pair of languages from a set of 25. The results described in the six papers in this issue (plus the previously published paper by Potgieter and Southwood, 2016) are consistent with the ones formerly described in the literature for a limited number of languages or language pairs. This justifies the conclusion that CLT is a tool which once normed may essentially change the diagnostic opportunities for bilingual children growing up in many different social and linguistic environments.

Acknowledgements

We are grateful to the editors of Clinical Linguistics & Phonetics – Nicole Müller and Martin Ball –for their support, advice and encouragement to produce this special issue. In addition, the result depends crucially on the work of many renowned and knowledgeable reviewers, who have spent considerable time and effort to make the contributions in this special issue readable. We wish to thank them all (listed in alphabetical order): Paul Fletcher, Carol Scheffner Hammer, Tina Hickey, Gisela Håkansson, Marta Korytkowska, Aviva Lerman, Bracha Nir, Loraine Obler, Lourdes Ortega, Ciara O’Toole, Elizabeth Peña, Gloria Ramirez, Dorit Ravid, Ludovica Serratrice and Janne von Koss Torkildsen. We also thank Elisabeth Holm for thorough editing of the articles.

Funding

This special issue would not be possible without support from various sources: The European Cooperation in Science and Technology (COST) through its funding of COST Action IS0804; The Research Council of Norway through its Centres of Excellence funding scheme (project number 223265) to MultiLing Centre for Multilingualism in Society across the Lifespan; Bi-SLI-PL project (grant no. 809/N-COST/2010/0 from the National Science Centre, Poland awarded to the Faculty of Psychology, University of Warsaw in collaboration with the Institute of Psychology, Jagiellonian University); International Visegrad Fund standard grant (no. 21420015, awarded to the Faculty of Psychology, University of Warsaw); University of Warsaw Foundation (travel grant awarded to Ewa Haman, 2014).

Additional information

Funding

This special issue would not be possible without support from various sources: The European Cooperation in Science and Technology (COST) through its funding of COST Action IS0804; The Research Council of Norway through its Centres of Excellence funding scheme (project number 223265) to MultiLing Centre for Multilingualism in Society across the Lifespan; Bi-SLI-PL project (grant no. 809/N-COST/2010/0 from the National Science Centre, Poland awarded to the Faculty of Psychology, University of Warsaw in collaboration with the Institute of Psychology, Jagiellonian University); International Visegrad Fund standard grant (no. 21420015, awarded to the Faculty of Psychology, University of Warsaw); University of Warsaw Foundation (travel grant awarded to Ewa Haman, 2014).