When the Test Developer Does Not Speak the Target Language: The Use of Language Informants in the Test Development Process

ABSTRACT

It is not unusual for tests in less-commonly taught languages (LCTLs) to be developed by an experienced item writer with no proficiency in the language being tested, in collaboration with a language informant who is a speaker of the target language, but lacks language assessment expertise. How this approach to item writing works in practice, and what factors play a role in it, is largely unrecorded, as are item-writing processes and practices in language assessment in general. Through a case study approach, this study sought to gain insights into test development practices in cases when essential item writer traits are spread across different people. Seven in-depth interviews with language assessment specialists and language informants involved in LCTL reading test development revealed a number of specific characteristics, and also challenges, to test developer recruitment and test development in this context. Findings indicate that this inherently collaborative approach brings with it a sophisticated system of “checks and balances” that may benefit item writing in some respects.

Acknowledgments

The authors would like to thank the participants, as well as the colleagues who provided feedback on drafts of this manuscript. This work was done in partial fulfillment of the first author’s Master’s Degree in Language Testing.

Notes

¹ The definition provided here is that of the National Council of Less Commonly Taught Languages. We appreciate, however, that the taught status of languages may vary in time and region, and thus the list may not be fixed.

² The term “NNS” may in fact not be a good fit, in the sense that the person may simply not speak (or know) the language at all.

³ Because of the nature of our participants’ assignments (see below), the present study specifically covers item development and does not report on the broader test development and quality assurance processes in place at the company and for the reading tests as a whole.