General Language Proficiency Revisited: Current and Future Issues

Abstract

This article explores a number of key issues that emerged during the panel discussion that followed the General Language Proficiency Symposium at the Language Testing Forum (LTF) 2010, celebrating the 30th anniversary of the LTF. The key issues that emerged during the discussion should be of interest to a wider audience, as they express current issues and concerns in the testing community in light of 30 years of research and development. These issues were whether language proficiency can be seen as unitary or divisible; the role and use of the Common European Framework (CEF) proficiency scales and levels when it comes to reporting test scores; the issue of test equivalence between high-stakes tests, also in relation to the CEF; and a general demand for developing assessment literacy among test users and stakeholders. Based on a summary of the LTF discussion, the article provides a state-of-the art review of relevant research that addresses the aforementioned listed key issues, followed by a future agenda for researching the reviewed areas.

Notes

¹ The discussion was recorded and summarised; the summary was drawn on in writing the paper.

² The particular attention during the panel discussion on issues related to the CEFR may add to the focus of this article on research in contexts where the CEFR has played a dominant role (i.e., Europe and North America). Although the widening spread of language testing practices across the world was not the focus of discussion at the LTF and hence is not discussed here, this does not imply that research and test development in other parts of the world were not recognized by the author.

³ Oller had applied factor analysis incorrectly thus drawing the wrong conclusions (Bachman, as cited in Chen, 2011).

⁴ Of interest, there are some tests where aspects of the BLC such as the speed of processing or linguistics elements are operationalized in the speaking part (e.g. Pearson PTE Academic: http://pearsonpte.com/pteacademic/; Versant: http://www.versanttest.co.uk/).

⁵ One could conceptualise a third dimension, that is, the yardstick or benchmark against which any language proficiency is assessed. The issue of what standardised or codified language could be used as such a yardstick would be worth further exploration; however, it was not discussed at the LTF and would go beyond the scope of this article. (I am grateful to an anonymous reviewer for this suggestion.)

⁶ One could argue for a possible interpretation that the CEFR is a band scheme that does not in and of itself make a claim about dimensionality. However, the background of the CEFR scale system (e.g., the model of communicative competence by Canale & Swain, 1980, and others upon which the scale system is based) and the hierarchical nature of the CEFR scale system strongly imply a multifaceted concept of proficiency (see also Hulstijn et al., 2010). The CEFR scale system offers different levels of abstraction, from the very general overall scale, to overall scales per skill, down to scales describing specific subaspects of different skills and linguistic components.

⁷ The summary of the discussion of language tests for immigration purposes is available at http://www2.warwick.ac.uk/fac/soc/al/research/conferences/ltf2011/language_testing_forum_2011_panel_discussion.pdf.

⁸ Although the CEFR scales have been empirically calibrated using teacher judgements (North, 2002), this does not amount to a validation of the scales for specific purposes, such as assessor-oriented or constructor-oriented purposes. Similarly, the statements in CEFR Chapter 9 (Council of Europe, 2001, pp. 180ff) that the proficiency scales might be useful for such purposes would need to be backed up by empirical validation.

⁹ An example for a non-European context can be found in Hong Kong, where the Workplace English Campaign was set up in 1999, supported by the Standing Committee on Language Education and Research; one of its aims was to “benchmark English standards for different occupational needs” (Qian, 2008, p. 102). Workplace English Campaign has developed five proficiency levels, which are claimed to be aligned to the ALTE and CEFR levels, and Workplace English Campaign recognises a number of international English language tests for employer information and to identify training needs. Alignment tables can be accessed at http://www.english.gov.hk/eng/html/wec_hkweb_alignment_table.htm. Knight (2001) reports a case study on the benchmarking.

¹⁰ The Dutch Grid is a classification system to characterise tests of listening and reading comprehension; the classification categories and options are derived from the CEFR. The Grid aims to facilitate test specification in order to relate the tests to the CEFR. The Grid is accessible online at http://www.lancs.ac.uk/fss/projects/grid.