Dimensions of test performance in English as a foreign language in different European settings: a two-level confirmatory factor analytical approach

Abstract

The present study explored data from a survey of students’ performance in English at the end of compulsory school in 6 European countries. The aim was to gain deeper knowledge of the internal structure of the test and to discuss similarities and differences between the different settings regarding patterns in language proficiency. The analyses, conducted by a factor analytic approach and 2-level structural equation modelling (SEM) techniques, indicated an overall English achievement factor at both student and school levels. Furthermore, an effect of format differences at the student level, constituted by a factor related to tasks demanding constructed response, was found. Three correlated modality factors related to listening, linguistic, and reading skills were identified but not further elaborated on as they improved the model only modestly. Considerable differences in between-school variation were found in the different settings. The results are discussed in relation to the instrument, student achievements, and the state of English in the participating countries.

Keywords:

• patterns of English proficiency
• European context
• two-level SEM techniques
• internal structure of test
• similarities and differences
• dimensionality

Notes on contributors

Gudrun Erickson is Professor of Education, focusing on language and assessment, at the Department of Education and Special Education, University of Gothenburg, Sweden. She has long experience of national and international projects on language assessment and is project leader for the development of national tests and additional assessment materials of foreign languages in Sweden. Her research focuses on collaboration and validation in large-scale test development, in particular on contributions from test-takers.

Lisbeth Åberg-Bengtsson is a Senior Professor of Education at the Faculty of Librarianship, Information, Education and IT, University of Borås, Sweden. One of her main research interests concerns dimensionality of large-scale tests. In another, she focuses on students’ comprehension of graphical representations such as graphs, diagrams, and statistical maps, as well as of illustrations in the form of explanatory pictures and models in educational materials.

Jan-Eric Gustafsson has since 1986 been Professor of Education at the University of Gothenburg. His substantively oriented research focuses on individual differences in cognitive abilities, and on determinants of educational outcomes at individual and system levels. His methodologically oriented research focuses on conceptual and technical issues of measurement within classical and modern test theory, and on the application of latent variable models to both non-nested and nested data.