Survey to Investigate Expectations of Achievement in Written English on English Language Degree Programmes in Europe

Abstract

From June to November 2003 an exploratory survey was made of the assessment of written English in a small sample of English language degrees in Europe. All programmes had similar components, the study of English language and literature, cultural or area studies, and English language development. Some involved the study of another language. Theoretical perspectives on test validity informed the construction of a questionnaire to collect quantitative and qualitative data from the programmes in the sample in order to investigate expectations about attainment of proficiency in written English. Responses were obtained from 32 English language degree programmes in 14 European countries: Austria, Belgium, Denmark, France, Germany, Italy, Latvia, the Netherlands, Poland, Portugal, Romania, Spain, Switzerland, and the United Kingdom. The sample was not large enough to be representative of such degrees throughout Europe, but this exploratory study revealed a wide variation between countries, within the same country and, in some cases, within the same department with regard to what was measured, the contexts sampled, and the assessment instrument adopted and measures taken to ensure the reliability of the assessment. The study revealed variable evidence to support the outcomes of the assessment across the sample, leading to the conclusion that there was wide variation in what is attained in proficiency in written English on these English language degree programmes, which is indicative of similar degrees across Europe. The framework used for data collection yielded themes, practices, and concerns that would inform the development of common guidelines for assessment.

ACKNOWLEDGMENT

I thank all those who responded to the questionnaire survey, also Bernard Hrusa-Marlow, Barry O'Sullivan, and anonymous reviewers for their careful reading and comments on earlier drafts of this article.

Notes

¹A reference guide for teachers, learners, and assessors that includes descriptions of language ability at six levels: A1, A2, B1, B2, C1, C2 (A1 = basic user and C2 = proficient user).

²The questionnaire was developed following interviews with five informants in different European countries about their assessment practices and piloted with five respondents. Please contact the author to obtain this questionnaire.

³Initial contact was made with European colleagues of members of the modern languages department in the author's university, the British Council in each European country, and members of relevant international and European organisations (the International Association of Teachers of English as a Foreign Language, the European Language Council, the European Centre for Modern Languages), and university Web sites were investigated in countries in which the languages are spoken by the author (France and Italy).

⁴The number of responses for each country is given in brackets.

⁵English language majors for nonnative speakers of English.

⁶Australia, Canada, Hong Kong, Japan, New Zealand, and Thailand.

⁷The questionnaire had originally been designed to elicit three categories of response (must be included, may be included, not included). However, the first two categories were collapsed into one for the final analysis because the distinction complicated the analysis and did not make a significant contribution to the results with regard to the research question.

⁸This reflects the level of variation in assessment practices identified by Cummings (2001, p. 222), leading him to conclude that “it is difficult to consider the goals or processes for learning ESL/EFL writing to be similar or even comparable across these courses.” Student could be achieving something different according to the preferences and values of the individual instructor.

⁹See Figueras, North, Takala, Verhelst, and Van Avermaet (2005) for a critical evaluation of the process of linking examinations to the CEFR.

¹⁰This study was undertaken in non-European as well as European countries.

Council of Europe. (2003). Relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment(Manual: Preliminary pilot version. DGIV/EDU/LANG 2003, 5). Strasbourg, France: Language Policy Division.

Council of Europe. (2005). Reference supplement to the preliminary version of the manual for relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment(DGIV/EDU/LANG 2005, 13). Strasbourg, France: Language Policy Division.