Constructive alignment in foreign language curricula: an exploration of teaching and assessment practices in Dutch secondary education

ABSTRACT

While Communicative Language Teaching (CLT) is recognised as an effective approach worldwide, its implementation in foreign language (FL) classrooms remains difficult. Earlier studies have identified factors impeding CLT implementation, such as a lack of communicative lesson materials or teachers' more traditional views on language learning.

In the Netherlands, CLT goals have been formulated at the national level, but are not always reflected in daily FL teaching and assessment practice. As constructive alignment between learning goals, classroom activities and assessments is a precondition for effective teaching, it is important to gain a deeper understanding of the degree of alignment in Dutch FL curricula and the factors influencing it. The current study therefore aims to take a systematic inventory of classroom practices regarding the translation of national CLT goals into learning activities and assessments. Findings revealed that teaching activities and classroom assessments predominantly focused on grammar knowledge and vocabulary out of context and, to a lesser extent, on reading skills. External factors, such as teaching and testing materials available, and conceptual factors, such as teachers' conceptions of language learning, were identified to contribute to the observed lack of alignment. Assessments in particular seem to exert a negative washback effect on CLT implementation.

KEYWORDS:

• Communicative language teaching
• constructive alignment
• assessment
• washback
• Foreign Language Education

Introduction

Communicative language teaching (CLT) is nowadays widely accepted and recognised as a productive pedagogical approach in foreign language (FL) education (Richards 2006). CLT aims to promote the development of communicative language skills in realistic situations (Littlewood and Yu 2011). The approach emphasises the learning of language through meaningful spoken and written interactions, with an appropriate degree of spontaneous production, rather than through memorisation and the learning of grammatical rules out of context (i.e. focus on forms, Long 1991). These formal aspects are certainly present in the communicative approach, but they are to be addressed within a meaningful context and always in combination with a focus on the development of communicative skills (focus-on-form; Long 1991). Table 1 above summarises the main characteristics of the communicative approach.

Table 1. Main characteristics of CLT (based on Larsen-Freeman and Anderson 2011).

CLT principles
Use of the target language in meaningful spoken and written interactions: the context in which a communicative activity takes place should be connected to students’ experiences, interests, and/or imagination.
Use of a ‘functional grammar’ approach. Attention to form aspects of language (grammar, vocabulary, pronunciation), embedded in meaningful, communicative activities.
Use of functional language activities integrating different language skills, with an appropriate degree of free production, that prepare learners for communicating in real-life situations.
Focus on the development of strategies to help students perform language tasks and/or to solve problems in communication.
Use of authentic materials instead of isolated representative sentences out of context.
Use of appropriate scaffolding that enables students to progress in their language development (with regard to form and/or content).

Although language teachers today recognise the need to teach languages in a communicative way, they do not always succeed in introducing this approach into their daily teaching practice (Little 2007, Kissau et al. 2015). This situation seems also to occur in the Netherlands and particularly in lower forms (Westhoff 2007, Moonen et al. 2013), which is the context in which the current study took place.

FL teaching in the Dutch educational system

The Dutch educational system is partly decentralised. The role of the government is limited to the determination of the subjects to be taught, the setting of attainment targets and the conception of final examinations. FL teaching in the Netherlands is CLT-oriented. Official curriculum documentation (College voor Toetsing en Examens 2019, Meijer and Fasoglio 2007) as well as standards for language teacher training (cf. https://10voordeleraar.nl/publicaties), mention the communicative approach indirectly by referring to the CEFR and CEFR-levels: students should be able to use different language skills in authentic situations in order to communicate effectively in a foreign language at a specific CEFR level depending on the class they are in and the language they are learning. In a recent proposal for updating the FL curriculum, elaborated by teachers, school leaders and experts, CLT principles are also mentioned only indirectly, by referring to the CEFR. The proposal does stress the importance, however, of paying more attention to the development of productive skills and social interaction (curriculum.nu 2019).

Foreign languages are compulsory subjects within the first three years of Dutch secondary education (for pre-vocational and per-academic streams). The present study focuses on contexts where FLs are taught as separate subjects. Over a 100 schools in the Netherlands, however, offer bilingual programmes in which English is used as the medium of instruction for half of the subjects, which are taught according to the principles of content and language integrated learning or CLIL (Michel et al. 2021). Although CLIL provides an interesting example of communicative content-based teaching for foreign languages, this is not the focus of this study, as the majority of schools in the Netherlands teach FLs as separate subjects.

In upper form, students can choose to (dis)continue studying other foreign languages next to English, which remains a compulsory subject. The final examination consists of two equally weighted parts, a national reading examination developed by the Dutch National Testing Institute (CITO), and school examinations developed and assessed by the schools themselves, covering all other skills. In lower form secondary education, all test formats and contents are chosen by (language) departments of individual schools. The government inspects schools on the achievement of the attainment targets but does not prescribe pedagogical approaches or textbooks and only provides suggestions for the number of teaching hours per subject. Therefore, schools, many of which are semi-private, can choose their own teaching approach. Furthermore, different FL departments within these schools will make their own plan on how to integrate attainment targets. Finally, at the classroom level individual teachers will also make their own choices on how to translate the national learning goals into daily teaching and assessment classroom practices. As such, interpretations of the general targets can differ from one level to another, and from one school to another (Scheerens 2017).

Although learning objectives at the national level are formulated from a communicative perspective, FL teachers, particularly in lower form, tend to focus more on knowledge of grammar rules, vocabulary and chunks taught out of context in teaching and assessment practices than on the development of language skills (Fasoglio et al. 2015, West and Verspoor 2016). In upper form education teachers focus on reading comprehension, which is the only skill tested in nationwide exams and which constitutes 50% of the final grade (Fasoglio et al. 2015). As a result, students who perform well during their school career and up to their final exams may still have difficulties or even be unable to communicate in the foreign languages they have learned (Schnabel et al. 2016). It is important to understand why the introduction of the communicative approach appears to be difficult in FL teaching in the Netherlands.

Implementation of CLT: influencing factors

In other educational contexts, a number of studies (Chang 2011, Coskun 2011, Nishino 2011, Kissau et al. 2015, Ahmed 2016, Sarab Anani et al. 2016, Zhu and Shu 2017) have sought to identify factors explaining the difficulty in integrating a communicative approach into daily teaching practice. Although such factors are always context dependent to a certain extent, two broad categories of similar influential factors emerged from the different studies: external factors related to organisational issues, policies and (lack of) schools support and conceptual factors, related to teachers’ training and experiences shaping their conception of CLT. Coskun's (2011) qualitative study, for example, conducted in the Turkish context and Sarab Anani et al.’s (2016) study in the Iranian context both report that, from a teachers’ perspective, challenges in the implementation of CLT mostly relate to external factors, such as large class size, lack of time to develop communicative teaching resources and traditional grammar-based examinations. Chang (2011) furthermore interviewed eight teachers from two universities in southern Taiwan on their perceptions and experiences regarding the integration of CLT into their curriculum in order to determine which factors promote or hinder this integration. She identified four groups of factors explaining the difficulty of introducing CLT, two types of external factors (i.e. educational system factors and CLT support), and two types of conceptual factors (i.e. teacher factors as well as student factors). The study shows, for example, that schools and national educational systems do not facilitate CLT integration as they place students in overcrowded classes with insufficient contact hours and as they impose exams that are not communicative (external factors). Her study also shows that teachers are not sufficiently trained to apply a communicative approach in practice and that students need to grow accustomed to this way of teaching (conceptual factors). In another study, Wang and Cheng (2009) explore the problems faced by language teachers in China in the transition to CLT. The necessary changes that Wang and Cheng identify are also related to both external (materials available, introduction of other types of activities and assessment) as well as conceptual factors (traditional view on language learning). According to this study, teachers need to change the way they think about FL learning and teaching. They need to move from a conception based on the acquisition of knowledge to a conception based on the development of language skills. Wang and Cheng also stress the importance of teachers having a clear understanding of what CLT is and knowing how to apply it in practice. Indeed, a large number of studies point out the gap between teachers’ beliefs about CLT on the one hand and the implementation of this approach in their teaching practice on the other. They indicate that many teachers who think they are using a communicative approach do not have a clear definition of it (Richards 2006, Burke 2011). This may be related to the fact that – despite the attention given to the CLT approach in the literature – language teachers may not have received sufficient and/or appropriate training in their teacher education programmes on the effective application of such an approach (Savignon 2017, Graus and Coppen 2016). Besides, student teachers’ own conceptions of language teaching also play a role in the way they interpret and later introduce CLT principles in their classrooms. This conception is shaped both by the way in which the pre-service teachers themselves have been taught foreign languages in secondary education, but also by their tutors in the schools during their internships, their colleagues and the textbooks they use (Graus and Coppen 2017). All these external and conceptual factors affect the way CLT objectives are translated into lesson goals, learning activities and assessments. They can lead to a lack of constructive alignment which may in turn explain the difficulty of introducing CLT in practice.

Constructive alignment: the crucial role of assessments

Constructive alignment (CA), an educational concept developed by Biggs (1996), highlights the importance of aligning learning objectives with learning activities and assessment practices in a curriculum to enhance students’ learning and achievement. Biggs and Tang (2007) defined four characteristics that a curriculum should meet to be constructively aligned:

• − Learning objectives must be clearly defined and specified in learning activities.

• − The learning environment should provide activities that enable students to achieve the learning goals.

• − The assessment should match and cover the learning objectives.

• − Student performance should be assessed using rubrics that bring together criteria related to the learning objectives.

In the development of an aligned curriculum, Biggs emphasises the importance of identifying the goals that students need to achieve in assessments before organising teaching and learning activities. Assessment practices indeed play a crucial role in constructive alignment, as tests are known to influence both teaching and learning, known as ‘washback’ (Alderson and Wall 1993, Green 2007). This washback can be positive when tests are aligned with learning goals, but can be negative when alignment is lacking.

If students have to learn to communicate in a foreign language (objective), assessments and learning activities are aligned when they are communicative in nature. In contrast to traditional test formats that test FL skills indirectly (e.g. multiple choice questions in which knowledge of specific vocabulary or grammar rules are tested), communicative tests aim to provide a more direct evaluation of students’ general FL ability. As such, communicative assessment tasks should be sufficiently authentic and diverse to match the variety and complexity of real-life communication situations (Morrow 2018).

In (partly) decentralised educational systems such as the Dutch one, schools and teachers have a lot of control over assessments which would, in principle, grant them with ample opportunity to generate positive washback (Hakim 2018). However, as Fasoglio et al. (2015) and West and Verspoor (2016) have indicated, FL tests developed by individual schools are not always communicative in nature, and there is a lot of variation in test formats and contents among schools and even language departments within the same school. This situation could explain the difficulty of implementing communicative teaching; if tests are not sufficiently communicative, learning activities will not be sufficiently communicative either, because teachers want to prepare their students to the tests (i.e. negative washback).

Current study

There are currently no studies available in the Dutch context that systematically investigate the coherence between the different elements of FL curricula, nor studies that seek to provide insight into the reasons and arguments underlying teachers’ decisions in using non- or less communicative activities and assessments, particularly in lower form of secondary education. As discussed above, both external and conceptual factors affecting teachers’ pedagogical choices can create a lack of alignment between CLT learning goals on the one hand, and non-CLT teaching and testing approaches on the other hand. It is important to identify which factors affect pedagogical choices at classroom level in a specific context in order to seek solutions to enable teachers in that context to teach and test in a communicative way, and as such allow them to ‘align’ their curriculum more effectively. The current study does so for the Dutch context, by exploring the degree of constructive alignment in Dutch FL curricula and the factors affecting teachers’ pedagogical choices regarding the translation of communicative learning goals into learning activities and assessments. As such, the research questions guiding the current study are:

RQ1: What is the degree of alignment between communicative learning goals, pedagogical approaches and assessment practices in FL curricula of lower form secondary education in the Netherlands?

RQ2: Which external and conceptual factors do teachers report to be of influence on the observed degree of alignment?

We used different measurement tools to gather both quantitative and qualitative data on the various components of the curriculum in specific schools and for different languages taught at A2 level. We selected the A2 level because it is the most common level to be attained for the different languages taught in lower form in the Netherlands.

Method

First, classroom observations (N = 31) were carried out to determine the extent to which national communicative goals were reflected in lesson goals and learning activities. Assessment programmes and test materials were then analyzed to establish their degree of alignment with the other two components of the curriculum (RQ1). A survey among students (N = 473) was conducted to complete the inventory and provide a student perspective on classroom practices regarding learning goals, learning activities and assessments. Interviews (N = 21) were then carried out to gain more in-depth insight into the factors influencing FL teachers’ pedagogical approach regarding the implementation of CLT goals in their teaching and assessment practices (RQ2).

Participants

Teachers

A group of FL teachers from different schools in the Netherlands was recruited for classroom observations, supply of test materials and interviews. They were selected on the basis of the following criteria: they should be certified, in-service and teach at least one class at A2 level of the CEFR. Thirty-two teachers from fourteen different schools and eighteen language departments were selected. The selected teachers had varied years of experience ranging from one to 25 years. The four most widely taught languages in Dutch secondary schools were represented in the participant group: two English teachers, seventeen French teachers, nine German teachers and three Spanish teachers. This apportionment does not reflect the distribution of different FLs in the Dutch educational system (i.e. English being the most widely taught FL in the Netherlands), but can be explained by the fact that this is a convenience sample from the network of the first author, a French teacher educator. Teachers from ten participating language departments (three German departments, three French, two Spanish and two English) supplied test materials.

Students

Students from A2-level classes of the participating teachers anonymously filled in a digital survey halfway through the school year. They were all lower form students aged 12–15. They filled in the survey about the language lessons taught by one of the participating teachers: English (N = 17 students), French (N = 308), German (N = 76) and Spanish (N = 72).

Instruments, data collection and data analysis

Classroom observations

All but one of the selected teachers (N = 31) were observed giving a lesson at level A2. Video recordings were made of the lessons observed. The teachers did not receive any prior instructions, except for to teach as they would usually do. They were observed using an observation tool developed by the Foreign Language Education Research Group at the Center of Expertise: Learning and Innovation of Utrecht University of Applied Sciences (HU). This observation tool aims to map the extent to which the pedagogical approach of FL teachers adheres to CLT principles. The instrument focuses on observable teacher behavior and communicative teaching activities for each of the language skills. Each category, focusing on a specific language skill, includes similar items for CLT activities based on the criteria that communicative activities should meet as described in the theoretical framework: meaningfulness, creativity (appropriate degree of free production or relevant comprehension activities), authenticity, use of scaffolding, and development of strategies. The format of the instrument is based on ICALT (International Comparative Analysis of Learning and teaching) (RUG 2019, Maulana et al. 2017). As in ICALT, each category could be scored on a four-point Likert scale representing the extent to which FL teachers implement the principles of communicative language teaching, ranging from 1 (predominantly weak) to 4 (predominantly strong). Additional field notes were taken during the observations. The instrument also included an open question on lesson objectives. Teachers were asked to confirm, specify or indicate after the classroom observation what the lesson goals were.

All observations were performed by the same observer (i.e. the first author). To guarantee inter-rater reliability five observations were conducted on-site by a second observer, namely by members of the first author’s research group, who co-developed and validated the observation tool. To check intra-rater reliability five observations were conducted a second time, a year later by the first author on the basis of the video recordings. Scores were then compared to the observations made on-site. For both inter- and intra-rater reliability Cohen's κ was run. We found that there was substantial agreement between the ratings (inter-rater reliability: κ = .617 (95% CI, .499 to .734), p < .0001; intra-rater reliability: κ = .616 (95% CI, .498 to .698, p < .0001).

The scores from the Likert scale were used to generate quantitative data that we analyzed descriptively for each observation category (i.e. meaningfulness, degree of free production, etc.). The additional field notes were used to interpret and illustrate the scores.

Open-ended responses regarding the lesson objectives were coded into three categories: lesson goals related to the development of FL skills, lesson goals related to the development of knowledge of grammar rules outside authentic situations (focus on forms) and (inter)cultural lesson goals.

Analysis of assessment programmes and test materials

In secondary schools in the Netherlands, assessment programmes (set of tests fixed over a school year in a specific grade) are developed per grade based on nationally formulated learning objectives. The assessment programmes for lower secondary education (A2 level) from a subsample of ten of the eighteen language departments participating in the study were compiled and analyzed in terms of the number of tests assigned per year, test formats, test content in relation to language skills or to language knowledge, and weighting assigned to each test. In each test program the skills or knowledge tested and the weighting of each test in the overall yearly average were indicated. We first inventoried which content was planned to be tested within each test. We identified four types of test content: (1) tests addressing one specific language skill, (2) tests addressing two or more language skills, (3) tests combining the assessment of one or more language skills with the application of grammar rules or the translation of words out of context and (4) tests addressing only the application of grammar rules or the translation of words out of context. We then inventoried the frequency and weighting of the different tests administered per school year, per language department.

In addition to the assessment programmes, the participating teachers submitted 43 representative language tests used in their schools at level A2. We recorded 88 tasks within the various tests and for 8 tests we also received their accompanying rubrics. We classified the different tasks according to the knowledge or skills they aimed to assess and the learning goals targeted. Tests assessing language skills and the accompanying rubrics were analyzed using a checklist of criteria that communicative tests should meet, as described in the theoretical framework. This checklist was adapted from the list of criteria developed by the National Institute for Curriculum Development in the Netherlands (SLO) and supplemented by the criteria defined in theory on communicative language teaching and testing (Brown 2005, Bachman 2010, Green 2013, Morrow 2018). The checklist was divided into three categories: construct validity, content validity and evaluation criteria (rubrics). Criteria related to CLT (i.e. meaningfulness, authenticity, unpredictability, appropriate degree of free production and skills integration) could be marked as present or absent.

Student survey

To add a student perspective to the inventory of classroom practices, participating teachers had their students complete an online survey with multiple-choice items. The questions focused on students’ perceptions of learning goals, the proportion of type of knowledge and skills covered during the lessons, and the degree to which this knowledge and these skills were tested. We first conducted a descriptive analysis of the results and examined the frequency of student responses to the various questions. We then compared the results with those obtained from the other instruments.

Teacher interviews

Twenty-one teacher interviews were conducted. The interviews were semi-structured and included 11 open-ended questions. The interviews averaged 20 minutes in length and were recorded digitally. The audio recordings were transcribed verbatim as Word documents and pseudonymized (T1 to T21) before being deleted in accordance with the approval obtained by the Faculty Ethics Assessment Committee Humanities of Utrecht University. Teachers were interviewed individually, following their observed lessons. The main objective was to better understand the basis on which teachers’ pedagogical choices regarding lesson goals, learning activities and assessments were made (RQ2). The first two interview questions concerned the choice of lesson goals in relation to CLT. The next three questions focused on the choice of learning activities. Finally, the last six questions concerned the choice of assessments.

Once collected and transcribed, the interview data were first analyzed interpretatively and thematically according to the steps specific to qualitative research analysis (Corbin and Strauss 2008): categorising the data into themes, identifying noticeable patterns, and synthesising the categorised data into general findings. We categorised the data into the following two themes, which were identified a priori based on our literature review: conceptual factors and external factors influencing teachers’ pedagogical choices. Under these themes, we then coded the data in an inductive way based on the respondents’ answers (Corbin and Strauss 1990). In this case, the codes were short sequences of words that described what kind of conceptual and/or external factors participants mentioned in their answers to the interview questions. Subsequently, codes were clustered under overarching categories (e.g. lack of time; school requirements). All themes, categories, and examples of codes within each category were then brought together in a coding scheme. The resulting coding scheme was used by the first author to analyze each of the participants’ transcripts (N = 21).

To ensure reliability, a member of the research group – English teacher, teacher trainer and PhD student – selected at random two interview transcripts and used the coding scheme to code them. Cohen's κ was then run to determine the agreement between the two coders. We found that there was substantial agreement (κ = .632 (95% CI, .415 to .815), p < .0001). The frequency with which each of the different codes appeared in the responses given by all the teachers interviewed was analyzed to identify noticeable patterns. In order to further illustrate, clarify and explain these quantitative data, quotations were selected and added to the results section.

Results

Degree of alignment (RQ1)

The results relating to the first research question concerning the degree of alignment of Dutch FL curricula are reported below under the following curriculum components: lesson goals, learning activities and assessments in relation to CLT learning objectives.

Lesson goals and learning activities

Table 2 shows the type of lesson goals identified in the observed lessons. Almost half of the lesson goals formulated relate to the development of language skills, while the other half focus on forms (e.g. grammar rules). Only a small proportion of the observed lessons included content related objectives, such as culture and literature.

Table 2. Lesson goals in observed lessons (N = 31).

Lesson goals	Frequency	Percentage
Developing FL skills	14	45%
Developing knowledge of grammar rules and vocabulary outside authentic situations (focus on forms)	13	42%
Developing content knowledge (e.g. culture, literature)	4	13%

In everyday teaching practice, however, the focus on grammar rules and words seems to be even more accentuated. When asked directly in the interview following the classroom observation, 35% of the 21 teachers honestly admitted that the communicative focus of the lesson observed was not representative of the way they usually teach. One teacher said for instance: ‘This lesson is not representative of the way I teach. I have already finished the regular program and the tests. My activities are usually mostly aimed at preparing the students for the tests' (T13).

Table 3 shows the extent to which the learning activities used during the observed lessons adhered to the criteria of CLT and, as such, could be expected to contribute to the development of communicative language skills. We notice that although most activities were meaningful (mostly more strong than weak), they did not ask for an appropriate degree of free production or functional comprehension in accordance with the CEFR level (mostly more weak than strong) and were not authentic enough (predominantly weak). Students were frequently asked, for instance, to answer multiple choice questions about a text or an audio document or they had to complete strongly guided writing or speaking tasks that were not authentic, such as the translation of isolated sentences out of context. Results furthermore show that most activities, especially those focusing on listening and speaking skills, were not accompanied by sufficient scaffolding and did not focus enough on the development of communicative strategies. For instance, students had to perform a speaking or listening task without any prior guidance and/or strategies. As a result (and as reported in the observer’s field notes), students either failed to complete the tasks or used non-communicative strategies, such as writing and translating a dialogue, and then tried to read it with a comprehensible pronunciation.

Table 3. Adherence to CLT criteria of learning activities in observed lessons focusing on the development of language skills (N = 31).

Language skills		Meaningful activities	Appropriate degree of free production/Use of functional comprehension activities	Authenticity	Appropriate scaffolding	Development of communicative strategies
	N	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
Speaking	14	1.93 (.73)	.93 (.92)	.43 (.65)	.79(.90)	.64 (.74)
Writing	9	2.11 (1.06)	1.56 (1.24)	1.11 (1.06)	1.11 (1.17)	1.33 (1.12)
Reading	13	1.69 (.75)	1.23 (1.02)	.69 (.75)	1.00 (1.71)	1.31 (1.03)
Listening	10	2.00 (.82)	1.60 (.97)	.60 (.70)	.30 (.49)	.30 (.49)
All language skills	46	1.94 (.84)	1.33 (1.04)	.7 (1.03)	.80 (1.19)	.90 (.84)

0 = not visible or predominantly weak; 1 = more weak than strong; 2 = more strong than weak; 3 = predominantly strong

An analysis of the students’ views on the knowledge and skills addressed in their FL lessons (Table 4) shows that their perceptions reflect our own observation that almost half of the lessons focus on grammar rules and vocabulary (cf. Table 2). Moreover, while we saw a relatively equal distribution of attention to the different language skills (i.e. listening, speaking, reading, writing) in observed proficiency promoting learning activities (cf. Table 3), according to students, activities focusing on reading skills are far more frequent in daily teaching practice than, for example, activities promoting speaking skills (Table 4).

Table 4. Knowledge and skills addressed most frequently in lessons according to students (N = 471).

Knowledge and skills	Frequency	Percentage
Grammar rules and words	209	44.5%
Reading skills	120	25.5%
Writing skills	36	14%
speaking skills	64	8%
Listening skills	34	7%
Culture	8	2%

Assessments

We analyzed the contents, frequency and weighting (i.e. percentage of the test result in the final grade) of tests present in the assessment programmes of ten language departments within the participating schools. We found a lot of variation in the test content and formats, frequency and weighting between the language departments. Almost all language departments test the knowledge of grammatical rules, vocabulary and chunks out of context alone or in combination with one or more language skills, but not in an integrated way.

Table 5 presents the average frequency and weighting per test format for the ten language departments. For the majority of the language departments tests assessing the knowledge of grammatical rules and/or vocabulary and chunks out of context constitute more than 50% of the final grade. The remaining 50% includes tests assessing language skills. Speaking skills are the least frequently assessed and have the lowest weighting in the determination of the final grade.

Table 5. Frequency and weighting of tests (N = 101) within assessment programmes of FL departments (N = 10).

Test content	Frequency	Average weighting
Knowledge of grammatical rules, vocabulary and chunks out of context.	36	36%
Knowledge of grammatical rules, vocabulary and chunks out of context combined with one or more language skills.	19	40%
Development of two or more language skills.	8	18%
Development of reading skills.	13	26%
Development of listening skills.	9	13%
Development of writing skills.	8	15%
Development of speaking skills.	8	11.5%

Table 6 presents the analysis of tasks in representative FL tests. Among the 43 tests made available to us by the participating teachers, we counted 88 tasks which we divided into seven categories. Frequencies in Table 6 indicate that tasks focusing on formal aspects of the language outside a communicative context were used the most (49.3%). Within tasks assessing language skills, multiple choice reading tasks were the most frequently administrated (10.6%). Tasks assessing language skills in an open (creative) way were scarce (4.4%).

Table 6. Focus of tasks (N = 88) in representative FL tests (N = 43).

Tasks	Frequency	Percentage
Application of grammar rules or vocabulary outside a communicative context	21	18.5%
Translation of words, sentences or chunks	35	30.8%
Multiple choice reading test	12	10.6%
Strongly guided writing task	4	3.5%
(Recorded) pre-written and prepared dialogue or presentation	7	6.2%
Open (creative) task	5	4.4%
Multiple choice listening test	4	3.5%

Table 7 presents the frequency with which communicative aspects or criteria were encountered in tasks assessing language skills (N = 32) within the FL tests submitted (N = 43). Table 7 shows that although most of the tasks were meaningful for the students, they were not often embedded in authentic situations, did not ask for a sufficient degree of free production in accordance with the intended CEFR level, and did not measure students’ ability to communicate spontaneously by adapting language to new situations. Furthermore, only one test assessed different language skills in an integrated way.

Table 7. Communicative aspects present in representative FL tests tasks assessing language skills (N = 32).

Communicative aspects	Frequency	Percentage
Meaningful communication	27	84.4%
Authentic situations	10	31.2%
Unpredictability	1	3.2%
Appropriate degree of free production	3	9.4%
Skills integration	1	3.2%

Eight of the 32 communicative tasks were evaluated with a rubric. As reported by the teachers, the submitted rubrics were all developed intuitively. Seven of the scales were numeric, meaning teachers could give a score per category using numbers or percentages. One scale was more descriptive, using three levels corresponding to ‘insufficient’, ‘sufficient’ and ‘good’ task performance. Criteria described per level for each category were mostly quantitative (e.g. less than 5 verbs are used correctly) or appreciative (e.g. the presentation is creative). All scales were analytic with different categories. They all included formal aspects of the language (such as grammar, vocabulary, spelling or pronunciation) and task completion. Only one scale mentioned other aspects, such as willingness to communicate (use of strategies). None of the scales included CEFR can-do statements.

Results from the test analyses coincide with students’ experiences regarding assessments. According to students, knowledge of grammar rules and words is also the most often tested and weighs the most heavily (Table 8). The results presented above show that, in general, neither the classroom activities nor the tests used in participating schools sufficiently cover the CLT learning goals as established at the national level and/or meet theoretical criteria of effective CLT implementation.

Table 8. Most frequently tested knowledge or skills according to students (N = 470).

Knowledge and skills	Frequency	Percentage
Grammar rules and words	346	74%
Reading skills	75	16%
Writing skills	34	7%
speaking skills	9	2%
Listening skills	6	1%
Culture	0	0%

Conceptual and external factors influencing FL teacher’s pedagogical choices (RQ2)

Below we present the factors the teachers in this study reported to be of influence on the determination of lesson goals, learning activities and assessments in relation to CLT learning goals. To the question on how learning goals were set for each lesson and how classroom activities were chosen, a large majority of the teachers we interviewed answered that the selection of lesson goals and learning activities was based on their textbooks, either exclusively (57%) or partially (33%). Some of the teachers in this study (24%) indicate that they often determine the lesson goals themselves and that most of the learning activities are based on the CEFR. Fewer teachers did not explicitly formulate any lesson goals (10%). A relatively large proportion of the teachers (43%) indicate that lesson goals were not directly linked to CLT. Table 9 presents the reasons frequently reported by teachers for not formulating lesson goals and select learning activities more in line with CLT.

Table 9. Factors of influence in the implementation of CLT learning goals and activities reported by FL teachers in interviews.

Factors	N ( =  21)	Examples of explanations given by teachers
Time and expertise	10	‘I find that difficult. I work full-time, so sometimes I just have to do something. Because I don't have much time to prepare properly and I think that it is a pity. So not all lesson goals fit and I'm not going to formulate lesson goals every lesson. I think that working outside the textbook is better but it requires a lot of preparation and I miss then a clear line. You develop while you are doing it and then it can get a bit messy.’ T11
		‘We have a textbook. We follow the book and we are not always happy with that. Especially because the grammar is not offered the way we want it. But if you have a textbook, you follow a certain step-by-step plan. But if I want to do it myself, it takes a lot of insight and a lot of time.’ T3
		‘We are busy with a lot of things, but what I want is to improve my lessons and have time to think about it, and that time is just not very often there’. T2
Agreements within FL departments	17	‘The problem is that the material (in this case the grammar rules) must be covered in preparation for the test as decided in the department.’ T9
		‘I try to do something ‘outside the textbook’ on a regular basis, but I have difficulty doing so because within our department it is important that we work in a consistent way. I would like to make more use of authentic texts.’ T8
		‘The problem is that the material (in this case the grammar rules) must be covered in preparation for the test as decided in the department.’ T9

For 71.5% of the teachers we interviewed, test formats and content have a significant influence on the way they shape their lessons. One teacher, for example, reports:

Test format does have an influence on my choices. When the test is very classical, like with a separate grammar section, I tend to explain grammar explicitly in class.

Some of the teachers we interviewed mentioned conceptual rather than external factors of influence on their pedagogical decisions regarding assessments, such as the idea that it is important to teach and assess words or grammar rules outside of context first in order to help students to improve language skills later on:

The emphasis, in my view, should be on grammar, because that's where almost everything depends on, so if students do that well, they will then write better. And then they will also recognize certain structures in texts, and that's why we made these choices. T21

I think it's important that students learn words, that they get a mark. Without marks, they won't actually learn them. It doesn't have much to do with the learning goals and that's a pity but it is a school system. T3

Students participate well when it comes to assignments for which they get marks. They score well on grammar and vocabulary, but less well on language skills. They think that learning a language is too difficult. T13

These examples show that the perceptions of some language teachers on how to learn a language influence their choices regarding what should be assessed. The reasons teachers in this study give us for the necessity of awarding marks to motivate students to learn grammar rules and vocabulary out of context indirectly reveal the value these teachers place on the prior acquisition of these skills in learning a language.

Discussion

The aim of this study was to draw up an inventory of FL teaching and assessment classroom practices in lower-form education in the Netherlands. More specifically, the first objective of the study was to determine their degree of alignment with the intended, CLT oriented Dutch FL curriculum. This was done by observing how the communicative learning objectives formulated at the national level were translated into lesson goals, learning activities, and assessments by individual schools and teachers. The results of the current study show a lack of alignment, especially regarding the national CLT goals on the one hand and the type of classroom assessments used on the other hand. When zooming in on learning activities first, we see that, although an important part of the lessons observed focused on the development of language skills, learning activities provided during the lessons were often very guided and did not ask for an appropriate degree of free production in line with CLT criteria (Larsen-Freeman and Anderson 2011). Moreover students reported a disproportionate distribution of attention across different skills in learning activities, with a predominant focus on reading skills. This tendency has previously been reported by the Dutch National Institute for Curriculum Development (Fasoglio et al. 2015). The results of the study support this tendency and identify the CLT criteria which were least discernible in the learning activities provided by FL teachers. We noticed that, although most activities were meaningful, they did not sufficiently include other CLT criteria, namely: authenticity, an appropriate degree of free production, appropriate scaffolding and a focus on the development of communicative strategies. When looking at the assessments, the results show that language skills were less often tested than knowledge of grammar rules and vocabulary and chunks out of context and weighed less in the overall average, whereas they are essential in real-life communication (Larsen-Freeman and Anderson 2011, Morrow 2018). In the assessment of language skills, we observed a disproportionate focus on reading skills. In addition, on the rare occasions that productive language skills were tested, they were not tested in a communicative way, but rather through heavily guided tasks. Furthermore, the rating scales used to evaluate these assessment tasks, if any, did not include CEFR can-do statements at A2 level. This can be taken as another indicator of the misalignment between testing practices on the one hand and CLT principles on the other. We can conclude that we see a lack of alignment between the communicative learning objectives formulated at the national level and the learning activities and assessments observed and reported by the FL teachers in this study, which do not (sufficiently) cover the communicative learning objectives. Results from the student survey furthermore confirm this lack of alignment between the intended communicative objectives on the one hand, and realised learning activities and the administrated tests on the other hand.

The second objective of this study was to gain more insight into the external and conceptual factors of influence on Dutch FL teachers’ pedagogical choices, to further explain the observed degree of alignment. Results show that teachers’ pedagogical choices are largely based on the textbook they use. The teachers in this study report that they lack the time and expertise to develop their own materials and that they need a textbook to guide their pedagogical choices. These external factors were also reported in other contexts (Chang 2011, Coskun 2011, Nishino 2011, Kissau et al. 2015, Ahmed 2016, Sarab Anani et al. 2016, Zhu and Shu 2017). In addition to these external factors identified, results show that teachers’ pedagogical choices also relate to their own conceptions of language learning, such as the importance of achieving knowledge of grammar rules and vocabulary prior to the development of language skills, and the importance of testing this knowledge directly in order to stimulate students to learn it. Whereas form-focused instruction may contribute to the development of language skills, it only does if this knowledge is developed within a meaningful communicative activity (Long 1991, Spada 2011). In our data, however, activities focusing on grammar rules and vocabulary and chunks were mostly provided out of context. Teachers’ conceptualizations of CLT were also reported to be of influence in the studies mentioned earlier (Chang 2011; Wang and Cheng 2009), but they may even be of greater influence in the partly decentralised Dutch educational system where teachers have a large degree of autonomy in the selection, development and implementation of teaching and testing materials.

Both our inventory of classroom practices as well as teachers’ reports on factors influencing the degree of constructive alignment in their curricula point towards a crucial role of assessments. Given the preeminent focus on grammar, vocabulary and strongly guided language use in observed assessment practices, and the washback effect tests are known to exert (Alderson and Wall 1993), it is not surprising that CLT implementation remains problematic. Moreover, the observation that reading skills are addressed relatively often in both teaching and classroom assessments in lower form FL classes might well be a washback effect of the exclusive focus on reading in the national final FL examinations in the Netherlands. The teachers themselves also pointed to test formats and content as a significant influence on the way they shaped their teaching.

Limitations

This study was conducted in the context of authentic classroom practice. This enhances ecological validity, but also implies that many factors may have had an (unintended) impact on the results.

A large number of the teachers participating in the project did so on a voluntary basis with the intention to work on their own professional development. This means that these teachers and the schools involved were aware of the purpose of the current study and were willing to develop professionally in this respect. As such, results from classroom observations and interviews may have been influenced by teachers’ commitment to the project.

Moreover, teachers were observed only once at a given moment in the school year and in a specific phase of their teaching program. It is a snapshot in time that cannot fully reflect the diversity of teaching and learning activities provided during a school year. The results need to be considered in this light. The student survey was introduced to triangulate or nuance the results from the classroom observations in this regard. However, it consisted of a multiple-choice survey filled in by students aged 12–15 during the lessons. Even if the results reflect an existing tendency due to the high number of respondents coming from different classes and schools, the reliability of the results cannot be fully guaranteed. Additional interviews could have been insightful to corroborate the survey findings.

Implications for practice

The results of this study indicate that although the large majority of the teachers in this study showed their intention to teach foreign languages by adopting a communicative approach, they may not succeed in doing so in an optimal way. This is largely due to external factors such as a lack of communicative materials available, a lack of time to develop lessons, and agreements made within FL language departments. The textbooks frequently used by FL teachers do not often include enough communicative activities and fail to put enough emphasis on such activities, by including tests that are not sufficiently communicative and mainly test knowledge of grammar and vocabulary out of context. Ideally publishers could place more emphasis on communicative activities and tests in FL textbooks. Moreover, teacher training programmes should put emphasis on the why and how of aligning an FL curriculum with the national CLT learning goals. They could explicitly train prospective teachers, for example, to use textbooks selectively and focus on the communicative activities they contain, to select CLT teaching and testing materials from other sources, and/or how to develop them themselves. In addition, in daily teaching practice teachers should be allocated more time to select, adapt or develop communicative teaching and testing materials. To guide teachers in this task, official curriculum documentation should not only refer to the CEFR but should also mention CLT principles more explicitly. Finally, language departments could be advised to collectively develop and implement communicative assessment programmes. This last point seems to be crucial since tests influence the pedagogical choices made by teachers regarding the selection of learning activities and time spent to develop students’ language skills. However, this also requires that FL teachers have a CLT oriented conception of language learning and know how to develop a communicative assessment program. Here we see an important role for teacher training programmes, as we already stressed above. In addition, design-based research in which teachers and researchers co-create, implement and evaluate communicative assessment programmes aligned with CLT goals could provide insight into characteristics of effective and realistic CLT assessment, and as such, inform both FL teaching practice and teacher training.

Disclosure statement

No potential conflict of interest was reported by the author(s).