The interplay between academic writing abilities of Dutch undergraduate students, a remedial writing programme, and academic achievement

ABSTRACT

Academic language proficiency, particularly academic writing, appears to be significantly related to academic achievement. The aim of the present study is to explore the relationship between written language proficiency and academic achievement, and to investigate the effects of a remedial academic writing programme. Academic writing was assessed by means of a Diagnostic Writing Test (DWT), administered to 5810 first year undergraduates in a Dutch university. Academic achievement was operationalised in terms of grade point average (GPA) and study credits obtained. One out of five students failed the DWT, even those who had obtained a (largely) sufficient exam mark for Dutch at high school. An additional analysis was conducted among over 300 students, in order to investigate the role of the DWT, as well as of other possible variables which may influence academic achievement (e.g. former training and time investment). Both the DWT and former training appeared to be related to GPA, but not the number of credits. Time-investment was associated with the number of credits obtained and GPA. Remediation appeared to be successful: about 75% of the participants who followed the course were able to convert a ‘fail’ on the DWT into a ‘pass’.

KEYWORDS:

• academic language proficiency
• academic writing
• academic achievement
• GPA
• diagnostic writing test
• remedial programme

Introduction

For a number of years, a lively debate has been taking place on the issue of the academic language proficiency of university undergraduates, and the specific linguistic skills that are required in order to achieve academic success. In the literature various terms have been used interchangebly (e.g. ‘academic success’, ‘study success’, ‘academic achievement’); in this contribution we will use the term ‘academic achievement’. Academic language proficiency, requiring mastery of abstract and discipline-specific vocabulary as well as knowledge of particular lexical and syntactic patterns, appears to be significantly related to academic achievement, as shown by a number of studies (August and Shanahan 2006; Schleppegrell and Colombi 2002).

In a review of 13 years of research into antecedents of university students’ grade point average (GPA), language proficiency did not correlate significantly with academic achievement (Richardson, Abraham, and Bond 2012). As universities were traditionally selective, with a linguistically more homogeneous student population, differences in language abilities did not pay a significant role. However, with widening participation and internationalisation, this situation has changed. Individual differences in language abilities have been found to predict academic outcomes (e.g. Daller and Phelan 2013). At the same time students with inadequate language skills appear to be seriously disadvantaged and hindered (Snow and Uccelli 2009; Uccelli et al. 2015). Next to linguistic problems, other factors than language proficiency (e.g. learning style, motivation, time investment, self-confidence, aptitude, socio-economic status) may also contribute to (lack of) academic achievement (e.g. Davies 2007). Similar problems have been observed in the Dutch context in particular, where educators have increasingly raised concerns about the lack of academic language skills of Dutch university students in their native language (Herelixka and Verhulst 2014; Raad voor de Nederlandse Taal en Letteren 2015; Rooijackers, Westen, and Van der Graus 2009).

The aim of the current study is two-fold. Firstly, we aim to explore the relationship between written academic language proficiency in Dutch and academic achievement of undergraduate university students during their first year of study (first and second semester). Our second objective is to investigate the effects of a remedial writing programme on academic language proficiency. Academic achievement is operationalised in terms of grade point average (GPA) and credits obtained (ECTS, European Credit Transfer System). Academic language proficiency is measured by means of a Diagnostic Writing Test (DWT), administered at the beginning of the first semester. The remedial course (‘Academic Writing’), set up for students who fail the test, aims to improve students’ written academic skills by providing extensive feedback on their writing and by coaching writing attitude and awareness (e.g. time management, respecting deadlines, focus not only on content but also on language). An additional analysis, which was based on self-reported data derived from a background questionnaire administered to a subset of the students, also took into consideration the role of other factors which might be related to academic achievement (e.g. former training and time investment).

The present paper reports on the results of the participants on the DWT, in relation to their academic achievement. After having defined the concept of academic language proficiency, we first describe the type of problems that undergraduate students have to face, followed by the design of the study and the analysis of the data from the background questionnaire. We then investigate the effects of the remedial programme on academic language proficiency. Finally, based on the findings of the study, we discuss implications for language testing, language policy, and remedial education.

Academic language proficiency and academic achievement

Several studies have shown that insufficient academic language proficiency in L1– in particular academic writing – may hinder academic achievement (Alderson, Clapham, and Steel 1997; Bachman 1991; Crisper and Davies 1988; Ferguson and White 1994; Hulstijn 2011, 2015). Compared to native students, L2 students do not enjoy the same level of academic success as L1 students and are at a greater risk of failing (Iannelli and Huang 2014; Paton 2007). What seems clear from this research on both L1 and L2 is that L1 and L2 learners appear to have many problems in common, as academic language is a special register that has to be learned by all students. Control over academic language is a requirement for challenging tasks, such as reading a scientific article, or writing a research paper or a literature review. Failure to understand discipline-related texts, to organise information, and to make appropriate lexical choices can thus be a serious obstacle (Biber 2006; Biber, Gray, and Poonpon 2011; De Wachter et al. 2013; Hyland 2009; Snow and Uccelli 2009; Turner 2011; Van Kalsbeek and Kuiken 2014).

An important finding in many of these studies is that the relationship between academic language proficiency and results obtained in the first year of study, in particular during the first semester, seems to be important and may even be predictive for success in the further study career. Hill, Storch, and Lynch (1999) established a relationship between language proficiency in L2, as assessed by means of TOEFL and IELTS scores, and the average marks obtained in the first semester. Daller and Phelan (2013) found that language proficiency, and especially vocabulary knowledge, appeared to be the key factor in predicting the final marks that students achieve. The study also showed that although standardised tests such as TOEFL and IELTS provide a valid cut-off point for entry to university, also other factors than language proficiency, such as cultural factors, motivation, and familiarity with the subject area appeared to influence academic achievement.

Academic language: definition and linguistic features

Despite the widespread concern about students’ insufficient language proficiency, there is no simple definition of ‘academic language’ and what it entails. In the last decades, several definitions have been proposed. Cummins (1980, 1981) distinguishes between Basic Interpersonal Communicative Skills (BICS) and Cognitive Academic Language Proficiency (CALP). BICS refers to conversational language use in a daily context. CALP is intrinsically more complex and refers to more abstract language use in formal, institutionalised settings (e.g. official reports, text books, or scientific publications). Despite the importance of Cummins’ work in raising awareness of the differences between conversational and academic language, it has not been specified which linguistic features are encompassed in CALP. This contrast between daily language and academic language is echoed in Hulstijn’s (2015) distinction between Basic Language Cognition (BLC) and Higher Language Cognition (HLC). Both for BICS vs CALP, and BLC vs HLC, it may, however, be hard to delineate the exact borders between what forms part of BICS or BLC, and what belongs to CALP or HLC.

Other studies have tried to describe a number of features which are thought to be typical of academic language. Krashen and Brown (2007) conceptualise academic language as being characterised by academic vocabulary, complex syntax, and complex discourse style. This characterisation, however, leaves open what exactly comes under the heading of ‘academic vocabulary’ and when syntax or discourse style are considered to be ‘complex’. Biber (2006) confines the concept to the university context, defining academic language as specific language patterns occurring across a broad range of spoken and written university registers. Academic language proficiency is described by Uccelli et al. (2015) in terms of knowledge and deployment of a repertoire of language forms that co-occur across disciplines. Along these lines, our conceptualisation of what constitutes (written) academic language proficiency follows Biber (2006) and Uccelli et al. (2015).

Attempting to classify the linguistic elements identified in the literature as being characteristic to academic language in their inventory of academic language features, Snow and Uccelli (2009, 119–131) subdivide these features into those referring to (1) interpersonal stance (detached, authoritative), (2) information load (conciseness, density), (3) organisation of information (embedding, use of discourse markers, argument structure), (4) lexical choices (diversity, precision, abstract/technical concepts), and (5) grammatical choices (complex sentences, use of abstract concepts as agents). Employment of these linguistic elements must be coordinated with genre mastery, disciplinary knowledge, reasoning skills, and use of argumentative strategies. The underlying rationale of these linguistic and cognitive requirements, as stated by Snow and Uccelli, are three communicative challenges: ‘representing the self’ (representation of one’s stance in the message to be transmitted; selection of a specific relationship with the audience), and ‘representing the message’ (conceptualisation and representation of ideas/thoughts in a particular context).

Hyland (2002, 2009) has noted that scientific texts – in comparison to other texts – contain a higher number of nouns, adjectives, and prepositions, whereas pronouns are less frequent. What characterises academic language is the relatively large number of nominalisations, particularly complex noun phrases, in which the noun is preceded and/or followed respectively by one or more pre- and/or post-modifiers (for example, ‘Cultural and Social Anthropology deal with many aspects of the social lives of people around the world, including our own society […] as well as the interrelationships of various systems in environmental adaptation and social change’; Biber 2006, 190). Other syntactic structures that often occur are passive sentences. With regard to lexical features, academic prose is generally characterised by abstract and specialised vocabulary (‘amplitude’, ‘chromatography’), particular collocations and lexical bundles (‘as a result of’, ‘the nature of’), discourse markers and connectors (‘alternatively’, ‘consequently’), expressions of stance and evaluation (‘actually’, ‘unfortunately’).

Obviously, as has been pointed out in many studies (Biber, Gray, and Poonpon 2011; Biber and Gray 2011; Snow and Uccelli 2009), there are a number of differences between oral and written academic discourse and between various disciplines. Personal pronouns, for instance, which may often be used in oral academic speech (‘We have to talk about some basic concepts. Let me show you them’), are hardly present in written discourse. Oral academic language may also contain many sentences introduced by ‘that … ’ (‘That the living standard of the people in the area was improved is the good thing. That’s what interests us’), as well as many adverbial clauses, such as conditionals (‘if’), causatives (‘because’) and temporals (‘before’). In contrast, written academic language is characterised by clausal embedding. As pointed out by Biber, the typical sentence structure of a written scientific text consists of a main clause with relatively few subordinate clauses, containing several noun phrases and a prepositional clause, for example: ‘Each new level of system differentiation opens up space for further increases in complexity, that is, for additional functional specifications and a correspondingly more abstract integration of the ensuing subsystems’ (Biber 2006, 76).

Writing skills of undergraduate students

Previous research has established that university students may start their studies with underdeveloped writing skills (e.g. De Wachter et al. 2013; Kuiken and Van Kalsbeek 2014; Snow and Uccelli 2009). As has often been observed in this strand of research, typical problems of both a lower order (spelling, grammar) and, particularly, a higher order (argumentation, organisation of information) are the following:

• - errors in spelling, punctuation and grammar;

• - problems with abstract/low-frequency vocabulary;

• - odd formulations and collocation errors;

• - use of an informal, colloquial register in a formal text genre;

• - difficulty to write in a concise, neutral and objective way;

• - poor sentence and/or text structure;

• - weak cohesive ties, few connectives and hedges;

• - problems with logical progression of the argument.

One may wonder why 18-year-old university students do not possess the writing (and reading) skills that could be expected at that age. With respect to the Dutch context, there are several reasons which may have contributed to the deficiency in writing proficiency. While the Netherlands formerly appeared in the top ten of best performing countries in international comparative studies like PIRLS (Progress in International Reading Literacy Studies) and PISA (Program for International Student Assessment), this is no longer the case. Reading scores of ten-year-old children in grade 4, as measured by PIRLS have decreased (Meelissen et al. 2012): Dutch pupils scored second out of 35 countries in 2011 (score: 554), tenth out of 45 countries in 2011 (score: 546). Kuhlemeier et al. (2013) and Kuhlemeier, Van Til, and Van den Bergh (2014) also observed that the writing ability of Dutch children has declined: although some children in grade 3 write as well as children in grade 6, the reverse is, unfortunately, also true. As a consequence, the writing ability of children in primary education turns out to depend largely on the school they attend and on the quality of the teacher.

According to PISA (Gille et al. 2010), the same picture emerges in secondary education, where the standards for writing proficiency set for the final year of high school are usually not attained by the majority of high school students (Van de Gein 2012). Furthermore, students between the age of 13–19 years who used to read for 23 min a day in 2013 appear to read no more then ten minutes a day in 2015 and experience less pleasure in reading a book (Wennekers, Van Troost, and Wiegman 2016), possibly due to their fascination for digital media, such as smart phones and computers (Valkenburg 2014). Finally, with more and more bilingual English-Dutch high schools in the Netherlands, English is gaining ground in secondary education, possibly at the expense of the proficiency level of Dutch high school children in their native language (De Graaff 2013).

In higher education, concerns about insufficient academic language skills and lack of academic literacy of prospective and first-year students have led to an array of recently adopted measures. Many universities in the Netherlands and Flanders have introduced a mandatory language test for all incoming students, in order to assess their level of academic language proficiency. Students who fail on the test are referred to – often compulsory – intervention and remedial programmes. The current contribution focuses on the evaluation of the DWT, developed at the university under investigation, and the effects of a remedial programme, which aims to train students’ academic writing skills.

Present study: design, research questions, and participants

Objectives and research questions

The main objectives of the present study are the following: (i) to investigate the relationship between written academic language proficiency, as measured by the DWT and academic achievement, in terms of grade point average (GPA) and credits obtained (ECTS, European Credit Transfer System), and (ii) to examine the effects of the remedial programme on the development of those skills. Our research questions are:

1. What is the relationship between the results on the Diagnostic Writing Test (DWT) and academic achievement, operationalised in terms of grade point average (GPA) and credits obtained?

2. What is the influence of background variables (i.e. former training and time investment) on academic achievement, operationalised in terms of grade point average (GPA) and credits obtained?

3. What are the results of the remedial programme ‘Academic Writing’?

Diagnostic Writing Test (DWT)

All first-year students who follow a Dutch-spoken Bachelor’s programme at the Faculty of Humanities of one of the universities in the Netherlands, sit the DWT at the start of the academic year. The DWT is a source-based writing test, which comprises both a reading and a writing component. Students have to read two source texts on the same topic (made available on line, prior to the test session) with a total length of approximately 3500 words. Both source texts are discipline related, as far as topic and content are concerned. There are six different sets of texts (12 texts in total). On the basis of these source readings, eight writing assignments have to be completed by the test takers, who are asked to compare the ideas and concepts expressed in the two texts, to paraphrase extracts of the text, or to formulate substantiated standpoints (750 words in total across eight assigments).

All texts are assessed along three main dimensions – content, comprehensibility, and accuracy – by individual raters (most of them teachers of academic writing), after a joint three-hours training session, in order to familiarise raters with the dimensions and criteria of the scale. Content refers to (a) how well students are able to paraphrase an extract of the source text and (b) formulate a substantiated standpoint. Comprehensibility addresses (c) the use of text coherence and cohesive devices, and of (d) concise, clear, and appropriate formulations. With respect to accuracy, both (e) lexical accuracy and (f) grammatical correctness are judged, and also (f) spelling and punctuation. Scores on these seven categories are attributed in terms of sufficient, marginal (i.e. on the edge) or insufficient. Students who score insufficient on two out of the seven categories or marginal on four out of the seven categories receive a fail. These students are referred to the compulsory remedial programme ‘Academic Writing’.

Participants and data

The participants involved in the study were five cohorts of university students (from 2013–2014 until 2017–2018), who all followed a Dutch-spoken Bachelor’s programme at the Faculty of Humanities of a university located in the Netherlands. For all these students (5810 in total) we present the results (in terms of pass/fail) obtained on the DWT. For the same five cohorts the findings are reported of all students (1017 in total) who were referred to the compulsory remedial programme ‘Academic Writing’. We also discuss the data derived from a student satisfaction survey study in which students were asked to express their degree of satisfaction with respect to the ‘Academic Writing’ course.

For a smaller sample of students, we examine the results in more detail. This sample consists of over 300 students who started in 2013–2014 at the department of Language and Literature. Data (self-reported) were obtained by means of the analysis of a background questionnaire. We present descriptive statistics of the final exam mark obtained for Dutch at high school and focus on the relationship between DWT-scores, former training and time investment on the one hand and study achievement on the other hand. Final exam marks for Dutch at high school range from one to ten, with 5,5 as cut-off point for pass/fail. Former training was subdivided into pre-university education, first year of higher vocational education and a miscellaneous category ‘other’. With respect to time investment four types of students were distinghuished: those who spent 32–40 h a week to their study, 24–32 h, 16–24 h and less than 16 h.

Results

Variables affecting academic achievement

We first looked at the overall results of the DWT of the cohorts of first-year students from 2013 to 2014 until 2017–2018 (see Table 1). Although the size of each annual cohort got smaller over the years (from 1794 in 2013–2014 to 774 in 2017–2018), due to both a decrease of the overall numbers of Humanities students and the increase of English-taught Bachelor’s programmes (at the expense of the number of Dutch-taught programmes), the percentages of students who passed the test remained more or less stable (on average over the years about 77%). This means that over the years, more than 20% of the students failed the test (on average about 23%; De Bakker et al. 2015; Van Meeteren 2015).

Table 1. Results on the DWT of first year humanities students (n = 5810).

Year	N	Pass (%)	Fail (%)
2013–2014	1794	78.3	21.7
2014–2015	1245	77.2	22.8
2015–2016	1111	78.6	21.3
2016–2017	886	71.3	28.6
2017–2018	774	79.4	20.6

We also took into account the effect of a number of background variables, derived from the background questionnaire. The analysis was conducted on a selected sample of participants. The first background variable that we considered was the former training of the students. The sample that was chosen for establishing the correlations between the DWT and former training consisted of 320 first-year students of Language and Literature (cohort 2013–2014).

As shown by the results contained in Table 2, the majority of the students (n = 213) had completed pre-university education. Of these students, 76.5% passed the DWT and 23.5% failed. Although these percentages appear to be in line with the overall results for the five cohorts of first-year Humanities students presented in Table 1, they also show that graduating from high school does not guarantee that students pass the DWT. Participants with other types of prior education (one year of higher vocational education) performed worse (pass: 49.5%; fail: 50.5%). The category ‘other’ consists of students with various types of former training (i.e. completed vocational education; pre-university education followed outside the Netherlands; university degree obtained elsewhere); of these students, 50.4% passed the test, and 49.6% failed.

Table 2. Relationship between the results on the DWT and former training of first year students Language and Literature 2013–2014 (n = 320).

Former training	N	Pass (%)	Fail (%)
Pre-university education	213	76.5	23.5
Vocational education 1st year	57	49.5	50.5
Other	50	50.4	49.6

We then investigated the relationship between the results obtained on the DWT and the exam mark for Dutch obtained at the final exam of high school (the minimum score that is required for entering university is 5.5). The analyses were again based on the same cohort of first-year students of Language and Literature (2013–2014). The findings are presented in Table 3. The total number of students (n = 307) is slightly different compared to Table 2, as some students (for instance, students with pre-university training outside the Netherlands) did not take Dutch as part of their final high school exam. Table 3 shows that of the 103 students who obtained a mark equal to or lower than 6.4, only 55.3% passed the DWT. What is more surprising, however, is that of the 147 students with a mark between 6.5 and 7.4, only 74.8% passed the DWT and 25.2% failed. Results were much better for students who obtained 7.5 or more on their high school exam for Dutch. As can be inferred from the table, a mark lower than 7.5 for Dutch at the exam for high school does not guarantee a pass on the DWT, let alone a mark of 5.5, which is considered to be a sufficient score for passing the high school exam for Dutch.

Table 3. Relationship between the results on the DWT and exam mark obtained for Dutch at high school for first year students of Language and Literature 2013–2014 (n = 307).

Exam mark	N	Pass (%)	Fail (%)
≤6.4	103	55.3	44.7
6.5–7.4	147	74.8	25.2
7.5–8.4	50	90.0	10.0
≥8.5	7	85.7	14.3

We subsequently examined the results in semester 1 and 2 in 2013–2014 of first-year students of Language and Literature (n = 320), in terms of study credits. The total number of credits to be obtained in semester 1 and 2 would ideally be 60 ECTS in total. The average number of credits of the cohort of students of Language and Literature turned out to be 32.6, which is only slightly more than half of the required number of ECTS in year 1. The standard deviation was, however, large (SD = 25.3), implying that there was a great deal of variation between individual students. One of the reasons therefor may be that some of the enrolled students combined two different Bachelor’s programmes and did not take all exams. The GPA was 7.2 (on a scale from 1 to 10; SD = .8). First we calculated the effect of the DWT on academic achievement in semester 1 and 2, in terms of number of ECTS and GPA (see Table 4). Although for semester 1 the DWT did not seem to be associated with the number of ECTS (F(1319) = .76, p = .467), it turned out to be related with the average marks (GPA; F(1319) = 4.51, p = .012). Results were similar for semester 2: no significant correlations for number of ECTS (F(1319) = .199, p = .729), but a significant correlation was found between DWT and GPA (F(1319) = 5.32, p = .006). We then looked at the role of two other factors that might be related to academic achievement: former training and time investment. Former training appeared to be significantly correlated with GPA in semester 1 (F(1319) = 3.44; p = .003) and 2 (F(1319) = 2.33; p = .033), but not with number of credits. Students who devote 32–40 h to their study obtain more ECTS in semester 1 (F(1,86) = 12.36; p < .001) and 2 (F(1,86) = 10.18; p < .001) compared to those with a time investment of 24–32 h. These students, in turn, obtain more ECTS in their first year than students who spend 16–24 h on study related activities. The latter, surprisingly, receive higher marks, both in semester 1 (F(1124) = 3.29; p = .021) and in semester 2 (F(1124) = 2.73; p = .032), possibly because this group consists mainly of part-time students.

Table 4. Variables affecting academic achievement of first year students Language and Literature 2013–2014 (n = 320).

Variables	Credits (ECTS)				Mark (GPA)
	Semester 1		Semester 2		Semester 1		Semester 2
	F	p	F	p	F	p	F	p
DWT	.76	.467	.199	.729	4.51	.012*	5.32	.006**
Former training	1.21	.299	1.41	.210	3.44	.003**	2.33	.033*
Time investment	12.36	.001***	10.18	.001***	3.29	.021*	2.73	.032*

*p < .05, **p < .01, ***p < .001.

Results of a remedial academic writing programme

As already mentioned in the previous sections, students who failed the DWT were referred to the compulsory remedial course ‘Academic Writing’. In addition to this course, they were also recommended to use other tools and online resources: a digital platform (taalwinkel.nl), which provides advice on oral presentations and academic writing; an online MOOC (moocbeterschrijven.nl), an open course for students in higher education, aiming to help them become more proficient writers; the Writing Centre (‘Schrijfcentrum’), where students receive individual support and feedback from a tutor concerning their writing problems; and the workshop ‘Spelling’, a three-hour crash course for students with spelling difficulties. In what follows we will focus on the remedial course ‘Academic Writing’.

The aims of ‘Academic Writing’, which consists of six meetings of two hours each, are three-fold: (1) consciousness raising: making students aware of the importance of writing a comprehensible text in an appropriate style and register, in relation to reader and audience; (2) where to find solutions: referring students to tools and resources for improving academic writing (including the ones mentioned above); (3) coaching of writing behaviour and attitude (e.g. development of efficient writing strategies and time management). In the course, students receive extensive, individual feedback on two writing assignments. General and frequently occurring writing problems are discussed in class – for instance, problems concerning the overall text structure and argumentation, organisation of information, use of anaphoric references, connectives and cohesive devices, style, and punctuation. Special attention is paid to the importance of planning and dealing with deadlines.

During five consecutive years (2013–2014 until 2017–2018), for the cohorts of students of whom the results on the DWT were presented above, data were also collected for the remedial course ‘Academic Writing’ (see Table 5). Over the years, the percentage of students who succeeded in converting a ‘fail’ into a ‘pass’ varies from 68% (2015–2016) to 82% (2013–2014), with an average of 75.8%. The course thus appears to be successful for three quarters of the students with weak academic language proficiency. It should, however, be noted that 10%–12% of the students who are referred to the course never attended (in most cases students who already have dropped out). A yearly average of about 15% of the students who failed the DWT had not demonstrated sufficient improvement at the end of the course. These were apparently the weaker students, who probably needed more support in order to reach the required level of academic language proficiency.

Table 5. Results of the remedial course ‘Academic Writing’.

Year	N	Pass (%)	Fail (%)	No show (%)
2013–2014	303	82	18*
2014–2015	214	77	11	12
2015–2016	195	68	21	11
2016–2017	186	77	13	10
2017–2018	119	75	14	11

*In 2013–2014 no distinction was made between students who failed on the remedial writing course and those who did not show up.

For the five cohorts (2013–2018), we also conducted a student satisfaction survey study in which students were asked to express their degree of satisfaction with respect to the ‘Academic Writing’ course, on a five-point Likert scale. The majority of the students (on average a high 70%) turned out to be (very) satisfied with the course, about 20% (on average) expressed a neutral judgment, while on average only about 6% were (very) dissatisfied with the course (see Table 6).

Table 6. Results of student satisfaction survey on ‘Academic Writing’.

Year	N	(Very) satisfied (%)	Neutral (%)	(Very) dissatisfied
2013–2014	216	71.5	23.5	5.0
2014–2015	113	65.1	26.0	8.9
2015–2016	109	59.6	31.2	9.2
2016–2017	129	76.7	19.3	4.0
2017–2018	53	90.6	5.7	3.7

Discussion and conclusion

Concerns about the insufficient academic language proficiency of Dutch university students, particularly academic writing, appear to be legitimate, given the results of the present study. When assessed by means of the DWT, one out of five students did not pass the test. Results are fairly stable over five consecutive academic years. Only 55% of the students who had obtained a mark of 6.4 or lower (out of ten) for Dutch at their final exam passed the DWT, and even approximately 25% of the students with a mark for Dutch between 6.5 and 7.5 failed the DWT. This means that a pass for Dutch at the final high school exam apparently does not guarantee a sufficient level of academic language proficiency when students enter university. Further analyses were performed, in order to answer the first research question: what is the relationship between the outcomes on the DWT (in terms of pass/fail) and academic achievement? It turned out that the DWT predicts the GPA (general point average) in year 1, but not the number of study credits obtained.

With respect to the second research question (which background variables affect academic achievement), it was found that former training predicted GPA, but not the number of credits obtained. Furthermore, students who invested 32–40 h in their study obtained more credits in semesters 1 and 2, whereas students with a time investment of 24–32 h had a better GPA, both in semester 1 and 2.

The third research question addresses the results of the remedial course (‘Academic Writing’). The main finding was that about three quarters of the students who had taken the course were able to convert a fail into a pass. It should, however, be noted that there was no control group of students who had not followed ‘Academic Writing’, as participation to this course was compulsory for those who had failed on the DWT. A satisfaction survey showed that approximately three quarters of the participants were (very) satisfied with the course. Only a small percentage of 6% of the students mentioned being (very) dissatisfied with the course. These findings turned out to be (more or less) consistent over the five years cohorts from 2013–2014 to 2017–2018.

What these results tell us is that the DWT fulfills an important role in detecting students with a weak academic language proficiency. They also show the importance and necessity of a remedial programme for students who need to improve their writing skills, by means of a course which focuses on detailed feedback, coaching, and increasing awareness of writing behaviour and attitude.

There are, however, some limitations to the present study and other issues that need to be further investigated. For reasons of time and practicality the analysis of the relationship between the results on the DWT on the one hand and study achievement and background variables on the other hand was based on a small subset of about 300 students out of a total of more than 5800 test takers of the DWT. Another limitation is that we only looked at this relationship with academic achievement during the first year of study. Now that data have been collected over five consecutive years, it is important to examine the role of the DWT over a longer stretch of time, for instance the whole Bachelor’s programme.

The next steps, therefore, will be to investigate the influence of the DWT on academic achievement in the long run. Also the relationship between DWT, study achievement, and the impact of individual learner factors (e.g. learning style, motivation, self-confidence, aptitude, socio-economic status) needs to be analysed in more detail. Another issue for further research concerns the efficiency and practicality of the DWT. The test in its actual form assesses both lower-order and higher-order writing skills. In this way, useful information is gathered about the academic writing proficiency of the test takers. These qualities make the DWT a time-consuming instrument, both for candidates who take the test and for raters who have to assess students’ writing performance. A future challenge will be to explore ways of collecting these relevant data in a less time-consuming way, in order to develop equally valid but more time-saving test formats which can provide the same amount of necessary information.

A final issue to be investigated concerns the growing amount of English-taught study programmes at Dutch universities, and the increased numbers of international students. Preliminary findings point into the direction that, even if students have obtained the required TOEFL or IELTS levels, they are still facing similar problems regarding their academic proficiency level in English as students of the Dutch-taught programmes. As such, it is crucial to conduct future studies, comparable to the current study, regarding the association between academic language proficiency in English and academic achievement.

Finally, it is important to point out that the problem of insufficient academic language proficiency has to be addressed both by secondary and higher education. Joint efforts have to be made, in order to develop a coherent, focused, and ongoing language curriculum, which facilitates students’ transition from pre-university education to higher education. Universities have to develop an ongoing academic language programme, both for Dutch and for English, starting in the first Bachelor’s year until the end of the Master’s programme. This implies that teachers should not restrict themselves to their own disciplines, but they should also help students to pay more attention to academic language use and the employment of appropriate formulations.

Disclosure statement

No potential conflict of interest was reported by the author(s).