The contribution of general language ability, reading comprehension and working memory to mathematics achievement among children with English as additional language (EAL): an exploratory study

ABSTRACT

An increasing number of high-stakes mathematics standardised tests around the world place an emphasis on using mathematical word problems to assess students’ mathematical understanding. Not only do these assessments require children to think mathematically, but making sense of these tests’ mathematical word problems also brings children’s language ability, reading comprehension and working memory into play. The nature of these test items places a great deal of cognitive demand on all mathematics learners, but particularly on children completing the assessments in a second language that is still developing. This paper reports findings from an exploratory study on the contribution of language to mathematics achievement among 35 children with English as an Additional Language (EAL) and 31 children with English as their first language (FLE). The findings confirm the prominent role of general language ability in the development and assessment of mathematical ability. This variable explained more variance than working memory in word-based mathematics scores for all learners. Significant differences were found between the performance of FLE learners and EAL learners on solving mathematical word-based problems, but not on wordless problems. We conclude that EAL learners need to receive more targeted language support, including help with specific language knowledge needed to understand and solve mathematical word problems.

KEYWORDS:

• English as an additional language (EAL)
• mathematics achievement
• general language ability
• reading comprehension
• working memory

Introduction

The role of language in mathematics learning and teaching is undeniably crucial. Building on the work of Bruner (1966), Haylock and Cockburn (2013) argue that language is one of the four key experiences of mathematics learning in addition to concrete manipulation, pictures and symbols. However, not only do mathematics learners have to learn a large number of technical and abstract terms that are specific to the subject (e.g. trapezoid and rhombus; sine, cosine and tangent functions), they also need to have a good knowledge of everyday vocabulary as mathematical problem solving has become increasingly more grounded in everyday contexts. Potential confusion arises when the boundary between these two types of language becomes blurry. For example, a lexically ambiguous term, such as odd can be taken to describe something that is strange or abnormal in the everyday context, but when it is used as a mathematical term in relation to numbers, it can be taken to describe any integer that cannot be divided exactly by 2. More examples of lexically ambiguous words include ‘volume’, ‘degree’ and ‘root’ among several other examples. To some mathematics learners, the boundary between these two types of language becomes even more blurry when they encounter homophones, that is words that sound the same, but have different meanings in different contexts, for example, ‘pi’ vs ‘pie’ and ‘serial’ vs ‘cereal’ (Adams, Thangata, and King 2005). The difficulties involved in distinguishing between the everyday usage of terms and their specific academic meaning illustrate Cummins’ (2003) distinction between basic interpersonal communicative skills (BICS) and cognitive academic language proficiency (CALP): the former is taken to refer to the language required for social situations while the latter refers to language which is necessary for academic success. Acquiring CALP in a second language is particularly challenging for learners for whom the language of instruction is not their first language. According to Cummins, these learners may take five to ten years to develop age-appropriate command of CALP.

In the context of England, such linguistic complexities are particularly relevant as the recently revised primary mathematics curriculum (Department for Education 2013, 4) emphasises the role of context in mathematics teaching and learning, for example, Year 2 pupils (6–7 years old) should be taught to ‘solve simple problems in a practical context involving addition and subtraction of money of the same unit, including giving change’. These linguistic features highlight the important role of mathematics learners’ vocabulary knowledge and reading comprehension skills, particularly in the context of solving mathematical word problems, which has increasingly become a standard assessment tool to measure learners’ mathematical understanding and performance. Not only can these word problems be found in England’s Standardised Assessment Tasks (SATs), they can also be found in other national tests, such as the USA’s National Assessment of Educational Progress (NAEP) and Australia’s National Assessment Program – Literacy and Numeracy (NAPLAN) as well as international tests, such as the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA). Whilst the linguistic features, as previously outlined, place a great deal of cognitive demand on all mathematics learners, they are arguably much more challenging for learners for whom English is an additional language (EAL) and who are, more limited in terms of their vocabulary knowledge and reading comprehension skills, particularly when compared to their peers who have English as their first language (FLE) (Burgoyne et al. 2009).

While there is considerable evidence that bilinguals (including EAL children) have smaller vocabularies than monolinguals, and smaller vocabularies are thought to be in part responsible for lower levels of reading comprehension in EAL children (e.g. Murphy and Unthiah 2015), there is virtually no research into the contribution of these variables to EAL children’s mathematics achievement. The current study aims to fill this gap by furthering our understanding of the contribution of reading skills and language ability to mathematical knowledge in EAL children.

EAL children’s academic achievement

Studies from different countries have found that EAL children’s mathematical achievement is generally higher than their language achievement. In Australia, for example, the gap in academic performance of EAL children is less marked in mathematics than it is for reading (Australian Curriculum, Assessment and Reporting Authority 2016). According to Strand, Malmberg, and Hall (2015), this is also the case in the UK where the scores of EAL children on mathematics assessments are always higher than on reading assessments at every age. At age 11, there is a sizeable gap for reading only, but no gap for mathematics, grammar, punctuation and spelling, whilst by age 16, EAL students are slightly more likely than FLE students to achieve a good pass in mathematics for the General Certificate of Secondary Education (GCSE), the examination taken in England at age 16. Yet, when compared with other contexts, ‘there is far less consistency with respect to EAL children’s L1 background’ in the UK (Murphy and Unthiah 2015, 34), with over 240 different languages reportedly spoken as a first language by primary school-aged children in the UK (Demie, Hau, and McDonald 2016). Furthermore, even though the number of EAL children in UK primary schools has doubled from 7.6% in 1997 to 16.2% in 2013 (Strand, Malmberg, and Hall 2015), much less research has been undertaken with EAL learners in the UK than elsewhere, underscoring the need for further studies of mathematics within that context. One possible reason for the limited amount of research conducted in the UK is that assessing EAL children with tests that were developed for monolinguals is not appropriate for many reasons and monolingual norms associated with these tests are often inappropriate for bilinguals (Martin et al. 2003; Gathercole 2013).

Furthermore, the heterogeneous nature of the EAL learner population in the UK makes it problematic to refer to the existence of an ‘EAL gap’. Strand, Malmberg, and Hall (2015), for example, note that there is a difference between (a) EAL students who report English as their main language and (b) EAL students who report a language other than English as their main language, with the first group obtaining higher scores than the second and they conclude that the range of achievement among EAL students is as wide as for FLE students. Understanding the causes of this variability is very important for policy makers and teachers. A recent report on secondary school attainment in England states that proficiency in English is the key factor behind the differences in academic achievement (Demie, Hau, and McDonald 2016).

Exploring potential relationships between general language ability, reading comprehension, and mathematical ability

As previously indicated at the end of the introduction section, general language ability and reading comprehension are closely linked to pupils’ ability to make sense of mathematical word problems. However, given the lack of research exploring the contributions of these variables to mathematical proficiency in EAL children, we thus hope to address this research gap. Each of these two variables will now be explored in turn.

General language ability

There is evidence to suggest that proficiency in mathematics is related to general language ability (e.g. Fuchs et al. 2015). We subscribe to the view that there are generic as well as specific components in individual language proficiency. According to Harsch (2014, 153) ‘language proficiency can be conceptualised as unitary and divisible, depending on the level of abstraction and the purpose of the assessment and score reporting’.

Investigations of general language ability and its relationship with mathematical ability, need, however, a valid and reliable measure of the former. Arriving at a single score (which is based on a set of separate scores for different skills or components of language proficiency) to represent unitary aspects of language proficiency is a challenge. There are few tests which claim to measure EAL learners’ general language ability, apart from the large scale tests that are used for admission to university, such as the IELTS or TOEFL for English which measure a wide range of skills which are then converted to an overall score. If such a test exists, it should correlate strongly with tests of different skills and/or components of language ability (vocabulary and grammar). Eckes and Grotjahn (2006) claim that the C-test, which was developed by Raatz and Klein-Braley (1981), does tap into general language proficiency, rather than specific skills (e.g. either reading or writing). In their study, the C-test correlates strongly with written as well as oral tests of German as a foreign language. Similar results were obtained by Dörnyei and Katona (1992) who show that the C-test can be used to measure global language proficiency. In Eckes and Grotjahn’s (2006) study, strong correlations were found between the C-test and vocabulary and grammar, although the correlations with vocabulary were stronger than those with grammar.

A C-test is a variant of the cloze test. Instead of deleting entire words, the second half of every second word is deleted (e.g. ‘The deleted parts are given in bold’), where the deleted parts of the words are given in bold. The test generally consists of five or six short texts in which approximately 20 gaps need to be filled. Using factor analysis and Rasch analysis, Eckes and Grotjahn (2006) provide comprehensive evidence that their German C-test is unidimensional. The C-test format could potentially reveal very interesting differences between EAL learners and children whose first language is English, because the C-test is an integrative test which does not only tap micro-level skills (word level skills), but also macro-level skills – some of the gaps can only be filled if the grammar or the lexis in other parts of the text are taken into account (Klein-Braley 1994). We know little about EAL learners’ ability to deal with macro-level skills, which is why the C-test could reveal important new information about the specific language proficiency profiles of EAL learners. The current project thus aims to contribute not only to the discussion about the differences in language proficiency between FLE and EAL mathematics learners, but also further our understanding of the suitability of this particular test format in assessing EAL children.

Reading comprehension

On the basis of the evidence reviewed in their paper, Murphy and Unthiah (2015) suggest that weaker reading comprehension skills may in part be responsible for lower levels of academic achievement among EAL learners. It might also be worth noting that one needs both decoding (word recognition) and linguistic comprehension skills (the ability to use lexical information to achieve sentence and discourse level interpretation) in order to be able to read (Hoover and Gough 1990). Decoding and linguistic comprehension are the two key dimensions of a widely used model of reading ability: the Simple View of Reading (Gough and Tunmer 1986). While most researchers in the field accept that mastery of these two dimensions is needed, it is also important to note that they are independent of each other. It is possible to have very good comprehension skills but poor decoding skills (as is the case in dyslexia), whilst there are also readers with good decoding skills but poor comprehension, often referred to as ‘poor comprehenders’ (Yuill and Oakhill 1991). The implications of poor comprehension for mathematics learning are apparent in a study by Pimperton and Nation (2010) of twenty-eight 7–8 years old children who took three tasks (of mathematical reasoning, verbal ability, and phonological short-term memory) over two testing sessions. Their findings regarding poor comprehenders’ significantly lower scores than controls on the mathematical reasoning task suggest that poor comprehenders need identifying early, in order to rectify potential linguistic deficiencies that impact on other aspects of learning. Building upon Pimperton and Nation (2010), Bjork and Bowyer-Crane (2013) examined the cognitive skills used by sixty 6–7 years old with mathematical word problems and numerical operations. The children in their study completed five tasks (reading accuracy, reading comprehension, phonological awareness, verbal ability as well as numerical operations and mathematical word problems) over two testing sessions. Their findings suggest poor reading comprehension is related to difficulties with mathematical word problems, though not numerical operations. The available evidence suggests that EAL learners have similar comprehension problems to those of poor comprehenders, whilst single word reading is less problematic for this group (Hutchinson et al. 2003; Burgoyne et al. 2009). In Burgoyne et al.’s (2009) study, EAL children even outperformed monolingual children in tests of decoding (accuracy of reading single words or connected text), but they clearly lagged behind their monolingual peers in listening and reading comprehension.

Vocabulary has been found to be a key predictor of reading comprehension among both first language learners (Ouellette and Beers 2010; Tunmer and Chapman 2012) and EAL learners (Melby-Lervåg and Lervåg 2014). Burgoyne et al. (2009, 742) conclude that ‘the weaker vocabulary skills of EAL learners […] place significant constraints on their comprehension of written and spoken text’. In particular, Burgoyne et al. (2009) suggest it is because of their smaller vocabulary that EAL learners are less able to make good use of written texts to support the formulation and expression of responses to comprehension questions. While vocabulary is recognised by many authors as being of key importance, little research has been done into the contribution of general language ability to reading comprehension and academic achievement. There is a lack of consensus as to whether aspects of oral language skills have a similar relation to reading comprehension for L1 and L2 learners. In a Canadian longitudinal study, Lesaux, Rupp, and Siegel (2007) followed L1 and L2 English-speaking kindergarteners to the fourth grade. It was found that a measure of syntactic skills (cloze task) in kindergarten predicted reading comprehension of both groups of children. Conversely, Droop and Verhoeven (2003) found a stronger effect of morphosyntactic skills (sentence repetition) on the reading comprehension performance of L1 Dutch speaking children (8–10 year olds) than on that of their L2 peers. Finally, Babayiğit (2014) explored the relationship between oral language (i.e. vocabulary and morphosyntactic skills [sentence repetition]) of English-speaking L1 and L2 children (9–10 year olds) in England. Evidence was found to suggest a predictive relationship between oral language and reading comprehension levels with a tendency for the relationship to be stronger in the L2 than in the L1 group. In a more recent study, Babayiğit (2015) extended her findings to show that weaker oral language skills (vocabulary, sentence repetition, verbal working memory) explained the lower reading comprehension performance of English-speaking L2 learners (10 years old). The current study aims to shed more light in this field by including a different measure of general language ability (C-test) among the variables studied in relation to EAL children’s academic achievement in the domain of mathematics.

Relationships between general language ability, reading comprehension and mathematical ability

A traditional method of measuring mathematical ability has been to assess pupils’ ability to solve word problems (Abedi and Lord 2001). Typically, pupils are expected to choose and collate relevant information from the problem, and to use them to solve the problem. Making sense of mathematical word problems can thus present a challenge for EAL learners whose command of English language is still developing. Vilenius-Tuohimaa, Aunola, and Nurmi (2008) subscribe to the view that there is a close relationship between reading comprehension and mathematical ability to the extent that reading comprehension skills appear to predict performance on mathematical word problems. Other studies appear to suggest that lexical complexity (e.g. word frequency, the use of idiomatic phrases and words with multiple meanings, and culturally-specific non-mathematical vocabulary items) of mathematics word problems might influence comprehension difficulties for EAL pupils with mathematical word problems (Abedi and Lord 2001).

Taken together, research exploring the relationship between reading comprehension, language and mathematical ability is fragmented, with previous studies arguably tending to focus on exploring either the relationships between mathematical performance and reading comprehension or between mathematical performance and vocabulary knowledge separately (e.g. Fuchs et al. 2015). Additionally, the findings are, to an extent, inconclusive. For example, while Vilenius-Tuohimaa, Aunola, and Nurmi’s (2008) study of 225 Grade 4 (Year 5) children in Finland found that children’s performance on solving mathematical word problems was strongly related to performance in reading comprehension, Imam, Abas-Mastura, and Jamil’s (2013) study of 666 public high school students in the Philippines found no statistically significant correlation between these two variables. The current study will attempt to contribute to this field by comparing and contrasting EAL and FLE children’s mathematical performance, and the relationship, if any, with their reading comprehension skills and general language ability.

Exploring relationships between children’s working memory performance and their mathematical ability

Closely linked to both linguistic and mathematical proficiencies is working memory or the mental space that is involved in the controlling, regulating, and maintaining of relevant information needed to achieve complex cognitive tasks (Miyake and Shah 1999). With particular regard to mathematical ability, such complex tasks require competence and control for specific procedures (e.g. arithmetic, algebra) as well as problem solving, which requires the temporary holding of information required to arrive at particular solutions, calling upon working memory resources. While numerous studies suggest that working memory is a significant predictor of mathematical abilities and outcomes (Passolunghi and Siegel 2001, 2004; Passolunghi and Pazzaglia 2004), the same predictive power was not found in other studies, for example, Bull, Johnston, and Roy’s (1999) study of 7 years old children and Swanson’s (2011) study of 6–7 years old children. Subsequently, the relationship between working memory and mathematical ability is far from clear, and indicate that there is some way to go before we can evaluate how working memory and mathematical ability might be interrelated, particularly in relation to EAL learners. This is thus another area that this study sets out to investigate.

The current study

The current study builds on the research outlined in the previous section by focusing on the following three research questions:

1. How does EAL learners’ performance on mathematics (word problems and wordless problems) tests differ from that of FLE learners?

2. How does EAL learners’ performance on reading comprehension, general language ability (English) and working memory tests differ from that of FLE learners?

3. To what extent are EAL children’s reading comprehension level, general language ability and working memory scores related to their mathematics word problem solving performance?

Methodology

Participants

This study focused specifically on Year 5 children (9–10 years old) because they are one of the two age cohorts (9–10 and 13–14 years old) that are tested, every four years, in the internationally-recognised Trends in International Mathematics and Science Study (TIMSS). One of the key benefits of such alignment is the access to a wealth of TIMSS’s publicly-released and well-constructed mathematics test items that set out to measure (9–10 years old) children’s mathematical proficiency in three different domains, namely number, geometric and data display. Additional details of these test items are included in the following section.

Fifty-two urban schools across the south of England were approached because of their reported statistics of EAL children in their schools. Nine schools were happy to take part in the study and the percentage of EAL children in these schools ranged from 46.8% to 73.7%, and the percentage of children who were eligible for free school meals (an index of deprivation widely used in the UK) ranged from 20.6% to 32.4%. Of these four schools, 35 EAL children (16 boys: 19 girls) and 31 FLE children (17 boys: 14 girls) agreed to participate, and they represented a wide range of ability levels in mathematics and reading. The EAL children in our study came from 11 different countries (China, Egypt, Germany, Iceland, India, Italy, Pakistan, Poland, Portugal, Romania or Uganda) and they spoke 10 different first languages (Arabic, German, Icelandic, Italian, Konkani, Luganda, Polish, Romanian, Telugu or Urdu) in addition to English.

Purposive sampling strategy was employed in this study whereby Year 5 children who had arrived in the UK from a non-English speaking country within the past five years were invited to take part. At the time of data collection, the children in the EAL group had been in the UK for 2.72 years on average (SD 1.61). The following distribution gives further details: 0–12 months – 9 children; 13–24 months – 11 children; 25–36 months – 5 children; 37–48 months – 4 children; and 48–60 months – 3 children (with missing data for 3 children).

Survey instruments and procedure

The data collection took place between December 2014 and March 2015, and in June 2016. A battery of five tests was administered to each child and this took around an hour per child to complete.

To assess mathematical achievement, the children were asked to solve 20 mathematical test items: five wordless arithmetic problems; five word problems on numbers (e.g. ‘Paint comes in 5 L cans, Sean needs 37 L of paint. How many cans must he buy?’ Correct answer: 8); five word problems on shapes and measures (e.g. ‘The school playground is a square. The playground is 100 metres long. Ruth walks all the way around the edge of the playground. How far does she walk?’ Correct answer: 400 metres); five word problems on data handling (e.g. ‘There is a picture of a scale with apples on it, with an arrow pointing towards a mark in between two points labelled 200 and 250. How much do the apples weigh?’ Correct answer: 220). The fifteen word problems were obtained from the Grade 4 (Year 5) TIMSS 2011 study with permission from the International Association for the Evaluation of Educational Achievement (IEA), Boston College. The 5 wordless problems (e.g. ‘28 × 29 = ?’ and ‘41 ÷ 5 = ?’) were created by the research team, as drawn from England’s primary mathematics curriculum (Department for Education 2013). Children had 25 minutes to complete the mathematics assessment.

Snowling et al.’s (2009) York Assessment of Reading for Comprehension (YARC) was used to assess children’s reading ability (accuracy, comprehension and rate). The test consists of two different components, namely a Single Word Reading Test (SWRT) and a reading comprehension test. The former comprises 60 words, ranging from ‘see’ and ‘look’ to ‘haemorrhage’ and ‘rheumatism’. For each child, an accuracy score and a reading rate was recorded for the SWRT. The SWRT score also informs the level of the first YARC reading passage they are expected to read. If children scored four or less in answering questions relating to the first passage, they would be asked to read a passage of a lower difficulty level. If they scored more than four, they would be asked to read a passage of a higher difficulty level.

General language ability was measured with a C-test, which consisted of three different texts. Two of these were created on the basis of texts from Corbett’s (2013) writing models for Year 5, and the third (Ali Baba) came from the animated tales for Year 5, published by the Hamilton Trust (2016). This third text was taken from a different source, because there were not enough suitable texts with clearly different topics in Corbett’s volume. We first created a C-test following the traditional deletion principle, deleting the second half of every second word, but found in piloting that this task was very difficult for the children. Therefore, we simplified it by deleting the second half of every third word instead. We also simplified some of the vocabulary items by replacing low frequency items by higher frequency words. For example, in The Stormy Rescue: ‘the rain had finally slackened’ was changed to ‘the rain had finally stopped’ and ‘she squinted out through the side windows’ became ‘she jumped out through the side windows’. While we acknowledge that in some cases we were unable to match the central meanings of the replaced words (e.g. ‘squinted’ vs ‘jumped’), in such cases our primary intention was to increase the likelihood that our texts would be accessible to both subject groups. Each passage contained 20 missing word parts (deletions in bold), for example, ‘She had left the window down just in case the fox came back’ (The Stormy Rescue). The maximum scores for each passage was 20.

To assess working memory, the children were asked to complete the backward digit span working memory task, from the Automated Working Memory Assessment (AMWA, Alloway 2007). In order to ensure that the working memory task was accessible to the EAL and FLE groups, we avoided vocabulary-specific working memory tasks (e.g. Gaulin and Campbell 1994), as well as domain-specific working memory tasks (e.g. Passolunghi and Costa 2016). The adopted task ranges from Span 2 (two digits) to Span 7 (seven digits). The discontinuation rule is to stop after 3 consecutive errors. If the child was successful on four trials of any given span level, they could proceed to the next level. Working memory scores were measured using longest backwards digit score.

To avoid any potential order effect, half of the children (comprising both EAL and FLE children) were asked to take the mathematics test before the reading comprehension test, and the other half were asked to take the reading comprehension test before taking the mathematics test.

Preliminary data analysis

The children’s responses on the reading comprehension were carefully interpreted and scored by members of the research team according to guidance provided in the YARC scoring manual. To investigate reliability, two members of the research team randomly chose the reading comprehension test response of ten children (from the total of 66 children), representing around 15% to be moderated. Of these ten children, five were those with EAL and the other five children were those with FLE. The inter-rater reliability for the reading comprehension was very high at 96.05%, calculated using Cohen’s kappa (Cohen 1960). We also computed ability scores for each of the two passages and a mean of these two variables following the procedures in the YARC manual. The reliability of the two reading comprehension passages was high with a Cronbach’s alpha coefficient of .873.

Similarly, the C-test marking was also moderated to ensure reliability. Two members of the research team marked the tests independently and then met to draw a list of children’s responses that required further discussions. Altogether, 20 responses across the five passages were highlighted, ranging from misspellings (e.g. cliff vs clif ), grammatical errors (e.g. bags vs bag ) and semantic differences (e.g. soon vs somehow ). While for adults it is common to accept only correctly spelled answers (Eckes and Grotjahn 2006), this is not necessarily the best approach with low-proficiency learners. In, for example, Linnemann and Wilbert’s (2010, 117) study of Grade 9 learning disabled students in Germany, ‘each correct restoration scored one point, that is, the original word or an orthographically, grammatically and semantically correct variation’. In our study, on careful consideration of the learner proficiency levels and the consistency of some minor errors, we took a slightly different approach and awarded a point for grammatically and semantically correct variations as well as simple misspellings (e.g. cliff vs clif ) as it can be argued that they still demonstrate comprehension by our young participants. The reliability of the language ability test was high with a Cronbach’s Alpha (CA) coefficient of .883. Finally, the mathematics test was highly reliable with a CA coefficient of .805.

Results

Research Question 1: How does EAL learners’ performance on mathematics (word problems and wordless problems) tests differ from that of FLE learners?

The mathematics test data were normally distributed and did not violate the assumptions for the use of parametric tests. Therefore, we first ran an ANOVA to see if there were significant differences between the total mathematics scores for the two groups. This turned out not to be the case (see Table 1 below). We expected the differences to be non-significant, because the total mathematics score consists of wordless and word-based problems and we expected differences to only emerge with the word-based problems. Therefore, we carried out a Multivariate Analysis of Variance (MANOVA) with the total wordless mathematics score and the word-based problems as dependent variables and the grouping variable (EAL versus FLE) as the independent variable, which was indeed significant (F (2,61) = 3.261, partial η² = .097, p = .045). Further analyses revealed that of these two, only performance on the word-based mathematics problem solving performance was significantly different between the two groups (F(1,62) = 6.543, partial η²= .095, p = .013). Among the three components of word-based mathematics items (i.e. numbers, shapes and measures, and data handling), the numbers component was the only one which was significantly different (F (1,64) = 6.153, partial η²= .088, p = .016).

Table 1. Results of a series of ANOVA and MANOVA comparing EAL and FLE mathematics test total mathematics and component scores.

Measure		EAL	FLE	F	p-value	Effect size partial η^2
		Mean (SD)	Mean (SD)
Total mathematics score		10.43 (4.2)	12.38 (5.6)	n.s.
Wordless arithmetic score		1.91 (1.25)	2.36 (1.64)	n.s.
Total word-based score		8.27 (3.2)	10.52 (3.8)	6.543	.013*	.095
Word-based component scores	Number	2.46 (1.9)	4.48 (4.4)	6.153	.016*	.088
	Shapes & measures	2.62 (1.33)	2.84 (1.86)	n.s.
	Data display	3.62 (1.23)	3.88 (1.21)	n.s.

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

Research Question 2: How does EAL learners’ performance on reading comprehension, general language ability (English) and working memory tests differ from that of FLE learners?

Before examining the relationship between reading comprehension ability, general language ability, working memory and mathematics scores, we first examined how EAL and FLE learners differ in these areas. Working memory scores and single word reading test (SWRT) scores were not normally distributed. Therefore, a series of non-parametric Mann Whitney U-Tests was conducted with the results displayed in Table 2. In line with previous research (e.g. Twist, Schagen, and Hodgson 2007), FLE learners outperformed EAL learners in reading comprehension. In fact, all language test scores were significantly higher for the FLE group than for the EAL group. Effect sizes ranged from .27 for the YARC reading comprehension test to 3.9 for the C-test. The high standard deviations for the latter indicate that there was considerable variation in performance within both groups too. However, there were no significant differences in working memory score between the two groups. The finding that SWRT scores were significantly lower for EAL learners is surprising in the light of Burgoyne et al.’s (2009) assertion that single word reading and decoding is less problematic for EAL learners compared to reading comprehension. This issue will be taken up in more detail in the discussion section.

Table 2. Results of Mann Whitney U-tests comparing EAL and FLE reading comprehension, single word reading, general language ability and working memory.

Measure	EAL	FLE	Z	p-value	Effect size (r)
	Mean (SD)	Mean (SD)
Reading comprehension (YARC)	39.32 (8.7)	50.23 (6.9)	2.172	.030*	.27
SWRT (YARC)	39.34 (12.90)	47.94 (7.38)	2.690	.007**	.33
General language ability (C-test)	34.78 (17.66)	48.81 (18.85)	3.201	<.001***	.39
Working memory	12.49 (6.09)	14.29 (4.86)	1.613	.107	.20

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

For the EAL group, length of residence correlated moderately and significantly with all measures: reading comprehension (r_s= .491, p = .008), general language ability (r_s = .373, p = .036), single word reading (r_s = .489, p = .004) and word-based mathematical problem solving (r_s = .399, p = .029). Unsurprisingly, those learners who had been in the UK longer performed better on all tests.

Research Question 3: To what extent are EAL children’s reading comprehension level, general language ability and working memory scores related to their mathematics word problem solving performance?

To answer this question a series of Spearman correlations was performed and the results are displayed in Table 3.

Table 3. Correlations between reading comprehension, SWRT scores, language ability, working memory and mathematics word-based score.

	Word-based mathematics Problem solving score
	EAL group	FLE group
Reading comprehension score	.642**	.296
SWRT score	.478**	.394*
General language ability (C-test) score	.450**	.460**
Working memory score	.374*	.484**

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

The results indicate that there were differences in the relationship between word-based mathematical problem-solving performance and measures of literacy between the EAL and FLE group. For the EAL group, there was a significant correlation between reading comprehension and word-based mathematics problem solving performance, which is in line with results of previous research (e.g. Abedi and Lord 2001). Single word reading ability, general language ability and working memory correlated significantly with mathematics scores in both groups. The correlation between, on the one hand, word-based mathematics, and, on the other hand, reading comprehension, general language ability and the SWRT support Clarkson’s (1992) claim that linguistic proficiency plays an important role in mathematics attainment.

A final remaining question is how we can explain the differences between the FLE and the EAL groups with respect to their word-based problem skills. A series of hierarchical regression analyses was conducted to investigate which factors explained word-based mathematics scores of the group as a whole, and then the FLE and EAL groups separately. While some measures were not normally distributed, the distribution of residuals was checked and found to be within normal distribution ranges, therefore confirming the appropriateness of using regression analysis. Firstly, a two-stage hierarchical regression analysis was conducted for all learners using the stepwise method. To control for the effect of working memory, this variable was entered at stage 1, which accounted for 16% of the variance observed (R² = .155, F (1, 59) = 10.858, p = .002). At stage 2, reading comprehension score, SWRT score and the language ability score were entered into the model. Language ability and SWRT were excluded in model 2, which comprises working memory and reading comprehension. The final model accounts for 42% of the variance in word-based mathematics scores (R² = .421, F (3, 56) = 26.658, p = < .001). Collinearity was assessed and the statistics fell well within the accepted range. As can be seen from the regression model statistics displayed in Table 4, reading comprehension score is the variable that explains the most variance in word-based mathematics scores for all learners.

Table 4. Regression analysis of word-based mathematics scores – all learners.

	Model 1			Model 2
Variable	B	SE B	β	B	SE B	β
Working memory	.259	.079	.394**	.156	.069	.237*
Reading comprehension				.201	.039	.539***
R^2	.155			.421
R^2 change	.155			.266
F for change in R^2	10.858**			26.658***

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

The same regression analysis was conducted on the groups of FLE and EAL learners separately to explore differences between the two groups of learners. For the FLE group, model 1 (including working memory only) explained 26% of the variance (R² = .255, F(1, 28) = 9.907, p = .004) (see Table 5). The reading comprehension score, the SWRT score and the language ability score were entered into model 2. For the FLE learners, reading comprehension and SWRT were excluded in model 2, which comprises working memory and language ability (C-test) only. The final model accounts for 44% of the variance in word-based scores for FLE learners (R² = .443, F (1, 28) = 9.464, p = .005). When entered in the model alone, language ability accounts for 29% of the variance (R² = .288, F (1, 29) = 11.752, p = .002).

Table 5. Regression analysis of word-based mathematics scores – FLE learners.

	Model 1			Model 2
Variable	B	SE B	β	B	SE B	β
Working memory	.396	.126	.505**	.317	.114	.404**
Language ability				.090	.029	.446**
R^2	.255			.443
R^2 change	.255			.188
F for change in R^2	9.907**			9.464**

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

For the EAL group, again using stepwise hierarchical regression, the final regression model included reading comprehension only since working memory and language ability were excluded. SWRT was not included in the model as collinearity statistics were well above acceptable levels. The model explained 44% of the variance in word-based mathematics scores for EAL learners (R² = .436, F (1, 29) = 22.458, p = < .001) (see Table 6).

Table 6. Regression analysis of word-based mathematics scores – EAL learners.

	Model 1
Variable	B	SE B	β
Reading comprehension	.241	.051	.661
R^2	.436
R^2 change	.436
F for change in R^2	22.458***

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

A significant finding of this study is that general language ability, as measured by the C-test, plays a key role in performance on word-based mathematics assessments for FLE learners. Additionally, C-test scores correlated strongly with SWRT for both groups and with reading comprehension for both groups, but in particular for the EAL group (see Tables 7 and 8). Potential reasons for such high correlations will be taken up in the discussion.

Table 7. Correlations between language ability, reading comprehension and SWRT for FLE learners.

	SWRT	Reading comprehension	Language ability (C-test)
SWRT	*
Reading comprehension	.524**	*
Language ability (C-test)	.696***	.313	*

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

Table 8. Correlations between language ability, reading comprehension and SWRT for EAL learners.

	SWRT	Reading comprehension	Language ability (C-test)
SWRT	*
Reading comprehension	.536**	*
Language ability (C-test)	.750***	.693	*

*Significant at the .05 level 2-tailed **significant at the .01 level 2-tailed ***significant at the .001 level 2-tailed.

Discussion and conclusions

The findings of the current study underline the prominent role language plays in the development and assessment of mathematical ability. Consistent with the findings of previous studies, the results show that FLE learners significantly outperform EAL learners in the word-based component of the mathematics test only. The results of this study also indicate that there are differences in how reading comprehension ability is related to the mathematical word problem solving performance for FLE and EAL learners. Our results are in line with previous research that has shown English reading comprehension ability to be related to mathematical word problem solving performance for EAL learners (e.g. Abedi and Lord 2001). However, contrary to the findings of Vilenius-Tuohimaa, Aunola, and Nurmi (2008), the correlation between reading comprehension and word-based mathematics for FLE learners is not significant, which confirms findings of Imam, Abas-Mastura, and Jamil (2013). The lack of correlation between reading comprehension and word-based mathematics for the FLE learners may be accounted for by the higher level of reading comprehension of these learners who do already possess the linguistic knowledge required for the mathematics test.

A strength of the current study lies with the inclusion of the language ability (C-test) and SWRT alongside a measure of reading comprehension, which allows for a more in-depth investigation of the contribution of different aspects of language and reading ability to the assessment of mathematical knowledge in EAL children. Our results differ from those of Burgoyne et al. (2009) in that the FLE learners in our study outperform the EAL learners on the Single Word Reading Test. Burgoyne et al. (2009) found that EAL learners’ decoding ability was better than that of FLE learners, and argue that vocabulary knowledge rather than decoding is the key factor in reading comprehension ability among EAL learners. The differences between the two studies could be due to the fact that our sample included EAL children who had been in the UK for a relatively short period of time. Burgoyne et al. (2009) comment that efforts to improve reading ability should be focused on comprehension strategies. Research with young language learners (e.g. Samo 2009) has shown that less successful readers with lower levels of language proficiency employ local, bottom-up strategies thereby focusing on smaller units (word or sentence level) when constructing meaning, whereas more proficient and successful learners may employ a wider range of top-down, global strategies for text comprehension. The C-test encourages learners to use a combination of bottom-up, word-based strategies along with top down, text-based strategies. While SWRT and C-test scores are highly correlated for the EAL learners, the regression analysis showed that the C-test scores explained variation in mathematics scores over and above what is explained by word decoding skills. Our finding that general language ability as measured by the C-test correlates significantly with reading comprehension levels of both L1 and L2 students furthers evidence stemming from previous pieces of research in English (Lesaux, Rupp, and Siegel 2007; Babayiğit 2014, 2015) and in Dutch (Droop and Verhoeven 2003). More importantly, it extends our knowledge of how general language ability might affect comprehension in the context of word-based mathematics problem solving for FLE and EAL students. While general language ability scores predicted FLE students’ performance in word-based mathematics problems, it was reading comprehension scores that predicted EAL students’ performance. This finding together with the strong correlation between C-test scores and reading comprehension of EAL students might still support the important role of general language ability for EAL word-based problem solving performance. Taken together the findings from the two groups of children suggest that the relationship between language ability and mathematical word problem solving performance may not follow the exact same trajectory for FLE and EAL learners.

A further interesting finding of the current study was that working memory was more strongly associated with mathematical word problem solving performance in the FLE group than in the EAL group. One possible explanation for this is that the effect of working memory is eclipsed by EAL learners’ reading comprehension levels. Another issue could be that EAL learners found the backwards digit span task very difficult, possibly because the number system is not as automatised in EAL learners as in FLE learners. It was evident that some learners were still not completely familiar with numbers in English, which is a linguistic rather than a numeracy issue. In future research, a non-verbal test of working memory should be used with EAL learners to isolate this from linguistic knowledge.

Within the EAL group of learners, there was a great deal of variation in scores on all measures and also in ‘length of residence’ in the UK. The results of the current study emphasise its importance when accounting for the variation seen between learners as this factor correlated strongly with all outcomes. Some learners had only been resident in the UK for several months, whereas others had lived in England for nearly five years. Those learners who had been resident for five years actually performed on a par with monolingual peers. On the one hand, this is a very positive finding. On the other hand, this highlights questions around the suitability of assessment instruments (both school-based and for research) for learners with less exposure to English and therefore lower proficiency levels. Length of residence is a factor that needs to be considered when assessing progress.

While the current study was situated in the UK and has made references to the UK context, it is our strong belief that our findings are also applicable to other settings. As mentioned, this is particularly relevant when an increasing number of high-stakes mathematics standardised tests around the world place an emphasis on using mathematical word problems to assess students’ mathematical understanding.

The results presented here have immediate implications for both practice and policy. The current study provides empirical evidence of the variability between our FLE and EAL learners in terms of language ability, which accounts for differences both between and within groups in terms of mathematics and literacy scores. This suggests that teachers should focus on those EAL learners with lower level general language ability and also focus on explicit vocabulary learning to improve reading comprehension. This could be done by, for example, making explicit the specific mathematical meaning of lexically ambiguous words (e.g. ‘root’, ‘volume’, etc.) at the beginning of mathematics lessons, to help children develop their CALP.

Additionally, the use of mathematics-specific illustrated storybooks to teach mathematical concepts to young children (e.g. Harper Collin’s Math Start series, Kane Press’s Mouse Math series, etc.) has recently been found to help develop children’s linguistic and mathematical abilities (Purpura et al. 2017), though research exploring the effect of using mathematics-specific illustrated storybooks on the development of linguistic and mathematical abilities of EAL children specifically would be particularly useful. In relation to practitioners, the use of this particular resource should be seriously considered by mathematics teachers not just in England (where the study took place), but elsewhere too.

In addition, the results also indicate that test designers should consider the complexity of the language used in mathematical word problems. Research by Abedi and Lord (2001) showed that simplification of language in mathematics tests benefited lower proficiency learners.

Finally, as the current study has shown that the C-test is potentially a powerful tool to measure generic language ability, future studies should focus on the role of such holistic measures in uncovering the differences between the specific abilities of EAL learners and FLE learners.

Acknowledgements

The research team would like to thank the Year 5 teachers and children in this study. Finally, the research team is grateful to Professor Suzanne Graham and Dr. Holly Joseph (University of Reading) and Dr. Marie Therese Farrugia (University of Malta) for commenting on an earlier draft of the manuscript. Any errors remain solely the research team’s responsibility.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Natthapoj Vincent Trakulphadetkrai http://orcid.org/0000-0002-0706-7991

Louise Courtney http://orcid.org/0000-0002-2075-5617

Additional information

Funding

This work was supported financially by the University of Reading’s Research Endowment Trust Fund (RETF) and the University’s Centre for Literacy & Multilingualism (CeLM).