Cognitive Skills, Math-Related Emotions, and Beliefs Explaining Response to Arithmetic Fluency Intervention

Abstract

We examined the associations of cognitive skills, math-related emotions and beliefs, and gender with responses to an arithmetic fluency intervention. Elementary school children with dysfluent arithmetic skills (N=69) participated in an arithmetic fluency intervention (with and without self-efficacy support) implemented in small groups in schools for 12 weeks. Hierarchical regression models including cognitive skills or math-related emotions and beliefs predicted 21% to 50% of the variation in the intervention response, i.e., improvement in arithmetic fluency. Cognitive skills were associated with the response mainly among boys, whereas math-related emotions and beliefs explained more among girls. Thus, both cognitive and non-cognitive factors and their interaction with gender should be considered when identifying non-responders who may need more individually tailored support at school.

Keywords:

• Arithmetic fluency
• cognitive skills
• gender differences
• math-related emotions and beliefs
• response to intervention

Introduction

INTERVENTION RESEARCH ON cognitive or academic skills has mainly aimed to improve the understanding of intervention efficacy at the mean level. Few studies have examined the individual differences in intervention responses or analyzed the factors contributing to them. Although it is important to acquire knowledge about mean-level effects from interventions, the reportedly large interindividual variation in responses and the sizeable proportion of participants not benefitting from interventions that are shown to be efficient at the group level (Kearns & Fuchs, 2013) necessitate research on the factors affecting responses. Knowledge of child-level attributes affecting intervention responses is essential for developing effective interventions, enhancing children’s learning, and mitigating the negative long-term consequences associated with learning difficulties (e.g., low education, unemployment, problems with mental well-being; Aro et al., 2019; McLaughlin et al., 2014). To gain an understanding of the factors affecting response, we investigated the effects of cognitive skills and math-related emotions and beliefs. Moreover, as previous cross-sectional and longitudinal studies in math have revealed, the predictive variables of skill development and their effects may vary according to gender (e.g., Devine et al., 2012; Harris et al., 2021); thus, possible gender differences in the predictors of the individual response to a math intervention were taken into account.

A few studies have identified child-level characteristics as a way to explain responses to interventions targeted at basic academic skills. These studies have mostly been conducted in reading or reading comprehension contexts (e.g., Cho et al., 2015; Vaughn et al., 2020). Only a few studies have been conducted in math contexts, and because of the heterogeneity of research interest across different math contexts (for fractions, see Fuchs et al., 2014; for arithmetic and word problems, see Powell et al., 2017), knowledge of the factors related to the response to math interventions is even more scarce than that in reading contexts. Although poor arithmetic fluency – that is, deficits in fact retrieval – has been demonstrated to be a central characteristic in math difficulties (Geary, 1993), few studies have focused on the interindividual variation in response to interventions targeting arithmetic fluency (e.g., Powell et al., 2017). In our previous study using the same data (Koponen et al., 2017), we found that children receiving an arithmetic fluency intervention that guided the children toward discovery and use of arithmetical facts that they knew (e.g., 5 + 5) to solve unknown problems (e.g., 6 + 5), that is, a derived fact strategy, demonstrated improvement in arithmetic fluency. The two intervention groups that received similar arithmetic intervention, either with or without self-efficacy (SE) support, did not differ in their intervention gains, yet they outperformed the control groups receiving either the corresponding intervention in a reading context (children with a low reading fluency) or business-as-usual teaching. However, not all children showed improvement; therefore, further research is needed to increase the understanding of the observed variation in intervention response. This study addressed the question of child-level characteristics that were predictive of the responses to this intervention.

In intervention studies concerning math, the focus has been on the predictive importance of the initial math skill level (Fuchs et al., 2019; Kohn et al., 2020; Powell et al., 2017) and on cognitive skills, such as working memory (WM; Fuchs et al., 2014; Powell et al., 2017) or language skills (Powell et al., 2017). However, predictive effects from non-cognitive factors – such as learning-related emotions, self-beliefs, and motivation – rarely have been studied, even though they have been demonstrated as being associated with math skill development (Barroso et al., 2021; Grigg et al., 2018). Similarly, gender differences in predictive factors have not been investigated in intervention studies, even though studies examining the predictors of math skill development have found differences, particularly in math-related emotions and beliefs (e.g., Devine et al., 2012; Huang, 2013; Sorvo et al., 2017). A large amount of research has indicated that boys and girls show little or no difference in actual mathematical performance (e.g., Lindberg et al., 2010). In contrast, there has been consistent evidence of gender differences in math-related beliefs and emotions. Girls tend to rate themselves lower when assessing interest in math (e.g., Sewasew et al., 2018) and in their self-beliefs, including SE (Huang, 2013), of math and express more anxiety about mathematics (Devine et al., 2012; Sorvo et al., 2017). However, there is clearly less empirical evidence on whether the associations between math-related emotions and beliefs and math skills are different among girls and boys. This is especially true among intervention studies, where it remains unknown whether emotional and motivational attributes are differently related to the intervention response depending on gender. In thist study, we adopted a comprehensive approach that included cognitive skills (general and arithmetic-related cognitive skills) and math-related emotions and beliefs (math anxiety, task interest, and SE) as the possible predictors of the response to an intervention, analyzing them separately for girls and boys.

Predictors of development and response to intervention in arithmetic

Early performance in math is a strong predictor of later mathematical achievement, but so far, the rather narrow set of intervention studies explaining the individual variation in responses suggests that initial math skill seems to predict the response to math interventions only vaguely (Fuchs et al., 2019; Kohn et al., 2020; Powell et al., 2017). Powell et al. (2017) found that neither initial arithmetic nor number skills were related to responses to an arithmetic intervention among second-grade children with mathematics difficulties. Similar results have been found among at-risk learners in first grade (Fuchs et al., 2019) or when math training was provided via a computer-based learning program (Kohn et al., 2020). However, some findings have indicated that initial reading skills can contribute to responsiveness to arithmetic interventions (Kohn et al., 2020; Powell et al., 2009). Children with comorbid reading and math difficulties have been shown to benefit less from pure drilling interventions with repeated rehearsal in arithmetic than those children with just a math difficulty (Powell et al., 2009, 2020). These results could be linked to the findings of partially shared cognitive underpinnings in reading and arithmetic (e.g., Koponen et al., 2020) that influence the rapid retrieval of phonological information, such as arithmetical facts, from long-term memory; this may explain why children with reading difficulties often struggle in arithmetic fluency and why pure repeated rehearsal is not an efficient way to support children with difficulties both in arithmetic and reading fluency because training relies on their weakness. Although it seems that training calculation strategies are a more promising approach for children with comorbid fluency problems (Powell et al., 2020), we do not know what kind of role reading fluency plays when predicting the response to an arithmetic fluency intervention with a derived fact strategy approach.

Previous studies have identified domain-general cognitive factors that can predict arithmetic skills, but whether the same factors predict intervention responses in arithmetic remains underexamined. Numerous studies have found that WM predicts the development of arithmetic skills (for a meta-analysis, see Peng et al., 2016), and a growing body of evidence suggests that besides reading fluency, rapid automatized naming (RAN) is a strong predictor of arithmetic fluency (for a meta-analysis, see Koponen et al., 2017). In contrast, the research on these factors’ effects on intervention response is scarce. One exception is Powell et al. (2017), who found that WM contributed to the response to an arithmetic intervention; that is, children with greater WM capacity gained more than those with a lesser WM capacity. Empirical evidence on RAN as a predictive factor in arithmetic fluency intervention surfaced only from one single-case study, indicating that naming-speed deficit was linked to difficulties in acquiring and/or retrieving arithmetical facts (Koponen et al., 2007); however, more evidence is needed.

Similarly, although visuo-spatial skills and vocabulary have been demonstrated to predict arithmetic development (LeFevre et al., 2010; Zhang et al., 2014), there is a lack of studies exploring what kind of roles general cognitive capabilities play in response to an arithmetic intervention. In a study by Kohn et al. (2020), general intelligence was found to correlate with intervention response in arithmetic when using a computer-based learning program as an intervention tool. Still, our understanding of the effects of cognitive skills in response to arithmetic fluency training is limited. On the one hand, more efficient cognitive resources may facilitate the acquisition and implementation of new strategies and abilities. On the other hand, participants with lower cognitive resources can benefit more from cognitive interventions because more room for improvement exists.

Previous research on math-related emotions and beliefs as the predictors of responses to a math intervention is even scarcer than that on cognitive factors. Although attributes, such as SE (e.g., Grigg et al., 2018), math-related anxiety (e.g., Barroso et al., 2021), and task interest (e.g., Grigg et al., 2018) have been found to be related to math achievement, they have not been studied as the predictors of the response to an intervention – except for a recent study by Kohn et al. (2020), who found that initial math anxiety predicted intervention response. Because the cognitive skills seem to explain only small portion of the individual variance in response to intervention targeted at academic skills (e.g., Stuebing et al., 2015), studies examining the predictive effects from emotions and beliefs on responses to intervention are needed. For example, Stuebing et al., (2015) showed that across multiple studies, individual cognitive skills accounted for 2.6–6% of the variance in gain of reading interventions.

Continuous and categorical approaches to examining the response to an intervention

Both continuous and categorical approaches have been used to operationalize response to intervention. For instance, continuous approaches, such as postintervention score ‘normalization’ (e.g., Torgesen et al., 2001) and improvement slope (e.g., Fuchs et al., 2004), have been used to define responses. With categorical approaches, the participants are designated as adequate or inadequate responders based on predefined criteria, for example, using a control group gain score as a cutoff (e.g., Krowka & Fuchs, 2017). The advantage of the continuous approach is that it allows for analyzing or controlling for the effects of covariates. For example, the impact on the intervention effect pertinent to both initial skill level and grade level needs to be considered. By using continuous measures and analyzing linear relationships between the outcome and predictive variables, possible factors related to the intervention responses across the whole distribution can also be examined. However, the categorical approach allows for examining risk characteristics that are prevalent among those who do not respond to the intervention. Detecting the nonresponders’ specific characteristics could help identify children who likely do not benefit from a particular intervention. This knowledge is relevant for teachers and clinicians, helping them tailor individual support, and for researchers, helping them improve and target intervention programs. In this study, we employed both continuous and categorical approaches to study predictors of individual variance in response to arithmetic fluency intervention.

Use of dual criteria to define nonresponders under the categorical approach

Currently, no widely accepted criterion exists for defining an inadequate response to an intervention; thus, diverse methods have been used to set the cutoff. However, using only one criterion may elicit false positive or false negative designations (Fletcher et al., 2014). For example, defining the cutoff point solely based on improvement between pre- and postintervention assessments among typically achieving controls or examining improvement solely within the intervention group (e.g., using the intervention group’s mean or median as a basis for the cutoff point) has its limitations. The first uses a criterion that is difficult for children with learning disabilities to reach, and the latter ignores information related to peer development. Similarly, the use of age- or grade-normed scores as the basis of the cutoff criterion may fail to identify those responders who improved markedly compared with their own initial level but still do not demonstrate progress similar to typically learning peers. Correspondingly, those not reaching age- or grade-equivalent performance levels on the postintervention assessment because of being far behind their age or grade level during the preintervention assessment are not identified as responders, despite their progress. Furthermore, contrasting the improvement of intervention participants to peers with matched initial skill levels allows for controlling for learning outside the intervention among children with similar skill levels. To address these limitations, dual criteria have been recommended (e.g., Barth et al., 2008), such as growth during the intervention and final status (Fletcher et al., 2014). In this study, the dual criteria applied included both absolute (raw score) improvement during the intervention (contrasted with peers with equally low preintervention arithmetic skill levels) and the individual’s change in grade-equivalent standard scores.

This study

In this study, we focused on the effects of cognitive skills and math-related emotions and beliefs on intervention responses to an arithmetic fluency intervention among children in grades 2–5. In our previous studies (Koponen et al., 2017), the arithmetic fluency intervention was found to be effective at the group level. In this study, we employed two approaches to study predictors of individual variance in intervention response more closely. First, we adopted a continuous approach and predicted gain scores with continuous preintervention measures, including initial skill levels in arithmetic and reading, as well as cognitive and non-cognitive attributes as predictors. Second, as a categorical approach, we identified responders and nonresponders based on the dual criteria, examining how the appearance of risk characteristics related to cognitive and math-related emotions and beliefs was associated with the grouping. To the best of our knowledge, no previous studies have targeted response to an arithmetic intervention by utilizing continuous and categorical approaches or targeting both cognitive predictors and math-related emotions and beliefs.

The research questions for the continuous approach were as follows: 1a) Which cognitive skills and math-related emotions and beliefs are associated with intervention gain score after controlling for initial skill level? 1b) Are there different predictive attributes for boys and girls?

The research questions for the categorical approach were as follows: 2a) Are specific risk characteristics related to cognitive skills and math-related emotions and beliefs equally present among responders and nonresponders? 2b) Do boys and girls in the responder and nonresponder groups differ with respect to their cognitive skills and math-related emotions and beliefs?

Methods

Participants

This study was part of a longitudinal study (SELDI, 2013–2015) focusing on elementary school children’s self-beliefs, motivation and reading, and math skills, as well as their interventions. The data were collected from 20 schools in urban and semiurban areas of Central and Eastern Finland. The total sample comprised 1327 children (638 girls and 689 boys) from grades 2 to 5. Written consent was obtained from the guardians of the participants. The research procedure was evaluated by the University of Jyväskylä Ethical Committee. After screening this larger sample using at or below the 20th percentile in arithmetic fluency as a criterion for poor performance, 240 children were chosen to participate in an individual assessment. Ultimately, 69 children (36 girls and 33 boys, M_age = 9.5 years, SD = 0.9 years) were confirmed in an individual assessment as being slow in arithmetic fluency, and their schools had adequate special education resources for conducting the small group intervention. These schools were randomized to have the arithmetic intervention (with or without SE intervention; see below for details) instead of reading intervention. Of the final sample, 20 (29%) participants were second graders, 26 (38%) were third graders, 20 (29%) were fourth graders, and three (4%) were fifth graders. In principle, children from second to fourth grade were included, but a few fifth graders who fulfilled the inclusion criteria were also included based on special education teachers’ specific requests.

Intervention

Two arithmetic fluency intervention groups, with (N = 31) and without (N = 38) SE support, received an identical manualized intervention focusing on a derived fact strategy, but only about half the children received SE support, here comprising elements focusing on enhancement of SE. Special education teachers implemented the intervention in small groups of four to six children twice a week (45 min per session) for 12 weeks. Furthermore, the children played familiar games embedded with arithmetical content twice a week for 15 min (for more detailed information, see Koponen et al., 2017). As part of the derived fact strategy training, the children were instructed to compare arithmetic facts according to magnitude (e.g., 5 + 5 vs. 6 + 5, i.e., 6 + 5 totals one more than 5 + 5), which helped in guiding the children toward discovery and the use of arithmetical facts that they know (e.g., 5 + 5) to solve unknown problems (e.g., 6 + 5) based on conceptual understanding. The teachers were instructed to encourage children to verbalize their thinking and strategies, as well as point out that several strategies can be used and that each child should find the strategies best suited for them. In the SE support intervention, the elements aimed at enhancing math SE explicitly targeted the four sources of SE (i.e., mastery experience, vicarious experience, verbal persuasion, and physiological and emotional states; Bandura, 1997), including several forms of feedback and practices to ensure that each child’s progress was demonstrated explicitly (for a more detailed description, see Koponen et al., 2017, 2020). Math intervention groups with and without SE elements did not differ in their arithmetic fluency development during the intervention (Koponen et al. 2017). Several methods were used to ensure the interventions’ fidelity. First, the teachers were trained in small groups and provided with session-by-session manuals and materials. Second, meetings and phone conversations were arranged to monitor adherence to the intervention protocols. Third, the teachers completed a checklist diary, marking completed intervention sessions and noting any exceptions in the intervention activities or participants’ attendance. The average number of activities completed without exceptions (e.g., ‘did not have enough time’) was 97%.

Measures and procedure

Procedure

This study utilized data from two pre-assessment and one post-assessment. The 12-week-long interventions started at the end of January 2014. A postintervention assessment was conducted right after the intervention ended in April. The group-administered arithmetic fluency task and questionnaires covering SE, math anxiety, and interest in math were presented before and after the intervention by trained research assistants, who gave the children prewritten instructions and read all the questionnaire items out loud one by one to ensure that everyone could answer them, regardless of their reading skills. Text-reading fluency was assessed on an individual basis.

Academic skills

Arithmetic fluency. Arithmetic fluency was assessed using a 2-min paper-and-pencil group test of addition fluency (Koponen & Mononen, 2010, unpublished) comprising 120 randomly ordered sums with addends smaller than 10 (0 was not included). The child was instructed to solve the problems as accurately and quickly as possible and write down their answers. The score was the number of correct answers within the time limit. The better of the two baseline scores was used as the initial arithmetic fluency score, and the gain score for arithmetic fluency was calculated by subtracting the initial arithmetic fluency score from the postintervention score.

Text reading. Reading fluency was assessed using a text-reading task (Salmi et al., 2011) that was administered individually. The child was asked to read an informational 120-word text out loud as accurately and fast as possible. The score was the number of correctly read words within the time limit of 90 s.

Cognitive attributes

Visuo-spatial skills and Vocabulary were assessed using the WISC-IV (Wechsler, 2010) block design and vocabulary subtests.

Working memory was assessed using WISC-IV’s digit span forward and backward subtests.

Rapid automatized naming was assessed using the object subtest of the test of rapid serial naming (Ahonen et al., 1999), in which five familiar objects were arranged in a fixed pseudo-random order, each replicated 10 times. The time taken to name the objects was used as the score.

Math-related emotions and beliefs

Math anxiety was assessed using three items from the Math Anxiety Questionnaire (MAQ; Thomas & Dowker, 2000): ‘How anxious or calm would you be if you were unable to do … something in mathematics/a mental calculation task/math homework?’ The children responded on a 5-point scale with pictures of faces depicting varying options from 1 (very calm) to 5 (very anxious). Cronbach’s alpha was 0.77.

Math interest was assessed using four questions from the Academic Self-Description Questionnaire I (ASDQI; Marsh, 1990): ‘How much do you hate/like mathematics and look forward/enjoy math classes’ The children responded on a 5-point scale ranging from 1 (not true at all) to 5 (true). Cronbach’s alpha was 0.93.

Self-efficacy in learning math was assessed with two questions developed based on the guidelines outlined by Bandura (2006): ‘How certain are you that you can … learn to calculate more fluently/accurately?’ The children rated the strength of their confidence using a 7-point scale ranging from 1 (I’m totally certain I cannot) to 7 (I’m totally certain I can). Spearman’s rho was 0.72.

Criteria for response, no response, and risk

We used dual criteria to classify each child based on their intervention responses into response or no-response groups. The first criterion for inclusion in the response group was that the child’s gain in arithmetic fluency (raw score) had to be higher than the median of their low-performing peers not participating in the intervention. For this purpose, a subgroup of low-performing peers (the lowest 20th percentile) was selected from the total sample, and their median gain during the time corresponding to the intervention was calculated separately for each grade level and used as the value of the ‘expected gain’. The second criterion was based on the change in each child’s standard scores (pre-to-post) based on normative grade-level data; this indicated whether the child had approached or lagged further behind the grade-level mean. Thus, the child was classified as belonging to the response group if a) they improved more than the ‘expected gain’ and b) their performance approached the grade mean. The child belonged to the no-response group if a) they improved less than the ‘expected gain’ and/or b) their standard score did not change from pre-assessment to post-assessment or decreased. To study whether low scores in cognitive skills and math-related emotions and beliefs form risk characteristics for low/no response in the arithmetic fluency intervention, we used a cutoff of 1 standard deviation (SD) compared with the grade-level mean; that is, the child was designated as having a cognitive or math-related emotions and beliefs risk if their age-standardized score in cognitive skills, interest or SE was below or at −1 SD, and the child was designated as having risk for anxiety if the age-standardized score was above or at 1 SD.

Data analyses

We first ran correlation analyses between the gain score and predictive variables, and then, hierarchical regression analyses were conducted to analyze the association between the predictive variables and gain score. The models were built first for the whole group and then separately for boys and girls. The first hierarchical regression analysis was conducted to analyze the predictive power of grade level, gender, intervention group, and initial arithmetic and reading fluency (control variables). To answer the first research questions, two more hierarchical regression models were built: the first targeted associations between cognitive variables and the gain score, and the second targeted associations between math-related emotions and beliefs and the gain score. In both analyses, the initial arithmetic skill level was controlled for when examining the predictive variables’ effects. Because of the small number of participants, separate analyses were conducted for the cognitive skills and math-related emotions and beliefs. Cook’s distances found no significant cases to be multivariate outliers (i.e., all values were < 4/n and did not differ from the general trend in visual analysis), and residual distributions supported the models’ accuracy. To answer the second research question, cross-tab and χ² tests were used to analyze whether certain risk characteristics were distributed equally to the two response groups, and separate multivariate analyses of variance (MANOVA) were performed for cognitive and math-related emotions and beliefs attributes to compare the response groups and genders.

In addition, because the participants were nested within 20 schools and 75 classes, intraclass correlations (ICCs) were calculated to examine the proportion of the variance in arithmetic skills because of school and class. The ICCs of the arithmetic skill variables by the school and classes were all nonsignificant and mainly close to zero (Range ICCarithmetic_pre = 0.002 − 0.067 and ICCarithmetic_post = 0.010 − 0.130), suggesting that the variation in arithmetic skills was mainly within-level variation.

Results

Preliminary analyses

The descriptive statistics are presented in Table 1. No gender differences were detected in the gain score, initial levels of arithmetic and reading fluency or the seven predictive variables covering cognitive and math-related emotions and beliefs. All the correlations between variables are presented for all participants in Table 2, and correlations in gain score and initial arithmetic fluency with the predictive variables are presented separately for girls and boys. Moreover, as an additional analysis, correlations were also conducted separately for two intervention groups (arithmetic skill or skill plus SE) to examine whether differences between the groups existed. Statistically significant and moderate size correlations with the gain score were detected for RAN, with smaller correlations for visuo-spatial skills, WM, and SE among all participants. Among boys, large and significant correlations with the gain score were found for RAN and WM and moderate correlations for visuo-spatial skills. Among girls, moderate and significant correlations were found for RAN and math anxiety. These associations seemed to be related specifically to intervention gain because only RAN was also moderately associated with initial arithmetic level and only among girls. Finally, correlations in both intervention groups mainly reflected the same correlational structure as among all the participants: moderate to small correlations with the gain score were detected for RAN (r = 0.30–0.54), visuo-spatial skills (r = 0.31–0.30), WM (r = 0.23–36), and SE (r = 0.27–0.29). An exception was a negative correlation between gain scores and math anxiety (r = −0.35), which was found only in the arithmetic group. However, scatter plots (Appendix) revealed that the association between the gain score and math anxiety varied mainly between genders. Altogether, considering that the preliminary analyses indicated differences mainly between genders rather than the intervention groups and because of the small sample size, the regression models were run separately only for boys and girls.

Table 1. Descriptive statistics on the study variables.

	Whole sample (N = 69)					Girls (N = 36)		Boys (N = 33)		t-Test
	Min/Max	Mean	SD	Skewness	Kurtosis	Mean	SD	Mean	SD	t (68), (sig.)
Gain score	−5.00/29.00	6.71	7.17	0.80	0.65	6.81	7.16	6.60	7.30	0.12 (ns.)
Initial arithmetic skills	9.00/30.00	19.68	5.26	−0.02	−0.61	20.34	5.30	19.00	5.20	1.05 (ns)
Initial reading skills^a	−2.45/2.27	−0.55	1.03	.046	−0.25	−0.53	1.03	−0.58	1.04	0.13 (ns)
Visuo-spatial skills^a	2.00/17.00	8.65	3.22	0.34	−0.13	8.58	2.71	8.73	3.80	−0.19 (ns.)
Vocabulary^a	1.00/15.00	7.90	2.75	−0.13	−0.02	7.72	2.42	8.10	3.13	−0.55 (ns.)
Working memory^a	3.00/14.00	8.06	2.36	0.31	0.12	8.53	2.21	7.50	2.45	1.79 (ns.)
Rapid automatized naming^b	−3.99/1.27	−0.65	1.14	−0.90	0.59	−0.57	1.06	–0.74	1.23	0.59 (ns.)
Math anxiety^b	−1.78/2.00	0.32	0.95	−0.04	−0.51	0.46	0.93	0.16	0.95	1.32 (ns.)
Math interest^b	−1,91/1.65	−0.24	1.05	0.06	−1.18	−0.27	1.04	−0.21	1.07	−0.21 (ns.)
Math SE^b	−3.50/.97	−0.53	1.05	−0.52	−0.22	−0.54	1.01	−0.52	1.12	−0.06 (ns.)

Note. ^aVariables are standardized (M = 10; SD = 3); ^bVariables are standardized (M = 0; SD = 1); i.e., p > 0.05. Math SE means math self-efficacy.

ns: non-significant

Table 2. Correlations of the study variables.

	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.
1. Gain score	1.00	−0.01/0.05	−0.08/0.25	0.15/0.39*	−0.25/0.26	−0.02/0.57**	0.34*/0.53**	−0.36*/0.21	−0.09/−0.05	0.23/0.32
2. Initial arithmetic skills	0.02	1.00	0.20/–0.07	−0.22/–0.19	0.36*/–0.38	0.32/0.00	0.35*/–0.04	−0.05/–0.04	−0.11/0.10	0.03/0.04
3. Initial reading skills^a	0.08	0.07	1.00
4. Visuo-spatial skills^a	0.28*	−0.20	0.11	1.00
5. Vocabulary^a	0.02	−0.04	0.24	0.18	1.00
6. Working memory^a	0.26*	0.21	0.29*	0.19	0.33**	1.00
7. Rapid automatized naming^b	0.43**	0.16	0.28*	0.10	0.24	0.21	1.00
8. Math anxiety^b	−0.09	−0.03	−0.11	−0.29*	−0.06	−0.26	0.04	1.00
9. Math interest^b	−0.07	−0.02	−0.16	−0.29*	−0.06	−0.26*	0.04	−0.40**	1.00
10. Math SE^b	0.27*	0.04	0.07	−0.20	0.25	−0.08	0.14	−0.25*	0.16	1.00

Note. *p < 0.05, **p < 0.01. On the first and second rows, correlations of intervention gain and initial skill level with predictive variables are presented for girls/boys. ^aVariables are standardized (M = 10; SD = 3); ^bVariables are standardized (M = 0; SD = 1).

ns: non-significant

Continuous predictors of gain scores

The regression analysis with control variables (grade level, gender, intervention group [arithmetic skill or skill plus SE], and initial arithmetic and reading fluency levels) indicated that these variables together predicted only 6.6% of the gain score variance, and none of the steps or predictive variables were significant. Therefore, these variables were not included in the subsequent hierarchical regression analyses, except for the initial arithmetic fluency level. In the cognitive models (Table 3), initial arithmetic fluency was entered during the first step, and block design and vocabulary, here as general cognitive skills, were entered during the second step, while arithmetic-related cognitive variables were entered during the two following steps: WM during the third step and RAN during the fourth step. The block design β was significant during the second step before entering WM into the model, but only the last step with RAN elicited a significant change that explained 18% of the gain score variance. The final model explained 29% (adjusted R² = 23%) of the variance and indicated that more fluent preintervention RAN was associated with better gains in arithmetic fluency during the intervention. As Table 3 shows, the cognitive model for girls explained about 24% (adjusted R² = 10.0%) of the gain score variance, and only the last step with RAN produced a significant change, explaining 13%. The cognitive model for boys explained 50% (adjusted R² = 38%) of the gain score variance, and the third step with the WM and fourth step with RAN were significant, explaining 16% and 13% of the variance, respectively. To sum up, as the correlations suggested, the cognitive models indicated differential effects from the preintervention cognitive variables among boys and girls, with better cognitive skills associated with a better intervention response, particularly among boys.

Table 3. Hierarchical regression models for predicting the gain score with cognitive variables.

	Total (N = 69)		Girls (N = 36)		Boys (N = 33)
	R^2/ΔR^2β	ΔF(df1;df2)	R^2/ΔR^2β	ΔF(df1;df2)	R^2/ΔR^2β	ΔF(df1;df2)
Step 1 Initial skill	0.00/0.00 0.02	0.17 (1;58)	0.00/0.00 −0.01	0.01 (1;32)	0.01/0.01 0.07	.011 (1;24)
Step 2 Initial skill BD/VOC	0.08/0.08 0.08 0.29*/−0.03	2.28 (2;56)	0.11/0.11 0.15 0.21/-0.32	1.83 (2;30)	0.22/0.21 0.26 0.35/0.28	2.98 (2;22)
Step 3 Initial skill BD/VOC WM	0.12/0.04 0.02 0.24/−0.10 0.22	2.47 (1;55)	0.11/0.00 0.15 0.21/−0.32 −0.02	0.01 (1;29)	0.37/0.16 0.14 0.23/0.05 0.47*	5.20 (1;21)*
Step 4 Initial skill BD/VOC WM RAN	0.29/0.18** −0.05 0.18/−0.18 0.19 0.44***	13.48 (1;54)**	0.24/0.13 0.02 0.19/−0.31 0.06 0.39*	4.69 (1;28)*	0.50/0.13 0.14 0.16/−0.04 0.35 0.41*	5.20 (1;20)*

Note. ***p < 0.001; **p < 0.01; *p < 0.05.

BD: WISC-IV Block Design; VOC: WISC-IV Vocabulary; WM: WISC-IV Digit Span; RAN: rapid automatized naming of object

In the models covering math-related emotions and beliefs (Table 4), initial arithmetic fluency was entered during the first step, and block design and vocabulary were entered during the second step. The three predictive variables (anxiety, interest, and SE) were entered separately during the following three steps. The order of the predictors was selected based on how general or specific math context are typically used when measuring these concepts reflecting their theoretical framework. Only the final step with SE elicited a statistically significant change that explained about 10% (p = 0.016) of the gain score variance. In the final model, a block design registered a significant effect (β = 0.33; p = 0.021), which was also close to significant in the previous steps (p = 0.054 and p = 0.080). The final model explained 21% of the variance (adjusted R² = 11%). This analysis indicates that after controlling for initial arithmetic skill level effect and general cognitive skills, higher preintervention SE contributed to a better intervention response.

Table 4. Hierarchical regression models for predicting the gain score with math-related emotions and beliefs.

	Total (N = 69)		Girls (N = 36)		Boys (N = 33)
	R^2/ΔR^2β	ΔF(df1;df2)	R^2/ΔR^2β	ΔF(df1;df2)	R^2/ΔR^2β	ΔF(df1;df2)
Step 1 Initial skill	0.01/0.01 0.07	0.25 (1;56)	0.00/0.00 −0.01	0.01 (1;32)	0.03/0.03 0.18	0.74 (1;22)
Step 2 Initial skill BD/VOC	0.05/0.05 0.10 0.22/−0.04	1.36 (2;54)	0.11/0.11 0.15 0.21/−0.32	1.83 (2;30)	0.19/0.16 0.34 0.23/0.33	1.97 (2;20)
Step 3 Initial skill BD/VOC Anxiety	0.09/0.04 0.09 0.27/−0.07 −0.20	2.15 (1;53)	0.26/0.15 0.14 0.28/-0.33 −0.40	6.06 (1;29)*	0.22/0.03 0.37 0.16/0.39 0.19	0.72 (1;19)
Step 4 Initial skill BD/VOC Anxiety Interest	0.9/0.03 0.09 0.26/−0.07 −0.22 −0.07	.020 (1;52)	0.32/0.05 0.12 0.19/−0.40* −0.46** −0.27	2.15 (1;28)	0.22/0.00 0.37 0.17/0.38 0.20 0.03	0.01 (1;18)
Step 5 Initial skill BD/VOC Anxiety Interest Self-efficacy	0.21/0.11 0.09 0.33*/−0.15 −0.18 −0.12 0.37*	7.20 (1;51)*	0.42/0.10 0.16 0.27/−0.50** −0.42* −0.310.35*	4.78 (1;27)*	0.36/0.14 0.35 0.25/0.29 0.24 −0.06 0.41	3.61 (1;17)

Note.  **p < 0.01; *p < 0.05.

BD: WISC-IV Block Design; VOC: WISC-IV Vocabulary; WM: WISC-IV Digit Span

In the model for girls, the third step with anxiety and the final step with SE were significant, explaining 15% and 10% of the gain score variance, respectively. The final model for girls explained 42% (adjusted R^{2 =} 29%) of the variance, with vocabulary, anxiety, and SE exerting significant effects. Thus, girls with lower anxiety and vocabulary levels and a higher SE level before the intervention benefited more from the intervention. The final model with the math-related emotions and beliefs for boys explained 36% of the variance (adjusted R^{2 =} 13%), and none of the steps was significant. However, the fifth step with SE (β = 0.41) approached significance and explained 14% of the variance (p = 0.074). To sum up, the math-related emotions and beliefs were more predictive among the girls than boys.

Associations between risk characteristics and response group status

Of all the participants, 31% (32% of the girls; 30% of the boys) belonged to the no-response group, and 69% (68% of the girls; 70% of the boys) belonged to the response group. Thus, girls and boys were distributed equally in the two response groups. Cognitive and non-cognitive risks were also distributed equally between genders. Cross-tabs and χ² analyses conducted to study the associations between cognitive risks and response group status indicated an association between response group and risk in RAN (Adj. Res. = 2.8; χ² (1) = 7.88; exact p = 0.007) and in WM (Adj. Res. = 2.1; χ² (1) = 4.45; exact p = 0.052). Other associations were not significant. Of the children with RAN risk, a larger proportion than expected belonged to the no-response group (59%) and less than expected to the response group (41%). However, most of the children (78%) without RAN risk were in the response group, with a minority (22%) in the no-response group, indicating that age-equivalent performance in RAN increased the probability of responding to the intervention. Children with WM risk were distributed more evenly in the no-response and response groups (48% and 52%, respectively), but most (78%) participants with age-equivalent WM performance were in the response group. No significant associations emerged between risk in math-related emotions and beliefs and response status.

Cognitive and non-cognitive attributes across genders and response groups

In the MANOVA of the cognitive skills, a significant response group x gender interaction was found (F[53,4] = 3.11; p = 0.023, η_p² = 0.190), indicating that cognitive skill differences between the two response groups varied according to gender (Figures 1 and 2). The pairwise analyses indicated significant interaction effects for vocabulary (F[56,1] = 9.26; p = 0.004, η_p² = 0.142) and nearly significant interaction effects for block design and WM, with both having the same statistical values (F[56,1] = 3.87; p = 0.054, η_p² = 0.065). Among boys, the no-response group showed weaker performance than the response group in vocabulary, block design, and WM, whereas among girls, the opposite was true in vocabulary, and no clear differences between response groups were found in block design and WM. A significant main effect in the response group was found for RAN (F[56,1] = 5.67; p = 0.021, η_p² = 0.092); that is, RAN was weaker among those in the no-response group, regardless of gender. The MANOVA with math-related emotions and beliefs revealed significant response group x gender interaction effects (F[56,3] = 2.86; p = 0.045, η_p² = 0.133), indicating that the boys’ and girls’ characteristics related to math-related emotions and beliefs in the two response groups differed (Figures 1 and 2). The pairwise analyses indicated significant response group x gender interaction effects for anxiety (F[58,1] = 5.01; p = 0.029, η_p² = 0.080), with the girls in the no-response group registering higher anxiety levels than the girls in the response group, though the opposite was true for boys.

Figure 1. Profiles of the cognitive predictive variables of the response and no-response Groups by gender.

Distribution information of each cognitive predictive variables for non-responding and responding girls and boys covering minimum, median and maximum value as well as outliers.

Figure 2. Profiles of the beliefs and emotions of the response and no-response groups by gender.

Distribution information of math-related emotions and beliefs for non-responding and responding girls and boys covering minimum, median and maximum value as well as outliers.

Discussion

This study aimed to identify the factors that influence low-performing school-age children’s responses to an arithmetic fluency intervention by analyzing the associations between intervention response, cognitive preintervention attributes, math-related emotions and beliefs, and gender. The main finding was that both cognitive skills and math-related emotions and beliefs explained the variance in the intervention response, but that the predictors partly depended on gender. Based on both continuous and categorical approaches, cognitive skills were associated with the intervention response being stronger among the boys, and math-related emotions and beliefs explained more of the response among the girls.

Cognitive skills as predictors of response

Although an arithmetic fluency association has been demonstrated between RAN and WM in developmental studies (e.g., Koponen et al., 2017; Peng et al., 2016), few intervention studies have examined them as predictors of response to arithmetic fluency training. Our continuous approach analyzing the cognitive predictors of the response indicated that after controlling for the effects based on initial arithmetic skill level and general cognitive skills, RAN predicted the intervention gain both among boys and girls and that WM was a significant predictor of intervention gain among boys. Moreover, an interaction was found between gender and response group, indicating that the boys in the no-response group had lower levels in all cognitive skills, including WM and RAN, while among the nonresponder girls, performance was lower only in RAN.

Overall, our findings have shown RAN as a significant predictor of intervention response, corresponding with those results of correlational studies (e.g., Koponen et al., 2017), as well as with findings from a single-case intervention study (Koponen et al., 2007), hence revealing an association between rapid naming deficit and the use of fact-retrieval strategies after an intervention among children with language impairments. In this study, the intervention program that was used aimed to reduce the need for direct memory retrieval by focusing on a limited set of arithmetical facts that must be learned by heart (well-known arithmetical facts). However, direct retrieval is still needed when the child solves an unknown arithmetical problem by deriving the answer based on a known arithmetical fact. Thus, it is understandable that RAN, which requires the ability to fluently access and retrieve phonological information (associated with visual stimuli) from long-term memory, contributed to the intervention response. However, the response group also contained children with RAN deficits (41% of the group), suggesting that naming problems were not fundamental to being able to benefit from this kind of intervention.

Our findings on WM affecting the intervention response correspond with Powell et al. (2017), who found that children with a larger WM capacity gained more in an arithmetical intervention than those with a smaller WM capacity. Thus, in the present intervention, in which the focus was on training in derived fact strategies, better WM capacity contributed to intervention gain through children’s ability to retrieve factual knowledge from long-term memory and store it for a short time while deriving an answer to an unknown addition problem by using a known fact. However, considering that no previous studies have examined gender differences in the predictors of an intervention response, more studies are needed to confirm our finding that WM was a significant predictor only among boys. The categorical approach revealed that age-appropriate WM indicated good possibilities of benefitting from derived fact strategy training because 78% of those with no WM deficit belonged to the response group. Still, half the children (52%) with low WM also benefited from the intervention. Finally, although the effects from general cognitive skills were not the focus of this study, we wanted to control for their effects because an earlier study by Kohn et al. (2020) found a small contribution from general intelligence to intervention responses in arithmetic. In our study, visual skills and vocabulary made only a small contribution to the gains among the whole group. However, as noted above, the group of nonresponder boys registered lower levels in all cognitive skills, including visual and vocabulary skills. However, among the girls, weaker verbal skills were related to better gains in arithmetical fluency. Interestingly, the association between verbal skills and intervention gain became stronger after math-related emotions and beliefs were added to the model, suggesting interactional effects from the predictive attributes. However, among the nonresponder girls, verbal difficulties were not severe, and as a group, their verbal skill level fell just below the expected age level. Among the nonresponder boys, verbal difficulties were more pronounced. Interventions included explicit instruction in small groups and modeling and practicing verbalization of one’s own thinking and strategies, which might be appropriate training only for children with mild problems with verbal skills, perhaps not for those with more severe difficulties in verbal skills. However, considering that no mean-level differences existed between boys and girls in any measured cognitive skill, the gender x response group interaction remain partially unexplained, which means that more studies are needed.

Initial arithmetic and reading skills as predictors of response

Neither initial arithmetic nor reading skill level predicted the response to the arithmetic fluency intervention. The finding concerning the initial skill level in math corresponds with prior math intervention studies (Fuchs et al., 2019; Kohn et al., 2020; Powell et al., 2017). In this study, the participants were selected based on whether they showed dysfluency in a group-administered arithmetic fluency task and during an individual assessment in which fact-retrieval difficulties were further confirmed. Therefore, all participants had equal room for development. Moreover, the intervention was targeted specifically at their skill levels by providing arithmetic fluency training that focused on effective calculation strategies to compensate for their difficulties in direct fact retrieval. Our findings concerning initial reading level as a nonsignificant predictor of response to arithmetic intervention corresponded with previous intervention studies finding that predictive effects from initial reading skills vary based on the intervention’s target (for a review, see Powell et al., 2020). Those interventions that focused on teaching calculation strategies similar to those practiced in this study were found to be similarly effective for children with comorbid reading and math difficulties, as well as for those with single math difficulty (Powell et al., 2020). However, children with comorbid reading and math difficulties were found to benefit less from pure drilling interventions with repeated rehearsal than children with math difficulty alone (Powell et al., 2020). Together, these findings suggest that the response to derived fact strategy training does not depend on the level of reading fluency, and derived fact strategy training can be beneficial, despite having comorbid reading difficulties.

Math-related emotions and beliefs as predictors of response

Our findings concerning the math-related emotions and beliefs as predictors of response to an arithmetic fluency intervention highlighted preintervention SE’s relevance, showing it as being related to one’s capability to learn to calculate more fluently and accurately. Among girls, lower anxiety levels were related to better intervention outcomes, as discussed below. Children with high SE are assumed to exert more effort and demonstrate more persistence with learning (Bandura, 1997); thus, our findings’ indication that children with high SE in math education benefit more from arithmetic skills training than those with low SE seems reasonable. In our previous study (Koponen et al., 2020) using the same data, children with initially high or increasing math SE improved more during the arithmetic fluency intervention when compared with children with low math SE. The math SE scale used in that study assessed beliefs about the current capability to use arithmetic skills in task situations and daily activities, as well as the ability to learn to calculate more effectively. This study’s findings indicate that a very short scale with a few items to assess a child’s beliefs about whether they can learn to calculate more fluently and accurately can predict intervention response. Taken together, the evidence suggests that SE may play a pivotal role in math learning and should be considered in future interventions, as well as in daily practice at schools.

Gender differences in response predictors

The differences between girls and boys emerged in the cognitive response predictors and math-related emotions and beliefs. Although both predictive attributes made significant contributions to the continuous gain score at the group level, separate analyses for girls and boys indicated that an association with non-cognitive predictors was found mainly among girls (except for SE), whereas cognitive predictors were found to exert significant effects mainly among boys (except for vocabulary in analyses of math-related emotions and beliefs in girls). Furthermore, a comparison of cognitive skills between the response groups indicated that the boys in the no-response group had lower levels of cognitive skills than the boys in the response group. However, the cognitive profile did not distinguish among the girls in the two response groups.

Although previous studies investigating the response to arithmetic fluency interventions have not considered gender differences in cognitive skills and math-related emotions and beliefs, gender-related differences have been found in the use of arithmetic strategies (Carr & Alexeev, 2011) and in the predictors of skill development and their predictive effects (e.g., Devine et al., 2012; Harris et al., 2021). Research has found gender differences in calculation strategy use (e.g., Carr & Alexeev, 2011): boys are more eager to use fact retrieval and mental calculation strategies in arithmetic processing than girls, who rely more often on concrete manipulatives, such as their fingers. Using mental calculation strategies requires more WM capacity than relying on manipulatives, and perhaps among boys, better WM enabled them to benefit more from derived fact strategy training, which was the focus of the intervention program. However, this hypothesis needs to be verified empirically. It should also be emphasized that WM deficits were not more common among boys than girls, nor were there significant mean-level differences between the genders.

Our gender-related results indicate that lower anxiety before the intervention predicted better intervention responses among girls but not among boys. Kohn et al. (2020) found that initial math anxiety was a significant predictor of an individual’s response to math training, but they did not analyze gender-related differences. Although our data do not allow for answering the question of why math anxiety was related to intervention response among girls and not among boys, different instructional elements of the intervention program could be considered in light of existing math anxiety literature and found gender differences. First, being placed under time constraints and answering questions/explaining solutions in front of others can trigger or increase feelings of math anxiety (e.g., Aschraft, 2002). In a previous study by Van Mier et al. (2018), math anxiety was negatively related to timed math performance, but only among girls, despite similar math performance and math anxiety scores. However, considering that the same time-limited arithmetic fluency task was used in our study in both the pre- and post-assessment and that the girls’ preintervention anxiety correlated only with intervention gain, the association that was found cannot be attributed merely to time constraints. Although the intervention was implemented in small groups comprising same-age peers with similar skill levels, it is not guaranteed that all children will experience the learning environment the same way and feel safe about expressing their mathematical thinking and strategies without the fear of failing or others reacting to them negatively. Corresponding to our suggestions and results, girls have been found to report more anxiety than boys regarding answering teachers’ questions in math class (Sorvo et al., 2017), and this should be considered in future intervention research, as well as when providing support at schools. Moreover, game-like training can be a stimulating and motivating form of practice for some children, while for others, emotional arousal during game situations might be viewed as more limiting. In the future, situational designs that measure these attributes and individual experiences and emotions during the intervention sessions across different activities are needed to clarify the mechanisms underlying the relationship between math anxiety and intervention gain to identify possible debilitating effects.

Practical implications and suggestions for future research

Our findings suggest that low performance on RAN and/or WM tasks, as well as difficulties in arithmetic fluency, are the risk factors associated with a low response to an arithmetic fluency intervention, hence requiring close monitoring of response to support, with a possible need for individual support customization. Future studies should examine whether more individually tailored instruction that can confirm a child’s awareness of different strategies and enhance the ability to recognize problems that are easy for them could compensate for certain risk factors, for example, low WM capacity. Discovering more efficient strategies built on the children’s own knowledge base can help decrease the demands on WM and compensate for difficulties in the direct retrieval of arithmetical facts. Besides cognitive risk factors, low SE and high anxiety, particularly among girls, might also contribute to the response to a support and, thus, should be assessed before and during the support. Furthermore, a supportive adult should attend to the child’s emotions during the observation period. Possible gender-related differences in self-reporting regarding emotions and beliefs should also be examined in future studies to avoid underestimating the importance of math-related emotions and beliefs among boys.

Limitations

This study’s limitations should also be examined. First, even though the pool of participants was relatively large for school-based intervention research, it was still too small to allow for comprehensive analyses. For instance, the cognitive and non-cognitive factors could not be included in the same hierarchical regression analyses, which was also the case for the interaction effect from gender and predictive variables. Larger sample size would allow to examine multiple moderating factors in the same model and control possible confounding factors such as role of processing speed explaining the relation between RAN and intervention gain in arithmetic fluency task, which both are time limited. Second, although the two intervention groups received the same strategy training, it is possible that SE support, which was provided to only half the participants, could have moderated the effect from some of the predictive factors – particularly non-cognitive ones. Although the two intervention groups’ correlation structures were generally similar, there could have been gender and intervention group interaction moderating the relationship between math anxiety and intervention gain. However, we were unable to test this statistically. Thus, in the future, it would be interesting to conduct another study using a larger sample size to explore whether we can find interactions between gender and intervention group when emotional and motivational factors are integrated with arithmetic skills training and contrasted with pure skills training. Third, our focus was on individual-level predictors, so we could not include elements related to the group (e.g., instructional climate, amount of peer support) or teacher, as well as child–teacher interaction, which can affect responses, thereby warranting further research.

Conclusion

Our findings indicate that a child’s cognitive skills, emotions, and beliefs influence their response to an arithmetic fluency intervention. The findings suggest that children struggling with arithmetic and who have low SE or a deficit in rapid naming may need more individually tailored support. Furthermore, the gender differences found suggest that particularly among girls, practices that reduce math anxiety may be needed, whereas among boys, the procedures that help compensate for WM deficits should be incorporated into interventions. The findings underlie the importance of integrating emotional and motivational factors into pedagogical and clinical interventions, as well as adjusting the program according to the participants’ cognitive and non-cognitive characteristics. In the future, situational designs that measure individual beliefs and emotions during intervention sessions using both self-reporting and autonomic nervous system activity are needed to clarify the characteristics and mechanisms that explain the responses to an intervention.

Additional information

Funding

This work was supported by Academy of Finland.

Appendix

Figure A1. Scatterplots of relation between intervention gain and math anxiety presented separately for girls and boys and two intervention groups.