Impact of home-based cognitive or academic intervention on working memory and mathematics outcomes in pediatric brain tumor survivors: the Keys to Succeed pilot randomized controlled clinical trial

ABSTRACT

Pediatric brain tumour survivors experience deficits in mathematics and working memory. An open question is whether it is most optimal to target direct cognitive skills (i.e. working memory) or focus on specific academic outcomes (i.e. mathematics) for in remediation. We conducted a pilot randomized controlled trial to determine the feasibility of comparing a working memory versus mathematics intervention. Pediatric brain tumor survivors (7-17 years) were randomly assigned to Cogmed or JumpMath interventions, or an Active Control/Reading group. All participants received Educational Liaison support and completed ~12-weeks of home-based intervention with weekly, telephone-based consultation in one of the three conditions. Standardized assessments of auditory and visual working memory, mathematics calculation and reasoning were completed pre- and post- intervention. Twenty-nine participants completed the interventions; 94% of parents reported a high degree of satisfaction with the interventions and ease of implementation. Participants in JumpMath demonstrated improved mathematics calculation from pre- to post- intervention (p=0.02). Further, participants in both Cogmed and JumpMath showed evidence of pre- to post- intervention improvements in auditory working memory relative to controls (p=0.01). The Cogmed group also showed improvements in visual working memory (p=0.03). Findings suggest that targeted intervention is feasible in survivors of pediatric brain tumors, though with a relatively low recruitment rate. With preliminary findings of improved calculation and working memory following JumpMath and working memory following Cogmed, this pilot trial lays the groundwork for future programs that investigate different inteCognitiveRehabilitationrventions that may be applied to target the unique needs of each survivor.

KEYWORDS:

• mathematics
• pediatric brain tumor
• Cogmed
• JumpMath

Data availability statement

Requests for resources from the pilot clinical trial, including a copy of the trial protocol and aggregate data should be directed to and will be fulfilled by senior corresponding author, Donald J. Mabbott. Data from individual participants in the pilot trial is not available due to privacy and confidentiality; participants did not explicitly consent for their data to be shared.

Results

Participant flow

The trial was conducted from November 2015 to May 2018. Based on the eligibility criteria, we identified 94 patients that were eligible for the trial. Twenty-nine participants provided informed consent (or parental consent with participant assent, where applicable) and were enrolled (Figure 1 - Consort Diagram). We did not receive any reply to mail out letters sent to 47 eligible participants. Of the 15 eligible participants whose parents declined to have them participant, the majority simply did not return telephone calls after an initial response. Of those who provided an explicit reason for declining, 40% noted issues specific to their child’s condition (i.e., medical issues, fatigue, anxiety), 13% noted family factors (i.e., too busy; family stress), and a single parent noted prior experience with one of the interventions. Although a pre-specified sample of 45 participants was not reached, enrollment was discontinued as we reached saturation such that all eligible participants were identified and contacted within the sample and the trial was not prolonged to minimize any potential cohort effects and due to funding constraints. For three group analysis, 29 participants remain sufficient to detect a moderate effect size, with an alpha=.05, power=.80 (Guilford & Benjamin, 1978), but the effect size is larger than compared to a sample of 45 participants (i.e., .61 versus .48). Participant characteristics are shown in Table 1.

Table 1. Sample characteristics.

There were no differences between the three groups by sex (χ²(2) = .50, p > .05), history of mutism (χ²(2) = .50, p > .05), NPS score (F(2, 25) = .15, p > .05), or time since diagnosis (F(2,25) = .12, p > .05). Age at diagnosis (F(2,25) = 4.19, p = .03) and age at assessment (F(2,25) = 5.47, p = .01) differed between intervention groups: participants in the Cogmed group were older at diagnosis (mean age = 7.87 years) and assessment (mean age = 13.71 years) compared to the JumpMath (mean age at diagnosis = 4.49 years; mean age at assessment = 10.56 years) and Active Control/Reading group (mean age at diagnosis = 4.70 years; mean age at assessment = 10.28 years).

Baseline relations between working memory and mathematics: digits backward predicts mathematics ability over and above intellectual function

At baseline, mean standard scores for estimated FSIQ were within the average range (eFSIQ Standard score = 97.60) across the whole sample, as were the scaled scores for all working memory tasks (Digit Span backward = 9.32; Letter Number Sequencing = 10.0; Verbal working memory = 8.95; Symbolic Working Memory = 9.53). Standard scores on mathematics achievement were lower; while performance was in the average range for mathematics reasoning skills (Applied Problems = 90.0; Quantitative Concepts = 93.0), it fell in the low average range for mathematics calculation skills (Calculation = 83.0; Mathematics Fluency = 80.67). At baseline, increased performance on the digit span backward predicted increased mathematics achievement across the sample (Figure 2, Panels A-D). Table 2 includes the full models and standardized beta coefficients for the variables included for regression analyses. Digit Span backward and eFSIQ predicted Mathematics Calculation (R² = .77, p = <.001), Mathematics Fluency (R² = .83, p = <.001), and Quantitative Concepts (R² = .79, p = <.001), with digits backward accounting for a greater amount of variance. Digit Span backward and eFSIQ also predicted Applied Problems, but Letter Number Sequencing also contributed to the model (R² = .86, p = <.001). Verbal working memory and Symbolic working memory did not contribute to any of the models.

Figure 2. Scatterplots and linear trend line showing the relations between backward digit span scaled scores and mathematics achievement standard scores at baseline for the entire sample for: Calculation (Panel A), Math Fluency (Panel B), Applied Problems (Panel C), and Quantitative Concepts (Panel D).

Table 2. Stepwise regression models relating working memory to mathematics ability at baseline.

Pilot trial: adherence and feasibility

The recruitment rate for the trial was 31%. In terms of adherence, 25 of the 29 participants completed 40 sessions. Three participants completed less than 40 sessions − 30, 33, and 39 sessions, respectively. A single participant completed 50% of the sessions but then had to withdraw from the trial due tumor relapse. There was no difference between the intervention conditions in terms of the number of sessions completed (JumpMath = 40.11; Cogmed = 38.90; Active Control/Reading = 40.33; F (2, 25) = .65, p > .10). Participants in the Cogmed intervention required more weeks to complete the same number of sessions as the JumpMath and control interventions (JumpMath = 10.44; Cogmed = 14.80; Active Control/Reading = 10.67; F (2, 25) = 6.65, p = .005). Of the total sample who completed the trial (n = 28), 20 (71.43%) participants completed the intervention within the 12-week period. The remaining eight (28.57% of the total sample; M/F = 6/2; Mean Age at Baseline = 13.86; Mean Age at Diagnosis = 7.68; Mean Time Since Diagnosis = 5.60; Tumor Site = 5 posterior fossa/2 suprasellar/1 cerebral hemisphere; Tumor Type = 3 medulloblastoma/2 craniopharyngioma/1 meningioma/2 ependymoma; Radiation Type = 4 cranial spinal/4 focal; Mean NPS = 6.25; Mutism = 3) participants completed the intervention within 13 to 25 weeks. Most of these participants (7 out of 8) were in the Cogmed condition – as noted above. All participants were included in the subsequent analyses, however. Reasons cited for taking additional time to complete the intervention included school commitments, vacation, and illness. Following consent to participate in the study, the retention rate from baseline to post-intervention was 96.4%, with one participant withdrawing due to relapse. Finally, the frequency of Educational Liaison support contacts that participants received during the intervention did not differ between the conditions (JumpMath = 3.11; Cogmed = 3.20; Active Control/Reading = 3.89; F (2, 25) = 1.65, p > .10).

In terms of satisfaction, all parents reported either agreeing or strongly agreeing that they felt comfortable supporting their child using the intervention, calling staff with questions about the intervention. Further, all the parents also reported either agreeing or strongly agreeing that the interventions were easy to follow. Most parents reported either agreeing or strongly agreeing that their child benefited from the intervention – thought 2 parents in the JumpMath condition reported somewhat agreeing that their child benefitted. Finally, all parents reported either agreeing or strongly agreeing that they would recommend the study to other families and they found the recommendations provided by the Educational Liaison beneficial.

Pilot trial: mathematics and working memory outcomes

Mathematics calculation and reasoning

A significant multivariate main effect of group was observed across the Mathematics Calculation difference scores (Calculation and Mathematics Fluency subtests) from baseline to post-intervention (Wilk’s Λ = .63, F (4, 42) = 2.77, p = .04; see Figure 3, Panels A and B for difference scores). A significant univariate effect was observed between intervention groups on the Calculation (F (2,22) = 5.10, p = .02; η_p² = .32), but not the Mathematics Fluency (F (2,22) = 1.43, p = .26, η_p² = .12) subtest. Pairwise comparisons revealed that JumpMath yielded increased Standard Scores on the Calculation subtest compared to the Active Control/Reading (p = .02): the JumpMath group improved by a mean of 8.20 Standard Score points while the control group declined by 2.94 Standard Score points (mean difference = 11.14). A single participant displayed reliable but not clinically significant improvement (Table 3). The Cogmed group did not differ from the Active Control/Reading group. There was no multivariate effect across Mathematics Reasoning (Applied Problems and Quantitative Concepts subtests) difference scores from baseline to post-intervention (Wilk’s Λ = .84, F (4, 42) = .471, p = .08; Figure 4, Panels C and D). The pattern of findings did not change when age at diagnosis and age at assessment were included in the models as covariates.

Figure 3. Difference scores Calculation and Math Fluency tasks. Boxplots show difference scores used in the MANOVAs examining group differences from: baseline to post intervention for Calculation (Panel A), pre to post intervention including the Active Control – Reading participants subsequently assigned to Cogmed and JumpMath for Calculation (Panel B), baseline to post intervention for Math Fluency (Panel C), and pre to post intervention including the Active Control - Reading participants subsequently assigned to Cogmed and JumpMath for Math Fluency (Panel D). True/unadjusted mean (solid black diamond) and median (solid line) is shown for each group. Cogmed is shown in red, JumpMath in Blue, and Active Control - Reading in Green. the upper and lower limits of the boxplots are the third and first quartiles (75th and 25th percentile), respectively. the whiskers extend up to 1.5 times the interquartile range from the top (bottom) of the box to the furthest datum within that distance: Difference scores for each participant are shown as solid black circles.

Figure 4. Difference scores from baseline to post intervention for working memory tasks. Boxplots show difference scores used in the MANOVA examining group differences between Cogmed, JumpMath, and Active Control - Reading for: Digit Span Backward (Panel A), Letter Number Sequencing (Panel B), Verbal Working Memory (Panel C), and Symbolic Working Memory (Panel D). True/unadjusted mean (solid black diamond) and median (solid line) is shown for each group. Cogmed is shown in red, JumpMath in Blue, and Active Control - Reading in Green. the upper and lower limits of the box plots are the third and first quartiles (75th and 25th percentile), respectively. the whiskers extend up to 1.5 times the interquartile range from the top (bottom) of the box to the furthest datum within that distance: Difference scores for each participant are shown as solid black circles.

Table 3. Percentage of patients classified into each clinical significance category based on standard scores from baseline to post-treatment based on the Jacobson & Truax method.

Working memory

A significant multivariate main effect of group was observed across working memory difference scores from baseline to post-intervention (Wilk’s Λ = .33, F (8, 30) = 2.77, p = .02; see Figure 4, Panels A-D for difference scores). In terms of univariate effects, a significant effect was observed between the intervention groups on the Digit Span backwards (F (2, 18) = 8.59, p = <.001; η_p² = .49) and Symbolic Working Memory (F(2, 18) = 4.40, p = .03, η_p² = .33) tasks. There were no differences between groups on the Letter-Number Sequencing or the Verbal Working Memory subtests (Fs (2, 18) < 1.57, ps > .10; η_p²s >.15).

Pairwise comparisons revealed that Cogmed training resulted in increased scaled scores on the Digit Span backwards task compared to the Active Control/Reading group (p = .01): The Cogmed group improved by a mean of 1.49 scaled score points and control group declined by a mean of 2.05 scaled score points (mean difference = 3.54). Forty percent of participants in the Cogmed group displayed clinically significant improvement on the Digit Span task (Table 3). There was also a significant difference between the Cogmed and Active Control/Reading group on the Symbolic Working Memory subtest (p = .03). The Cogmed group improved by a mean of 1.99 scaled score points and the control group declined by .90 scaled score points (mean difference = 2.88) – though clinically significant improvement was not observed for any individual in the Cogmed group (Table 3). Notably, JumpMath also yielded a significant increase on Digit Span backwards scaled scores compared to the Active Control/Reading group (p = .01). The JumpMath group improved by a mean of 1.10 scaled score points – though no individual displayed clinically significant improvement (Table 3). We also implemented the multivariate model with age at assessment and age at diagnosis included as covariates as these variables differed between the groups. Both the Cogmed and JumpMath groups displayed significant increases on the Digit Span backwards task relative to the Active Control/Reading group (ps < .05), but no increase was now evident for Symbolic Working memory (p > .05).

Discussion

This trial is unique in that it is the first to compare a program of rehabilitation that examines targeted cognitive intervention versus academic remediation on working memory and mathematics skills in survivors of pediatric brain tumors. The use of a home-based program for mathematics skill remediation – JumpMath – in children and adolescents with an acquired brain injury from the curative treatment of a brain tumor is particularly novel. While this was a pilot trial, our findings demonstrated that it is feasible to deliver targeted intervention in the context of Educational Liaison once participants are enrolled in the program (i.e., 86% adherence compared to 30% reported in the literature). We note that while our recruitment was consistent with other clinical trial focused on cognitive recovery conducted in our hospital, it was much lower than recruitment rates reported for solely Cogmed trials reported in the literature. We discussed strategies for improving recruitment below. Notably, the trial provides preliminary evidence for improved mathematics and working memory outcomes following specific cognitive and academic rehabilitation programs. This is hopeful given the typical declining trajectory of cognitive function observed in pediatric brain tumor survivors treated with radiation (Decker et al., 2017; Scantlebury et al., 2016).

In a sample with average intellectual abilities and working memory skills but with evidence of mathematics deficits, we observed at baseline that auditory working memory (Digit Span backwards) predicts mathematics achievement, even after accounting for estimated IQ. These findings are consistent with the known relations between working memory and mathematics in children with mathematics difficulties (Mabbott & Bisanz, 2008). This data, which supports the expected relations between working memory and mathematics in children and adolescents treated for a brain tumor, provides a sound rationale for focusing on these areas in cognitive and academic remediation within this medical population.

Further, we present evidence that a clinical trial of targeted cognitive or academic intervention is feasible in survivors of pediatric brain tumors, though the recruitment rate was relatively low, and some challenges were evident for specific interventions. Our findings build on prior literature that has demonstrated that Cogmed has some feasibility as a remote intervention (Cox et al., 2015). Unlike interventions for children treated for ALL, however, feasibility of Cogmed completion with pediatric brain tumor survivors continues to be an issue across enrollment and additional weeks required for treatment completion (Hocking et al., 2019). Consistent with this our recruitment rate (31%) was similar to intensive intervention trials in children – such as exercise interventions (Chaddock-Heyman et al., 2013; Davis et al., 2011; Krafft et al., 2014) - and was lower than other remote trials of solely Cogmed in a more heterogenous group of children treated for cancer (Cox et al., 2015).

Parents who did not participate in the present trial noted time demands, fatigue, and family stress as reasons for non participation. Future trials should focus on strategies to increase recruitment rates to such a program that involves multiple interventions. These may include addressing the perceived challenges of participation with families during the recruitment materials and making it easier for families to follow-up on initial queries using social media and use of virtual appointments. We did observe 100% adherence to the intervention with most participants (71%) completing the intervention within the anticipated 12-week time frame. Though, consistent with previous work in children with brain tumors (Hocking et al., 2019), the time to complete the Cogmed program was longer than the other interventions. We observed a high degree of satisfaction among parents whose children participated in this study, with all parents reporting ease of intervention implementation and that they would recommend these interventions to other families.

We also observed that home-based intervention showed evidence of improved mathematics and working memory outcomes compared to active controls – even in the context of Educational Liaison support for all conditions. We observed that JumpMath remediation appeared to yield improved mathematics calculation and working memory skills relative to the active controls – although clinically significant improvement for specific individuals was not evident. Such findings provide preliminary evidence that home-based mathematics remediation may generalize to improvements in a specific cognitive function, such as working memory. Notably, our findings are consistent with those observed in a population of children treated for Acute Lymphoblastic Leukemia who showed improved calculation, applied mathematics, and visual working memory following an in-person clinic based mathematics intervention (Moore et al., 2012). Our findings provide preliminary support for home-based JumpMath as a potentially effective intervention for improving mathematics outcomes in children and adolescents treated for brain tumors.

We saw improved performance for two measures of working memory – the Digit Span backward and Symbolic working memory subtests -following Cogmed training compared to active controls. Although this improved performance was limited to 1–2 scaled score points, the control group demonstrated decline in scaled score points. Furthermore, clinically significant improvement was evident in 40% of participants on the Digit Span backward task, and 22% of participants displayed reliable improvement on the Symbolic working memory subtest. It is notable that we observed findings consistent with any improved outcomes, particularly in light of typical trajectory observed in pediatric brain tumor survivors treated with radiation of poorer working memory over time (Kahalley et al., 2020). Intervention may mitigate against declines in scaled score performance. Our findings add to the previous research that has demonstrated some working memory benefits following Cogmed training (Conklin et al., 2015; Hardy et al., 2013). While not examined in our study, other work has documented longer-term maintenance of original gains in working memory and processing speed (Mendoza et al., 2019). Conklin et al. (2015) noted that while Cogmed has been associated with improvements in reading comprehension and mathematical ability in non-oncology samples, it is important to assess its impact on academic achievement in childhood cancer survivors (Conklin et al., 2015). It is notable that we did not observe Cogmed-related training effects for mathematics outcomes, providing potential evidence that targeted working memory training may not generalize to this mathematics in this population. The working memory training in Cogmed may have not generalized to the numerical processing required for mathematics calculation and problem-solving. Indeed, other studies have also found that training-related improvements for Cogmed are limited to working memory and do not generalize to other cognitive domains such as mathematics (Dunning et al., 2013; Hardy et al., 2013; Melby-Lervag & Hulme, 2013; Sprenger et al., 2013). Taken together with our findings, Cogmed may be most effectively applied to target specific cognitive abilities, rather than academics more broadly in children and adolescents treated for a brain tumor.

Our findings should be considered in light of some limitations. First, as is the case for many interventions focused on cognitive remediation,(Cramer et al., 2011) we acknowledge that our sample size was relatively small. We also note that despite randomized allocation, participants in the Cogmed condition were older than those in the JumpMath and Active Reading conditions. Future, larger scale studies are needed to evaluate the relative impact of Cogmed versus JumpMath with pre-specified primary and secondary outcomes. The pilot nature of the trial limited our ability to examine different demographic and medical variables that may predict better responsiveness to the Cogmed and JumpMath interventions. In a larger efficacy trial, it will be important to consider such effects. For example, studies suggest that females treated for brain tumors are particularly vulnerable to cognitive late effects in childhood (Krull et al., 2018). Performance bias is an issue in open label trials. In this trial the Educational Liaison was aware of group status. While it is possible the nature of support provided by the Liaison may have been influenced by group status, we note that the frequency of support did not differ between groups. Further, as this was a feasibility trial, we only report on short-term outcomes. Future studies should include time points beyond 12-weeks to examine the long-term maintenance of improvements in cognitive and academic functioning across time. While beyond the scope of the current study, it may be beneficial to investigate whether improvements in working memory and mathematics in the research environment translate to better performance at school. This is particularly relevant given higher rates of special education services or grade retention in pediatric oncology survivors (Barrera et al., 2005). Similarly, this study was limited to performance-based, standardized measures to evaluate improvements in working memory and mathematics. The inclusion further academic or vocational attainment would be important to evaluate the real-world indicators of functional benefits of these interventions. Limitations directly related to the specific interventions should also be noted. First, parent involvement was greater in the JumpMath program as compared to the Cogmed intervention. This could have impacted outcomes both positively (with youth receiving additional support in acquiring these skills) or negatively (in terms of increased family burden). The findings should also be considered in light of the comparability of adherence monitoring across interventions. As compared to the Cogmed intervention, which utilized an automatic, electronic recording system, activity logs were used for monitoring adherence to the JumpMath intervention. This approach to monitoring adherence may be less reliable than built-in adherence monitoring and may have resulted in inaccurate recording of intervention. Finally, we note that despite randomization, the participants in the Cogmed intervention were older the comparison groups. The older age of this group may have influenced the improvements seen following Cogmed intervention.

Despite well-documented knowledge of the cognitive late effects that emerge in pediatric neuro-oncology survivors and its negative impact on academic performance, there are limited empirically-supported interventions for this growing population of pediatric cancer survivors. This study provides preliminary evidence for the feasibility of an intervention program that provides Educational Liaison support and home-based interventions targeting specific cognitive and academic skills for improving mathematics. While these findings cannot translate to other cognitive intervention or academic remediation programs, these findings lay the groundwork for future larger efficacy trials comparing Cogmed versus JumpMath, including both the immediate and sustained benefits of these interventions. With the caveat that this was a small open label pilot trial, our findings provide hope that considering the dearth of interventions available to address cognitive and academic late effects in survivors of pediatric brain tumors, structured remediation may be effective in addressing these late effects. This work raises the intriguing possibility that in future work, different interventions may be applied to target the unique needs of each survivor, including the possibility of combining Cogmed and JumpMath in a multi-modal approach for addressing late effects.

Acknowledgements

The authors thank the patients and their families who volunteered their time to participate in this study.

Disclosure statement

No potential conflict of interest wars reported by the author(s).

Additional information

Funding

Funding was provided by the Paediatric Oncology Group of Ontario (POGO) and the Kavelman-Fonn Foundation.