Cognitive training for children with ADHD: composite cognitive score outcome in a randomized controlled trial

Abstract

Purpose of the article

Cognitive training for Attention Deficit/Hyperactivity Disorder (ADHD) has shown promising, although mixed results. In post-hoc analyses, we evaluate effects of cognitive training using a novel composite cognition score as the outcome for children attending at least 16 sessions of training, dose-response of training and associations between symptoms and cognitive functioning.

Materials and methods

Children (age 6–13) with ADHD were randomized to intervention (n = 26) or control (n = 34). For the current analysis, we restricted the intervention group to children, who completed at least 16 sessions of cognitive training (n = 26) and examined a dose response within that group.

Results

Cognition improved significantly in the intervention, but not control group. Amount of the completed training sessions correlated significantly with the amount of cognitive improvement.

Conclusion

Variations in dose and frequency of training may be an important source of the variance in previous studies.

Keywords:

• Cognition
• remediation
• non-pharmacological treatment
• composite score
• ADHD

1. Introduction

Attention Deficit/Hyperactivity Disorder (ADHD) is a common neurodevelopmental disorder heterogeneous with respect to pathophysiology and treatment outcomes. While pharmacological treatment has positive behavioral effects for both children and adults with ADHD, cognitive dysfunctions often remain despite optimal medication [1] and some parents are concerned regarding potential long-term side effects [2]. Some studies have shown that cognitive training can effectively address cognitive dysfunctions and reduce symptoms, potentially providing an alternative or supplement treatment [3,4]. Cognitive training is a broad area with different approaches focusing on different cognitive functions. Studies on working memory training have led the field [5,6] followed by attention and executive function training [7–9]. The goal of cognitive training is to improve the directly trained functions such as attention or working memory (near transfer) and the hope is that the improvement will generalize to other untrained cognitive functions and symptoms (far transfer). So far most cognitive training approaches show near-transfer effects, while far-transfer has been more difficult to achieve.

In a randomized control trial, we investigated the effect of cognitive training with ACTIVATE, an online intervention targeting attention, memory and executive functions in children with ADHD. As previously reported [10], we tested the intervention in individuals with ADHD age 6 to 13 years, and applied a range of different individual cognitive tests as outcome measures. We found no significant effect on our primary outcome, sustained attention, or on ADHD symptoms. We found a significant intervention effect only on planning ability. However, there was greater pre-post improvement in the intervention group across nearly all measures suggesting the value of considering all measures in a composite outcome variable to capture different patterns of improvement, reduce variance and increase statistical power, as recommended and done by others [10–12]. In addition, while our original intent-to-treat-analysis was important, the number of attended treatment sessions varied substantially (M = 26.2, SD = 15.89, range 0–48). In previous work, improvements in cognitive function were demonstrated in school children following 300–600 min of ACTIVATE training with further improvements after 800 min and training typically 2–3 times per week [13]. Based on this, we now explore effects only in children who participated at least twice per week (480 min over 8 weeks) and evaluate the relation between dose and improvements in cognition and symptoms. It is important to investigate the effects of the intervention on subjects who actually actively participated in the intervention. The objectives of the current study are to investigate changes in a composite cognitive score as the primary outcome and changes in ADHD symptom ratings as a secondary outcome. We expect a positive change as the same intervention has shown positive effects in a similar study [4]. Previous research [13] also indicates that there might be a dose-response relationship when using cognitive training, why we are also exploring this possibility in the current analysis.

2. Methods

This is a post-hoc analysis of the effects of a previously published trial which included 70 children age 6–13 years (M = 9.95, SD = 1.7) fulfilling DSM-IV criteria for ADHD and with IQ above 80. Children were randomized to 8 weeks of cognitive training with ACTIVATE and treatment as usual (TAU) (up to 6 times a week) (n = 35) or TAU alone (n = 35) [14]. Outcome measurement was collected before and after the 8 weeks of intervention.

2.1. Study population

Participants (n = 70) age 6–13 years were recruited at several Danish Child and adolescent psychiatric Departments from January 2013 to October 2015. These were children who were referred for assessment for ADHD or already in treatment for ADHD. A detailed protocol for that trial has been published previously [14]. Children were assessed using the Development and Well-being Assessment (DAWBA) [15, an online parent-rated questionnaire screening for ADHD symptoms. The scores of DAWBA were evaluated by a trained psychiatrist. If DAWBA showed a possible ADHD diagnosis, this was confirmed using the the semi-structured interview Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS)-ADHD section [16]. For the current analysis, we restricted the intervention group to children, who completed at least 16 sessions of the intervention (n = 26) and examined a dose-response effect within that group. Additionally, one subject in the control group was excluded as follow-up data were missing (n = 34). The final analysis cohort was on average 10.1 ± 1.7 (SD) years of age, 17% female (n = 10), with 58% of subjects on medication (= 35). The groups were balanced on these baseline factors. Cognition and ADHD-symptoms were assessed before (T0) and after intervention (T1) with the Cambridge Neuropsychological Test Automated Battery (CANTAB) [17] and the parent-rated ADHD-Rating Scale-IV (ADHD-RS) [18], respectively. Participants were medication-free 24 h prior to cognitive testing.

2.2. Intervention

Both the intervention and control groups received treatment as usual (TAU). Besides TAU, the intervention group used the first version of the computer program ACTIVATE six times a week for 8 weeks at home. The duration of each session was 30 min. The intervention consisted of three exercises: Catch the Ball, Butterflies, and What Comes Next. These games target a broad range of cognitive functions including sustained attention, working memory, and response inhibition.

2.3. Primary and secondary outcomes

A composite cognition score was the primary outcome in the current analysis and was calculated based on the following CANTAB measures: 1) Stockings of Cambridge (SOC) (Problems solved in minimum moves) is a spatial planning test. 2) Spatial Working Memory (SWM) (Between errors) is a test of the ability to retain and manipulate spatial information. 3) Attention Switching Task (AST) (Total commission and omission errors) is a test of the child’s ability to switch attention and to ignore task-irrelevant information. 4) Intra-extra dimensional set shift (IED) (EDS errors) is a test of rule acquisition, reversal, attentional set formation maintenance, shifting, and flexibility of attention. 5) Reaction time (RTI) (Mean five-choice reaction time) measures motor and mental response speeds and movement time. 6) Rapid Visual Information Processing (RVP) (Mean latency and RVP A’) is a test of sustained attention. We chose RVP A’ as a dependent measure as it corrects for false positives, thus indicating how good the subject is at target detection regardless of response tendency, while correct answers as measured by RVP Probability of Hit reported in the original paper are based only on hit rate.

Scores for the stop signal task (SST) were presented in the original report, but not included in the composite because of ceiling effects and very little variability in the original trial. For each variable, we created new z-scores for each subject using only the restricted sample at T0 and TI, based on the variable’s mean and standard deviation at baseline (among both groups combined). The composite score consists of the mean of these z-scores. As secondary outcomes, we included the total score on the symptom scale ADHD-Rating Scale-IV (ADHD-RS) and its scores on the two subscales Inattention and Hyperactivity/Impulsivity. The Danish version of the ADHD-RS consists of 26-items, comprising nine items on inattentiveness, nine items on hyperactivity/impulsive behavior and eight questions on oppositional behavior with adequate psychometric properties [19,20]. ADHD-RS has been widely used in research and clinics and has been shown to be sensitive to changes induced by pharmacological treatment [21].

2.4. Analysis

All analyses were performed using SAS, version 9.4 (Cary, NC). In the primary analyses, changes from T0 to T1 on the composite cognitive score were evaluated using repeated measures ANOVA to compare changes between groups over time. Paired t-tests were utilized post-hoc to examine pre-post changes within the group. Changes in secondary outcomes were measured using the clinical symptoms questionnaire ADHD-RS (Inattention, Hyperactivity, and Total scores) and evaluated using similar models. Correlations between the amount of training and cognitive improvement, and between cognitive and symptom improvement, were evaluated with Spearman’s Rho. As the a priori hypotheses for the primary outcome measures were unidirectional, one-tailed significance levels were reported.

3. Results

Composite Cognitive Score (see Figure 1): At baseline (T0), the mean z-scores for both the intervention and control groups were near zero indicating that the groups were balanced and each representative of the overall study sample (M = .02, SD = .54, and M = −0.02, SD = .54). Accordingly, the main effect of group was not significant but the group by time interaction on ANOVA approached significance (F (1, 58) = 1.89; p = 0.08, d = 0.36). Paired t-tests showed significant improvement in the intervention group (M = .02 ± .54 (SD) to .15 ± .59, t(25) = 2.08, p = .02, d’ = 0.41) and a small, non-significant decline in the controls (M = 0.02 to −0.04 ± .69, t(33) = −0.16, p = .35, d’ = 0.03).

Figure 1. Composite cognitive z- scores over time by group.

3.1. Secondary analyses

The number of training sessions completed in the intervention group ranged from 16 to 48 and was significantly correlated with cognitive improvement (spearman rho = .48, p = .01).

Across both groups, decreases were observed in total ADHD-RS scores (time effect: F(1,48) = 12.0, p = 0.001), and in both the inattentive (F(1,48) = 9.79, p = 0.003) and hyperactivity symptoms (F(1,48) = 9.47, p = 0.003) subscales. However, the observed decreases in symptoms did not differ between groups (group by time interaction: all p > 0.52). No associations between the magnitude of cognitive improvement and a decrease in symptoms were observed (see Table 1 for more details).

Table 1. Means and SD for the ADHD-Rating Scale-IV.

ADHD-RS			Inattentive Scale		Hyperactive Scale		Total Score
Group	Time	N	Mean	SD	Mean	SD	Mean	SD
Active	T0	21	18.19	3.94	14.57	5.81	32.77	7.11
	T1	21	15.86	5.61	12.38	6.22	28.24	9.98
Control	T0	29	16.86	4.37	13.69	6.56	30.55	9.10
	T1	29	15.28	4.73	12.17	6.62	27.45	9.55

SD: standard deviation

4. Discussion

Reanalysis of our data using the composite cognitive score showed significant improvement in cognition over time in the intervention group and small, non-significant decline in the control group. However, the group-by-time interaction on ANOVA did not reach significance, presumably due to variance in the sample when the groups were combined. This result could be also due to a lack of power to detect a change, or the mix of pathophysiologically different subgroups of patients sharing the clinical diagnosis of ADHD but with different treatment sensivities [22]. In addition, we found that having completed a higher number of training sessions was significantly correlated with cognitive improvement. This is an important finding, indicating a possible dose-response effect in cognitive training. A Spearman rho of 0.48 is considered a moderate correlation and it is relatively uncommon to find much stronger correlations between treatment interventions for cognitive and behavioral disorders and decreases in measures of cognitive function, because of other controlled influences of outcomes, including variability in general intelligence and learning, measurement error and incompleteness in terms of assessment breadth, and the well-recognized pathophysiological heterogeneity within the symptom-based diagnostic category of ADHD.

Compliance with treatment has been problematic in our original trial [14] and negative results of our intent to treat analyses might reflect noncompliance to the intervention. If the training does benefit those who complete it, efforts can then be directed at addressing issues of engagement, motivation, compliance, and optimal dose and frequency of training. One way to enhance compliance could be the development of more engaging games and a more elaborate reward system to keep the children interested in the intervention.

In a study of 93 children with ADHD symptoms, Wexler et al. [4] found a 30% reduction in the severity of ADHD symptoms measured with the questionnaire SNAP-IV [23] following ACTIVATE training, compared with a control period in the same children. Furthermore, objectively measured improvements in cognitive functions were associated with parent-reported improvement in the child’s ADHD symptoms. In the present post hoc analyses of our randomized trial, we did not find an effect of the intervention on parent-rated symptoms of ADHD. This finding could be related to the fact that the children received other treatments, including medications, which reduced the level and range of symptoms. Both groups showed significant reductions in clinical symptoms indicating that the control group received effective symptom treatment but without evidence of improvement in cognition. The improvement in cognitive function in this context suggests that the effects of medication and the other treatments are incomplete, hence the value of augmentation with cognitive training. Similar to Wexler et al. [4], we did find pre-post cognitive improvements in the intervention group. On the other hand we found reduced severity of inattention symptoms in both groups, but only a reduction of ADHD-RS total score and hyperactive symptoms in the intervention group. These differences in outcomes in our and Wexler et al. [4] trial may be related to differences in study designs, sample sizes and intensity of treatment. In Wexler et al. [4], the sample was 3-fold higher and the intervention lasted 15 weeks rather than 8 weeks and included physical cognitive exercises, thus providing better power to detect associations and also greater total dose of training. Indeed, the current study was powered (80%) to detect only large differences between groups (d = 0.74) and medium differences within groups (d’ = 0.50–0.57). Perhaps symptom reduction requires higher dose and/or longer time to manifest. Few children received other treatments in Wexler et al. [4] while in our study all children received other apparently effective treatments as usual contributing to symptom reductions.

Previous studies have shown significantly decreased parent-rated ADHD symptoms, improved executive functions, and associated ERP changes [4,24] when ACTIVATE was done together with physical exercises in unmedicated children, significant changes on fMRI and consistently greater, but not statistically significantly greater symptom reductions compared to controls when ACTIVATE was added to pharmacotherapy in children with persistent symptoms [25], significant dose-related improvements in executive functions in a large sample of school children [13] and transfer of effects to improved academic performance [26]. These findings together with the present results, and the absence of side effects, suggest that ACTIVATE could be a reasonable initial treatment for children for whom a non-medication treatment is desired or an augmentation treatment for children who have persistent symptoms or cognitive challenges after pharmacotherapy.

The studies together suggest that it could be important to investigate if cognitive training may have the potential to offer beneficial effects on cognitive and behavioral impairments in a dose-dependent manner. Future studies should be sufficiently powered to investigate optimal intensity and dose of training, and effects of cognitive training alone versus in combination with other evidence-based treatments for ADHD.

The strength of the current analyses is that we focus only on children who actually participated in the intervention, giving us the opportunity to investigate the effects of the received intervention. Nevertheless, there are several limitations. Our analysis was likely underpowered due to small sample size. Furthermore children and their parents were not blinded to group allocation.

5. Conclusions

Although there were no significant differences between the groups when considering the cognitive composite score and symptoms, cognition improved significantly in the intervention, but not in the control group. Amount of the completed training sessions correlated significantly with the amount of improvement. Variations in dose and frequency of training may be an important source of the variance in previous studies and important to investigate in future studies.

Ethical approval

This analysis has been approved by the Danish Data Protection Agency (ID.nr. 19/27545). The original trial was approved by the Regional Scientific Ethical Committee for Southern Denmark (nr.S20120096) and registered at ClinicalTrials.gov (NCT01752530).

Disclosure statement

Prof Wexler is founder and holds equity interest in the Yale Start Up company C8 Sciences which developed the intervention evaluated in this study and markets subsequent iterations of the program. All other authors have no conflicts of interest.

Data availability statement

Data can be requested from the Danish National Archives.

Additional information

Funding

We thank the funding organizations of the original trial: TrygFonden, Psychiatric Foundation of Region Southern Denmark, The Region of Southern Denmark’s PhD pool and University of Southern Denmark. Dr. Dalsgaard’s research is currently supported by grants from the European Union’s Horizon 2020 research and Innovation programme under grant agreements No 847879 (PRIME), and 965381 (TIMESPAN), and from Sawmill-owner Jeppe Juhl’s and wife Ovita Juhl’s Fund. Funders had no involvement in any aspect of the trial.