ABSTRACT
Our objective was to review the literature and quantitatively summarise the effectiveness of Goal Management Training® (GMT) (alone or in combination with other training approaches) in improving executive functions in adult populations. Ovid, Scopus, Web of Science, and ProQuest Dissertations & Theses Global were searched for articles citing “goal management training”. Any group trials (n > 3) in adults that used multiple-session GMT programmes were included in the analyses. Outcome variables were extracted and classified into one of nine cognitive measures domains: executive functioning tasks, everyday executive functioning tasks, subjective executive tasks rated by the patient, subjective executive tasks rated by proxy, working memory, speed of processing, long-term memory, instrumental activities of daily living and general mental health status questionnaires. A total of 21 publications, containing 19 separate treatment group samples were included in the final analyses. Significantly positive small to moderate effect sizes were observed in all cognitive measure domains (except speed of processing) with effects maintained at follow-up assessments for all followed-up outcome measures, except for subjective ratings by patients and proxy. The analysis suggests that GMT is an effective intervention, leading to moderate improvements in executive functions that are usually maintained at follow-up.
Introduction
Executive functions comprise a series of higher order cognitive abilities that are involved in the control of goal-directed behaviour, including planning, monitoring, inhibiting, maintaining sustained attention and task switching (Burgess, Veitch, de Lacy Costello, & Shallice, Citation2000; Stuss, Citation2011). Executive functions are believed to be under the control of large brain networks covering frontal, parietal and temporoparietal and subcortical regions (Corbetta & Shulman, Citation2002).
Executive deficits are among the first cognitive deficits to emerge with ageing (Hedden & Gabrieli, Citation2004) and a common occurrence in a variety of patient populations, including traumatic brain injury (TBI) (Kennedy et al., Citation2008), stroke (Chung, Pollock, Campbell, Durward, & Hagen, Citation2013), schizophrenia (Eisenberg & Berman, Citation2010), Attention Deficit Hyperactivity Disorder (ADHD) (T. E. Brown, Citation2008) and multiple sclerosis (MS) (Pepping, Brunings, & Goldberg, Citation2013). Despite distinct neuropathological processes, it is believed that the underlying mechanism – the disruptions of brain networks controlling executive deficits – is the same across populations. Therefore, similar rehabilitation approaches are likely to be effective across populations.
Goal Management Training® (GMT) (Levine et al., Citation2000, Citation2011) has become a leading rehabilitation approach for patients with executive deficits. It is a standardised meta-cognitive training programme containing approximately 20 hours of training, including psychoeducation, narrative examples, mindfulness practice and assignments completed both between and within sessions. Meta-cognitive programmes educate patients on how to gain awareness of their deficits, self-monitor and regain control over their ability to perform activities of daily living. Recent clinical care practice recommendations (Bayley et al., Citation2014) state that they have the strongest evidence supporting their efficacy.
GMT was first conceived by Robertson (Citation1996) at the Medical Research Council Applied Psychology Unit in Cambridge, United Kingdom (UK). The first published report of GMT, led by Levine and Robertson (Levine et al., Citation2000), included a case study of a patient with limbic encephalitis and a separate experimental probe of a brief (1-hr) trial of GMT in patients with TBI. Multi-session versions of GMT in a group format were subsequently implemented with older adults (Levine et al., Citation2007; van Hooren et al., Citation2007). In collaboration with Robertson and Manly, Levine expanded the training to include a mindfulness meditation component and applied it to patients with frontal lobe damage (Levine et al., Citation2011). The current version of GMT consists of 20 hours of training and its materials and manual were made commercially available in 2012 by Baycrest (Levine, Manly, & Robertson, Citation2012) (GoalManagementTraining.com).
The theoretical foundation of GMT holds that the sustained attention system maintains higher order goals in mind while inhibiting automatic processes (Robertson & Garavan, Citation2000). Executive deficits emerge from the disruption of this distributed system such that automatic processes prevail over higher order goals. GMT trains individuals to periodically stop ongoing behaviour to interrupt automatic processing, bring their overarching goal to mind, subdivide the overall goal into subgoals, and monitor performance. This is accomplished through psycho-educational instruction, narrative, within- and between-session exercises that illustrate concepts and principles. GMT incorporates mindfulness meditation (Segal, Williams, & Teasdale, Citation2002). Mindfulness meditation augments GMT by training the ability to bring attention to the present moment in order to monitor the relationship between current circumstances and higher order goals.
Aside from older adults, GMT has been applied to a variety of patient populations such as acquired brain injury (ABI) (e.g., stroke, traumatic brain injury, tumour) (Bertens, Kessels, Fiorenzato, Boelen, & Fasotti, Citation2015; Levine et al., Citation2011; Miotto, Evans, Souza De Lucia, & Scaff, Citation2009; Novakovic-Agopian et al., Citation2011; Spikman, Boelen, Lamberts, Brouwer, & Fasotti, Citation2010; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016), schizophrenia (Levaux et al., Citation2012), substance abuse (Alfonso, Caracuel, Delgado-Pastor, & Verdejo-Garcia, Citation2011; Valls-Serrano, Caracuel, & Verdejo-Garcia, Citation2016), MS (Richard, Citation2013), ADHD (In de Braek, Dijkstra, Ponds, & Jolles, Citation2012), spina bifida (Stubberud, Langenbahn, Levine, Stanghelle, & Schanke, Citation2013a, Citation2013b, Citation2014) and post-discharge intensive care unit (ICU) patients (Jackson et al., Citation2012). Some studies have used the classic GMT training, while others have combined GMT with other training approaches such as psycho-education (Levaux et al., Citation2012), problem-solving therapy (PST) (Miotto et al., Citation2009), errorless learning (Bertens, Fasotti, Boelen, & Kessels, Citation2013, Citation2015; Bertens, Kessels, et al., Citation2015) and emotional regulation (Tornås, Løvstad, Solbakk, Evans, et al., Citation2016).
A recent systematic review examining the effectiveness of GMT (Krasny-Pacini, Chevignard, & Evans, Citation2014) focused only on patients with ABI and did not quantitatively summarise the literature. Conclusions for such a qualitative analysis are determined by studies reporting significant effects. By contrast, a comprehensive quantitative systematic review summarises effect sizes of all available studies, enabling a more reliable estimate of the efficacy of GMT. Given the number of studies that have examined the effectiveness of GMT in a variety of samples other than ABI, and the fact that the underlying executive deficits are common across diseases, we conducted a systematic review of the entire known literature on GMT.
Methods
Eligibility criteria
Any studies that used GMT as a treatment for executive deficits were included in the meta-analysis, provided they included multiple sessions. Single-session interventions provide only a summary of GMT concepts and lack many of the additional components associated with a multiple-session GMT training. GMT studies applied to any adult subject samples were included in the review, including healthy and patient samples.
Information sources, search and study selection
The following electronic databases were searched: Ovid (Medline (1946-Present) and PsycINFO (1806-Present), Scopus (all years), Web of Science (all years), and ProQuest Dissertations & Theses Global for English language papers with the phrase “goal management training” as a term in titles, abstracts or references up until 2 February 2017. This ensured that papers citing Levine et al. (Citation2000, Citation2011) and Robertson (Citation1996) were included. We also searched for citations of Levine et al.’s (Citation2007) paper on GMT in older adults. All searches were limited to English. The literature search and screening were performed by one of the authors.
The following exclusion criteria were applied to the quantitative summary: (1) articles not addressing an intervention; (2) protocol publications; (3) review articles; (4) studies with insufficient information to calculate an effect size; (5) pediatric studies (GMT has been applied to pediatric populations, but the studies are only case series so far); (6) any studies with N ≤ 3 were excluded as these studies would not provide reliable estimates of variability that are needed to calculate an effect size (Rohling, Faust, Beverly, & Demakis, Citation2009).
In an attempt to find both additional published and unpublished GMT research studies and dissertations, we contacted all senior authors of all identified GMT studies as well as those who requested information about GMT from our laboratory. In addition to published manuscripts, unpublished studies, manuscripts in preparation, submitted or in-press manuscripts, undergraduate, Master’s, or PhD theses were requested.
Data collection process
All data were first extracted in a spreadsheet by one of the authors. The spreadsheet contained the variable name, means, standard deviations (SDs, or if not available mean difference) and sample sizes. Communication with two authors was initiated to clarify or provide additional information.
Data items
The following items relating to study design were collected for each study (): (1) Study Name; (2) Reason for Exclusion from quantitative analysis (where applicable); (3) Sample # Identifier for studies that were published in multiple publications; (4) n (GMT, Ctrl): number of subjects in GMT and Control Group; (5) GMT session characteristics: number of sessions, duration of each session, sessions/week and total hours of training; (5) Study Class: all studies were classified into one of three different classes as defined by Cicerone et al. (Citation2000). Class I studies were well designed, prospective, randomised controlled trials (RCTs) with a control group; Class II studies were prospective, non-randomized studies and Class III were studies without concurrent controls. (6) Etiology: diagnosis of the sample; (7) Recovery level: the duration between disease onset and treatment administration; (8) Classic (0)/Combined (1) GMT: this variable indicated whether the GMT programme included only GMT, or whether it was administered in combination with other therapies. (9) Control Group: the type of control group used in the study (e.g., active, usual care); (10) GMT protocol followed: this category kept track of the protocol used in the training when this information was available. (11) Number of sessions of training; (12) Total hours of training (Includes both GMT and other training); (13) Number of sessions per week and (14) Follow-up (F/U): indicated whether a follow-up assessment was performed and if so how many months after completion of the training.
Table 1. Summary of GMT studies.
Outcome variables
We categorised outcome measures into one of nine cognitive measure domains (for a similar approach, see Rohling et al., Citation2009; Vasquez & Zakzanis, Citation2014): executive functioning tasks (containing objective executive function assessments), everyday executive function tasks (tests assessing executive functions in a more naturalistic context), subjective executive ratings by the patient, subjective executive ratings and by proxy, working memory, speed of processing, memory (long-term), instrumental activities of daily living (iADL) and general mental health status questionnaires. A separate meta-analysis was completed for each cognitive measure domain. In order to obtain a single effect size per study, we took the average effect size across tests falling within the same domain. Tests with multiple outcomes were averaged to a single effect size per test. Thus, assessments with multiple outcomes, or studies with multiple assessments per cognitive measure domain, did not disproportionately contribute to the overall effect size. Appendix 2 (see supplemental) contain a list of all outcome measures and how they were averaged together.
The domains of executive functions (subjective and objective) were considered primary outcome variables most sensitive to the efficacy of GMT. Speed of processing, working memory, long-term memory, iADL and mental health status questionnaires were considered secondary outcomes. Although these measures were not expected to reflect direct effects of GMT, we reasoned that indirect effects could be garnered through improved attention and goal-orientation due to GMT. Furthermore, there is a significant relationship between executive functions and iADLs, with executive functions serving as a significant predictor of iADL change over time (Cahn-Weiner et al., Citation2007).
Statistical analysis
All data were analysed using Comprehensive Meta-Analysis (CMA) Software Version 3.3070 (www.meta-analysis.com). Most studies had published mean and SDs before and after each intervention, which allowed us to compute Hedges’ g and standard errors for each study’s assessed cognitive measure domain. The interpretation of the magnitude of Hedges’ g is identical to that of Cohen’s d (Cohen, Citation1988) (0.2 – small, 0.5 – medium and 0.8 – large). For Class III studies with only one group, we included an estimate of pre- to post-treatment correlation (Borenstein, Hedges, Higgins, & Rothstein, Citation2009a) (using either the test–retest reliability from neuropsychological assessment compendiums (Lezak, Citation1995; Spreen & Strauss, Citation1998) or, where not available, assuming a 0.75 test–retest reliability, resulting in a conservative estimate of the effect size). When necessary, the effect direction was recoded to account for different test scoring criteria (i.e., whether a low or high score represents a better outcome). We should note here that reaction times for some tests (e.g., sustained attention to response task) was expected to slow post-training due to increased monitoring and control.
As we included studies with different patient samples, levels of evidence and treatment intensities, we assumed that the effect size would vary from study to study and therefore used random-effects models to calculate all summary effects. To confirm this, statistical heterogeneity was calculated.
Statistical heterogeneity
In order to quantify how the effect size varies from study to study, we report on the Q statistic (a weighted sum of squares), the between-study variance (Tau2) and the ratio of true heterogeneity to total observed variation (I2) (Borenstein, Hedges, Higgins, & Rothstein, Citation2009b). A significant p-value associated with the Q statistic suggests that the effects vary. Tau2 is in the same metric as the effect size and reflects the absolute amount of deviation on that scale. I2 allows the reader to interpret heterogeneity independently of scale and provides us with the proportion of the observed variance that reflects real differences in the effect size.
Meta-regression
For cognitive domains that had a sufficiently large number of studies (≥10), meta-regression analyses were run to examine the relationship between the studies effect sizes and several study-level covariates, including study class, recovery level, GMT type (classic versus combined), number of GMT treatment hours and number of overall treatment hours (including GMT and any other type of training that may have been combined with GMT). Between-study variance (Tau2) was estimated using the residual maximum likelihood (REML) method to produce an adjusted R2 statistic.
Risk of publication bias across studies
As described in our search methods, we included both published and unpublished studies. To assess risk of bias across studies, we presented forest plots sorted according to weight, created funnel plots and calculated a Fail-Safe N (Nfs) and Egger’s regression intercept (Egger, Davey Smith, Schneider, & Minder, Citation1997). Please refer to the Appendix 1 (see supplemental) for this information.
Results
Study selection
includes a summary of our search results. Across four of the databases (PsycINFO, PubMED, Scopus and Web of Science) we identified 1295 references. These results were all entered in the reference management software package EndNote X7 and after removing duplicate references, we were left with 647 references whose titles, and where necessary abstracts, were screened. Seven additional studies were added as a result of our communications with other authors and screening through references of identified GMT articles or dissertations. One of these studies included results for one data set that was in preparation for submission (O’Connor et al., Citation2013). A total of 21 studies, containing 19 separate treatment group samples were included in the final analyses.
Figure 1. Flow of information through the different phases of the systematic review.
Study characteristics
The study characteristics for all identified GMT studies are presented in . Eight of the samples came from RCTs (Class I) (In de Braek et al., Citation2012; O’Connor & Levine, in prep; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013a, Citation2013b, Citation2014; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016; Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016; Valls-Serrano et al., Citation2016; van Hooren et al., Citation2007), five studies were non-randomised trials with a control group (Class II) (Alfonso et al., Citation2011; Levine, Schweizer et al., Citation2011; Levine, Stuss et al., Citation2007; Miotto et al., Citation2009; Novakovic-Agopian et al., Citation2011) and six samples were not compared with a control group (Class III). Four of the Class III samples came from well-designed RCTs where GMT alone was compared with a modified GMT treatment (Bertens, Kessels, Boelen et al., Citation2016; Bertens, Kessels, et al., Citation2015; Cuberos-Urbano et al., Citation2016), but given that the purpose of our analysis was to compare GMT with either no treatment or an active control condition that did not involve any GMT training, we treated those groups as separate studies with no concurrent control group. Since Class III studies are often excluded from meta-analyses due to lack of a control group, we ran all analyses with Class III studies excluded and these are included in the Appendix 1 (see supplemental) to ensure our results are not overly influenced by these studies ().
Table 2. Number of samples and number of participants according to etiology included in the meta-analysis.
A mean total of 300 patients were included in the treatment arms of those studies (where the number of patients varied by outcome measure, the mean number of participants was taken). The majority of patients were ABI (n = 139) (see ). Twelve of the study samples used some form of modification of GMT (e.g., additional psycho-education, or mindfulness or providing experimental cues) (see ). Where control groups were used, eight studies used some kind of active control group, while five samples were compared with usual care/wait-list type of group.
On average, each study sample contributed to five cognitive outcome categories, with the executive tasks category having the most samples (n = 17) contributing to the summary effect size, followed by the subjective executive (patient) category (n = 16) and the everyday subjective executive tasks category (n = 15).
The number of sessions ranged from six to twenty-four and session durations lasted from less than an hour to up to three hours. The total training hours (GMT plus any other training components) ranged from 10 to 45 hours, while GMT-alone hours ranged from four to twenty-three. The training took place at different stages of recovery across patients, but mostly with chronic-phase patients.
Seven studies included a follow-up, usually at six months post training, with each study contributing on average to four cognitive measure domains. Seven out of the 19 samples included in the analysis were classified by us as “classic” GMT and the rest consisted of “combined” approaches.
Effects of GMT on primary outcome cognitive measure domains
Comprehensive lists and full names of the assessments for each cognitive measure outcome within each cognitive domain are provided in the tables in Appendix 2 (see supplemental).
Executive function tasks were reported in 17 samples across 15 studies (Alfonso et al., Citation2011; Bertens et al., Citation2016; Cuberos-Urbano et al., Citation2016; Fahmi, Citation2013; Levine et al., Citation2011; Miotto et al., Citation2009; Novakovic-Agopian et al., Citation2011; O’Connor & Levine, in prep; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013b; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016; Valls-Serrano et al., Citation2016; van Hooren et al., Citation2007; Waid-Ebbs et al., Citation2014) of which six included follow-up (Levine et al., Citation2011; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013b; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016; Waid-Ebbs et al., Citation2014). A significant beneficial effect for GMT on executive functions tests was reported immediately after the training, Hedges’ g = 0.227, SE = 0.063, p < 0.001 and sustained at follow-up, Hedges’ g = 0.549, SE = 0.255, p < 0.031 (). There was significant statistical heterogeneity in the immediate post-training analyses (Tau² = 0.045; Q = 65.90, df = 16, p < 0.001; I² = 76%) and the long-term follow-up (Tau² = 0.366; Q = 101.75, df = 5, p < 0.001; I² = 95%) supporting the use of random-effects models.
Figure 2. Forest plot of studies examining everyday executive functions tasks immediately after training and at follow-up.
Solid squares = effect size of each study; size of squares = study weight (weighted by sample size); Lines = 95% confidence interval; diamond = summary effect; width of diamond = precision. TAP-A = Test of Attentional Performance Alertness, CFT = Category Fluency Test, LF = Letter Fluency, CPT = Continuous Performance Test II, CWI = Color-Word Interference Test, VF = Verbal Fluency, TMTB = Trail Making Test B, TEA = Test of Everyday Attention, SART = Sustained Attetion to Response Task, PASAT = Paced Auditory Serial Addition Test, CANTAB IST = Information Sampling Task, SOC = Stocking of cambridge, TMT = Trail Making Test, DF = Design Fluency, DV = Digit Vigilance Test, FAS = FAS Verbal Fluency Test, WCST = Wisconsin Card Sorting Test.
Everyday executive function tasks were reported in 15 samples across 13 studies (Alfonso et al., Citation2011; Bertens et al., Citation2016; Cuberos-Urbano et al., Citation2016; In de Braek et al., Citation2012; Levine, Schweizer et al., Citation2011; Levine, Stuss et al., Citation2007; Miotto et al., Citation2009; O’Connor & Levine, in prep; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013b; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016; Valls-Serrano et al., Citation2016) of which six included follow-up (In de Braek et al., Citation2012; Levine et al., Citation2011; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013b; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016). There was a significant beneficial effect for GMT on everyday executive functions tests immediately after the training, Hedges’ g = 0.297, SE = 0.087, p = 0.001, which was sustained at follow-up, Hedges’ g = 0.385, SE = 0.187, p = 0.040 (). Statistical heterogeneity was again observed in both the immediate post-training analyses (Tau² = 0.067; Q = 44.626, df = 14, p < 0.001; I² = 69%) and the long-term follow-up (Tau² = 0.141; Q = 17.20, df = 5 (p = 0.004); I² = 71%).
Figure 3. Forest plot of studies examining everyday executive functions tasks immediately after training and at follow-up.
Solid squares = effect size of each study; size of squares = study weight (weighted by sample size); Lines = 95% confidence interval; diamond = summary effect; width of diamond = precision. MSET = Modified Six Elements Test, BADS = Behavioural Assessment of the Dysexecutive Syndrome, MET = Multiple Errand Test, R-SAT = Revised Strategy Application Test, UPSA = UCSD Performance-Based Skills Assessment, MET-CV = Multiple Errands Test − contextualised version, VPT = Virtual Planning Test, SRLT = simulated real-life tasks.
Subjective executive function questionnaires rated by patients were used as outcome variables in 16 samples across 14 studies (Bertens et al., Citation2016; Cuberos-Urbano et al., Citation2016; Fahmi, Citation2013; In de Braek et al., Citation2012; Levine, Schweizer et al., Citation2011; Levine, Stuss et al., Citation2007; Miotto et al., Citation2009; O’Connor & Levine, in prep; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013a; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016; van Hooren et al., Citation2007; Waid-Ebbs et al., Citation2014) of which seven included follow-up (In de Braek et al., Citation2012; Levine et al., Citation2011; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013a; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016; Waid-Ebbs et al., Citation2014). There was a significant beneficial effect for GMT on patient rating scales immediately after the training, Hedges’ g = 0.136, SE = 0.061, p = 0.024, but not at follow-up, Hedges’ g = 0.128, SE = 0.131, p = 0.327 (). We observed statistical heterogeneity in the immediate post-training analyses (Tau² = 0.024; Q = 31.11, df = 15, p = 0.008; I² = 52%) and the long-term follow-up (Tau² = 0.065; Q = 16.35, df = 6, p = 0.012; I² = 63%).
Figure 4. Forest plot of studies examining subjective ratings of executive functions by patient immediately after training and at follow-up.
Solid squares = effect size of each study; size of squares = study weight (weigthed by sample size); Lines = 95% confidence interval; diamond = summary effect; width of diamond = precision. EFI = Executive Function Index, EOS = Executive Observation Scale, CFQ = Cognitive Failures Questionnaire, DEX = Dysexecutive Questionnaire, BRIEF-A = Behavior Rating Inventory of Executive Function – Adult, GMT-Q = Goal Management Training Questionnaire.
Ratings by proxy (caregiver or therapist) were used in nine samples across eight studies (Bertens et al., Citation2016; Fahmi, Citation2013; Miotto et al., Citation2009; O’Connor & Levine, in prep; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013a; Waid-Ebbs et al., Citation2014) of which four included follow-up (Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013a; Waid-Ebbs et al., Citation2014). There was a significant beneficial effect for GMT on proxy rating scales immediately after the training, Hedges’ g = 0.341, SE = 0.077, p < 0.001, but not at follow-up, Hedges’ g = 0.227, SE = 0.1481, p = 0.124 (). The test for statistical heterogeneity was not significant either in the immediate post-training analyses (Tau² = 0.010; Q = 9.84, df = 8, p = 0.276; I² = 19%) or in the long-term follow-up (Tau² = 0.032; Q = 4.77, df = 3, p = 0.190; I² = 37%).
Figure 5. Forest plot of studies examining subjective ratings of executive functions by proxy immediately after training and at follow-up.
Solid squares = effect size of each study; size of squares = study weight (weigthed by sample size); Lines = 95% confidence interval; diamond = summary effect; width of diamond = precision. EOS = Executive Observation Scale, CFQ = Cognitive Failures Questionnaire, DEX = Dysexecutive Questionnaire, BRIEF-A = Behavior Rating Inventory of Executive Function – Adult, GMT-Q = Goal Management Training Questionnaire. *rated by both spouse/caregiver and therapist.
Effects of GMT on secondary outcome cognitive measure domains
Working memory tests were reported in 10 samples across eight studies (Alfonso et al., Citation2011; Bertens et al., Citation2016; Cuberos-Urbano et al., Citation2016; Fahmi, Citation2013; Miotto et al., Citation2009; Novakovic-Agopian et al., Citation2011; O’Connor & Levine, in prep; Valls-Serrano et al., Citation2016). There was a significant beneficial effect for GMT on working memory tests, Hedges’ g = 0.438, SE = 0.150, p = 0.004 () with significant statistical heterogeneity (Tau² = 0.166; Q = 48.65, df = 9, p < 0.001; I² = 82%).
Figure 6. Forest plot of studies examining working memory and speed of processing immediately after training.
Solid squares = effect size of each study; size of squares = study weight (weigthed by sample size); Lines= 95% confidence interval; diamond = summary effect; width of diamond = precision. SOPT = Self-Ordered Pointing Test, CT = Consonant Trigrams, DS = Digit Span, LNS = Letter Number Sequencing, AS = Arithmetic Span, TMT = Trail Making Test, VAT = Visual Attention Test, DSMT = Digit Symbol Modalities Test.
Speed of processing tests were included only in five studies (Fahmi, Citation2013; Miotto et al., Citation2009; Novakovic-Agopian et al., Citation2011; Stubberud et al., Citation2013b; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016). The effect for GMT on speed of processing tests was positive, but not significant, Hedges’ g = 0.219, SE = 0.141, p = 0.120 (see ). Non-significant statistical heterogeneity was observed (Tau² = 0; Q = 3.30, df = 4, p < 0.001; I² = 0%).
iADL tasks were reported in seven samples across five studies (Bertens, Kessels, et al., Citation2015; Cuberos-Urbano et al., Citation2016; In de Braek et al., Citation2012; Spikman et al., Citation2010; Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016) of which three included follow-up (In de Braek et al., Citation2012; Spikman et al., Citation2010; Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016). There was a significant beneficial effect for GMT immediately after the training, Hedges’ g = 0.662, SE = 0.230, p = 0.004, which was sustained at follow-up, Hedges’ g = 0.390, SE = 0.190, p = 0.040 (). The statistical heterogeneity at the immediate post-training was quite large (Tau² = 0.334; Q = 141.65, df = 6, p < 0.001; I² = 96%) and smaller at follow-up, but due to the small number of studies it is not clear how reliable this estimate is (Tau² = 0.051; Q = 3.681, df = 2, p = 0.159; I² = 46%).
Figure 7. Forest plot of studies examining Instrumental Activities of Daily Living (iADL) immediately after training and at follow-up.
QOLIBRI = Quality of Life after Brain Injury, GAS = Goal attainment scaling, RRL = Role Resumption List, CIBIC-Clinician’s Interview-Based Impression of Severity and Change.
Six studies (In de Braek et al., Citation2012; Richard, Citation2013; Stubberud et al., Citation2014; Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016; Valls-Serrano et al., Citation2016) used mental health status questionnaires as outcomes, with four of them including follow-up (In de Braek et al., Citation2012; Richard, Citation2013; Stubberud et al., Citation2014; Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016). There was a significant beneficial effect for GMT immediately after the training, Hedges’ g = 0.309, SE = 0.230, p = 0.004, which was sustained at follow-up, Hedges’ g = 0.274, SE = 0.104, p = 0.008 (). Significant statistical heterogeneity was observed immediately post-training (Tau² = 0.072; Q = 12.04, df = 5, p = 0.034; I² = 58%), but not at follow-up (Tau² = 0; Q = 0.436, df = 3, p = 0.933, I² = 0%).
Figure 8. Forest plot of studies examining mental health status immediately after training and at follow-up.
SF-36 = The Short Form (36) Health Survey, HSCL-25 = Hopkins Symptom Checklist, BREQ = The Brain Injury Rehabilitation Trust Regulation of Emotions Questionnaire, PSS = Perceived Stress Scale, SCL-90 = Symptom Check List–90, POMS TMD = Profile of Mood States Total Mood Disturbance.
Long-term memory tests were included in five samples across four studies (In de Braek et al., Citation2012; Levine et al., Citation2011; Richard, Citation2013; Spikman et al., Citation2010; Stubberud et al., Citation2013b; Tornås, Løvstad, Solbakk, Evans, et al., Citation2016) (). The effect for GMT on memory tests was positive and significant, Hedges’ g = 0.269, SE = 0.114, p = 0.018 () with significant statistical heterogeneity (Tau² = 0.043; Q = 14.24, df = 4, p = 0.007; I² = 72%).
Figure 9. Forest plot of studies examining memory immediately after training. RBMT = Rivermead Behavioural Memory test; DR = Delayed Recall, IR = Immediate Recall; LM = Logical Memory; BVMT = Brief Visuospatial Memory Test, HVMT = Hopkins Verbal Learning Test.
A summary of all effect sizes across cognitive measures outcomes is provided in .
Table 3. Summary of effect sizes across cognitive measures domains.
Meta-regression
Given the high heterogeneity in the results, we wanted to examine if there were any moderator variables that explained some of the variance between studies. Meta-regression analyses were run only on cognitive domain areas where there were 10 or more studies (Subjective Executive Proxy domain was included as an exception with only nine studies). Only post-training effects were examined in the meta-regression. The number of studies with follow-up analyses was not sufficient for meta-regression analyses. Only univariate meta-regression analyses were run because at least 10 studies per variable are required for multivariate models. The results are presented in .
Table 4. Meta-regression analyses.
There was no relationship between study class and effect sizes across studies. This is supported by our ancillary analyses excluding Class III studies, which generally resulted in an increase in the overall effect size (see Appendix 1 in supplemental). Supplemental Figure 2 shows the relationship between effect size and study class for the executive functions tasks. Etiology (ABI versus otherwise), GMT type (Classic versus Combined) or Recovery level (months post-onset) also did not show any significant relationship with effect sizes.
The total number of GMT treatment hours showed a significant positive relationship with executive function tasks (p = 0.002) and everyday executive functions tasks (p = 0.046) showing that the more GMT treatment hours patients received the higher the gains were in executive function tasks. In fact, 44% of the inter-study variance in effect sizes in the executive functions domain were explained by the number of GMT training hours (). In the everyday executive functions domain, 20% of the variance was explained by the number of GMT training hours. Taking GMT treatment hours into consideration reduced I2 from 76 down to 67 in executive functions and from 69 down to 65 in the everyday executive functions domain. To determine whether increasing the number of any treatment (GMT or other) relates to effect size, we examined the relationship between total number of treatment hours (GMT+ any other combined approach) and effect size, and a significant effect was observed only in the executive functions domain (p = 0.03), where total treatment hours explained 26% of the variance between studies. Therefore, in this domain, the number of GMT treatment hours alone explained a larger proportion of the effect-size variance across studies than any treatment hours.
Figure 10. Relationship between GMT treatment hours and effect size in Executive Function Tasks.
GMT = Goal Management Training; Ctrl = Control, f/u = follow up; subj.= subjective, ICU = Intensive Care Unit; TBI = Traumatic Brain injury; ABI = Acquired Brain injury; ADHD, Attention Deficit Hyperactivity Disorder, MS = Multiple Sclerosis, SUD = Substance Use Disorder; CVD = Cerebrovascular Disease.
While the number of GMT treatment hours explains some of the variance between studies, the heterogeneity of effect sizes across studies was still high. As more GMT studies are published in the literature it may be possible to examine the effects of these variables through larger samples. Despite the high heterogeneity, the consistency in the direction of the effect across studies, with most studies showing positive effects, suggests that GMT training is beneficial to both ABI and non-ABI patients.
Risk of bias across studies
Risk of bias was assessed in four cognitive domains (executive functions, everyday executive functions, subjective executive ratings by the patient and working memory domains). Significant bias was observed in the executive functions tasks and the working memory domains (suggested adjusted value of 0.140 from 0.227 for executive functions and the value remaining the same for working memory). The number of studies required to nullify the effects were large in both cases (150 and 75 respectively) suggesting the effect to be robust (see Appendix 1 in supplemental for details).
Discussion
Although the prevalence of executive functioning impairment is unknown, the conditions that cause this impairment, such as TBI, stroke and other forms of unhealthy ageing, MS, and other conditions affecting distributed brain networks are highly prevalent. GMT is the only standardised, therapist-led, commercially available intervention for executive impairment. The empirical support for GMT is now at a stage where aggregate data can be evaluated quantitatively. Specifically, we sought to assess the effectiveness of GMT for improving executive functioning.
We identified 19 samples comprising 300 patients in the research literature on GMT, including unpublished papers such as doctoral dissertations. Overall, the results suggest that GMT has a beneficial effect in patients with executive deficits on all primary outcome executive functions cognitive domains. The size of the effects ranged from small to medium (0.136–0.341). Among the primary outcome measures, the greatest effect was observed in the subjective executive functions ratings by proxy, followed by objective tests of executive functions tasks and the everyday executive functions tasks with the size of the effect being moderate, but similar to what has been shown in the literature with respect to overall effects of cognitive training approaches for executive functions (Kennedy et al., Citation2008).
The effect of GMT on the primary outcome cognitive measure domains was maintained at follow-up for all domains that were re-evaluated with the exception of the subjective ratings of executive function as measured by executive function questionnaires rated by either proxy or patients. Questionnaire responses are based on subjective ratings and therefore may be less reliable and stable than test performance. The follow-up effects were also small to medium in size ranging from 0.084 (patient ratings of executive functions) to 0.553 (executive functions tasks domain). The results from objective and subjective executive measures can dissociate (Koerts et al., Citation2011; Meltzer et al., Citation2017; van der Hiele, Spliethoff-Kamminga, Ruimschotel, Middelkoop, & Visser, Citation2012) and it is, therefore, possible that the long-term effects of GMT could vary between the two assessment types. The heterogeneity of the effect sizes was higher and the summary effects were based on fewer studies at follow-up. Thus, the long-term effects of GMT on subjective measures should be re-evaluated once more studies accumulate in the literature.
Significant beneficial effects were also observed in our secondary outcome measures with small- to medium-size effect sizes maintained immediately after training and at follow-up. The only secondary outcome measure that did not reach a significant difference from the null effect was speed of processing. This lack of effect may have been due to the relatively small number of studies with speed of processing outcome measures (n = 5). Despite this, the mean effect size was 0.219, which was similar in size to the other outcome measures domains.
The largest summary effect size across all domains (including both primary and secondary) was observed in the iADL domain with a mean effect size of 0.666, which was also maintained at follow-up (0.415). This domain included general questionnaires about quality of life and instrumental everyday activities. While this result is quite encouraging, as it suggests that the effects of GMT may generalise to everyday activities of daily living, the literature could benefit from more studies that include more comprehensive iADL assessments.
GMT had beneficial effects on mental health status ratings by patients and on tests of long-term memory. The significant beneficial effect of GMT on long-term memory outcomes may be due to patients’ ability to control better their attention at encoding and possibly retrieval, which may have resulted in better scores after the training.
Quality of the evidence
Most of the included studies were Class I, or randomised controlled trials, that included a control group. Five out of the eight samples in this category were compared with active control groups and three were compared with wait-list/standard treatment (Stubberud et al., Citation2013a, Citation2013b, Citation2014; Valls-Serrano et al., Citation2016; van Hooren et al., Citation2007). Another important component of rehabilitation trials is to examine the long-term effects of the intervention and five out of eight studies included a 6-month follow-up assessment.
Five studies were classified as Class II (Alfonso et al., Citation2011; Levine, Schweizer et al., Citation2011; Levine, Stuss et al., Citation2007; Miotto et al., Citation2009; Novakovic-Agopian et al., Citation2011). All Class II studies contained a control group, but the assignment of participant to each group was not fully randomised. Long-term effects were examined in four of these studies, but in two studies the analysis could not be included due to patients participating in cross-over design trials.
Six samples were classified as Class III levels of evidence. Two of these samples were samples that were not compared with controls, but the other four samples came from studies that were well designed randomised trials that compared GMT alone with a combination of GMT with other types of treatment (errorless learning Bertens et al., Citation2016; Bertens, Kessels, et al., Citation2015; or GMT with Lifelog technologies Cuberos-Urbano et al., Citation2016). Given that the purpose of our analyses was to examine the effects of GMT (alone or in combination with other treatment approaches) we treated these groups as standalone samples without concurrent control group. In order to evaluate whether Class III studies artificially inflated the results we re-ran all analyses with Class III studies removed. The removal of these studies more often led to an increase of the effect size as opposed to a decrease. Furthermore, the meta-regression analyses showed no relationship between study class and effect size, further increasing our confidence that Class III studies did not contribute to an artificially inflated result.
Limitations
The significant heterogeneity in the effect size across studies suggests that the effect of GMT training varies significantly and at this point in time it is not entirely clear what influences this variability. In an attempt to determine the nature of the moderators, we ran meta-regression analyses for the domains that had a sufficiently large number of studies to allow meaningful analyses. The only significant moderator identified by us was the number of GMT treatment hours, with more GMT hours leading to larger positive effects of GMT in the executive functions and subjective executive functions tasks domains. It is likely that the variety of training programmes and patients further influences the effects of the training. Our attempt to compare ABI versus non-ABI patients and recovery level at this point showed no relationship with effect size, but as studies accumulate more refined analyses may be feasible.
The variability in intervention types stems from both researchers making their own modifications and from the fact that GMT has undergone its own evolution throughout the years, evolving from a single-hour intervention to a 20-hours, 10-session intervention (see ). That being said, a “combined” intervention does not necessarily mean that the intervention has been significantly modified. For example, some researchers have added additional psycho-education (In de Braek et al., Citation2012; van Hooren et al., Citation2007), or problem-solving therapy (Miotto et al., Citation2009; Spikman et al., Citation2010) or mindfulness (Alfonso et al., Citation2011; Novakovic-Agopian et al., Citation2011; Valls-Serrano et al., Citation2016), all of which are already components of GMT. Other modifications such as giving participants experimental cues (Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016; Tornås, Løvstad, Solbakk, Schanke, et al., Citation2016) or alerts or adding technology such as the GMT+ Lifelog (SenseCam and heart-rate recordings) were additions that did not add to the training programme per se, but may have provided more cues that allowed patients to possibly integrate the programme better into their daily life activities and to become more aware of their own absent-minded errors. The heterogeneity of the effect sizes of GMT studies may decrease with the public availability of GMT kits (GoalManagementTraining.com).
Finally, publication bias was assessed only in domains with 10 or more studies. It is possible that more domains may have shown publication bias, if there were enough studies to examine this in the other domains and as the literature grows that may be something to examine in future meta-analyses.
Agreements and disagreements with other studies or reviews
In one qualitative review of GMT in ABI patients (published up to December 2011) (Krasny-Pacini, Chevignard, et al., Citation2014), the authors concluded that there is a lack of evidence supporting the effectiveness of GMT alone, but that combined GMT approaches (Miotto et al., Citation2009; Spikman et al., Citation2010) may be more effective. More specifically, the authors cited PST, focusing on patients’ own personal goals, the use of daily life tasks as part of the training programme and between-session assignments, external cueing and training intensity of more than 15 hours as some of the ingredients that make combined GMT approaches more effective. As noted above, many of these components are already part of the regular GMT programme. For example, the PST programme requires patients to state their goal, make a plan, create sub-tasks and evaluate progress, and in case of goal failure, a new plan is implemented (Miotto et al., Citation2009). These exact steps are already included in detail in GMT (Levine et al., Citation2000). Furthermore, “focusing on patients’ own personal goals” and “the use of daily life tasks as part of the training programme and between-session assignments” are part of GMT. Given that GMT is usually run in groups, specific personal goals are not worked on in detail during the sessions, but patients are encouraged to set personal goals and work on daily life tasks as part of their take-home assignments. Finally, the current training consists of 20 hours of in-session training, so it is already more than the 15 hours of recommended training. While “combined” approaches often had higher effect sizes, there was no relationship between the type of GMT programme (combined or classic) and effect size. Therefore, currently there is no evidence that augmenting the intensity of PST, focusing on patients’ goals, or between-session assignments exceeding the current standardised GMT protocol improves efficacy. Our meta-regression analyses, examining ranges of 4–23 hours of GMT training, indicate that the more hours spent in GMT training, the greater the benefits in executive function tasks. It is likely, however, that there is a breaking point at which further increases in the number of GMT hours may not lead to further benefits and may even lead to reduced efficacy. For example, for some patients, the amount of between-session assignments can be overly burdensome. In these cases, adding additional assignments would be counter-therapeutic. External cues are the only component that are currently not part of GMT, but given the lack of studies that directly compare a combination of GMT and experimental cues to GMT alone, it is still unclear if the added effect is significant.
Implications for practice
A variety of patient populations present with executive deficits and there are few readily available manualised programmes that address those deficits. The current quantitative review suggests that GMT is an effective therapeutic approach that produces moderate treatment effects in a number of cognitive measure domains, including both objective and subjective measures of executive function, working and long-term memory, iADL and mental health status patient ratings. While the effects are somewhat reduced at follow-up, they remain significantly positive (with the exception of subjective ratings of executive functions by patients or proxy). Currently, GMT is the only therapist-led treatment programme for executive function impairment that is readily available in a standardised package.
The clinical readiness of the programme, combined with the positive treatment effects observed in this meta-analysis, provide support for its use in clinical populations. The summarised evidence here is based on patients with a variety of etiologies (ABI, older adults, substance dependence, MS, cerebrovascular disease (CVD), ADHD or spina bifida). These findings suggest that GMT may be applicable for improving executive functioning across a wide range of clinical conditions. However, as a metacognitive intervention, GMT requires at least a moderate degree of insight and awareness of deficits as well as intact mnemonic processing to carry educational information forward across treatment sessions. Therefore it is not recommended for patients with severely compromised insight or memory impairments. Patients with very severe executive deficits may also not be able to complete the programme due to the high metacognitive demands of this intervention.
Implications for research
One of the most inconsistent study design aspects of the trials included in the quantitative analyses is the variety of treatment intensities (number of sessions and duration of sessions) across trials. As a result, the best administration schedule of GMT is currently unclear. The current training manual consists of nine modules, normally administered in 20 hours of in-class training. While our experience is that this administration works in most patient populations, it is possible that the training could take longer with some populations. We recommend that future studies take advantage of the readily available GMT manual (GoalManagementTraining.com) in order to ensure that GMT is administered in a standardised fashion.
Furthermore, more studies should be conducted in populations other than ABI populations. While the variety of populations included in these analyses speak for the versatility of the training, most investigations of GMT in non-ABI populations included either single or at most two trials.
Finally, it is worth noting that study designs could benefit from larger sample sizes, which should lead to a decrease in the SEs of the effect size across studies.
Supplemental_File
Download PDF (255.8 KB)Supplement - Effect Sizes For Each Raw Outcome Measure
Download MS Excel (48.2 KB)Fig1.Suppl
Download TIFF Image (586 KB)Fig2_Supplement
Download PNG Image (12.4 KB)
Disclosure statement
B.L. receives a percentage of profits from the sale of GMT.
Additional information
Funding
References
- Adnan, A., Chen, A. J. W., Novakovic-Agopian, T., D’Esposito, M., & Turner, G. R. (2017). Brain changes following executive control training in older adults. Neurorehabilitation and Neural Repair, 31, 910–922. doi: 10.1177/1545968317728580
- Alfonso, J. P., Caracuel, A., Delgado-Pastor, L. C., & Verdejo-Garcia, A. (2011). Combined goal management training and mindfulness meditation improve executive functions and decision-making performance in abstinent polysubstance abusers. Drug and Alcohol Dependence, 117(1), 78–81. doi: 10.1016/j.drugalcdep.2010.12.025
- Archer, K. R., Coronado, R. A., Haislip, L. R., Abraham, C. M., Vanston, S. W., Lazaro, A. E., … Obremskey, W. T. (2015). Telephone-based goal management training for adults with mild traumatic brain injury: Study protocol for a randomized controlled trial. Trials, 16(1), 244. doi: 10.1186/s13063-015-0775-1
- Arnemann, K. L., Chen, A. J., Novakovic-Agopian, T., Gratton, C., Nomura, E. M., & D’Esposito, M. (2015). Functional brain network modularity predicts response to cognitive training after brain injury. Neurology, 84, 1568–1574. doi: 10.1212/wnl.0000000000001476
- Barekatain, M., Alavirad, M., Tavakoli, M., Emsaki, G., & Maracy, M. R. (2016). Cognitive rehabilitation in patients with nonamnestic mild cognitive impairment. Journal of Research in Medical Sciences, 21(7), 101.
- Baylan, S., & Evans, J. (2016). Development of a paradigm for studying the effects of brief goal management training with implementation intentions. Neuropsychological Rehabilitation, 68, 1–17. doi:https://doi.org/10.1080/09602011.2016.1256326
- Baylan, S. (2014). Imaging the effects of cognitive rehabilitation interventions: Developing paradigms for the assessment and rehabilitation of prospective memory (PhD.). University of Glasgow.
- Bayley, M. T., Teasell, R. W., Wolfe, D. L., Gruen, R. L., Eng, J. J., Ghajar, J., … Bragge, P. (2014). Where to build the bridge between evidence and practice? Results of an international workshop to prioritize knowledge translation activities in traumatic brain injury care. Journal of Head Trauma Rehabilitation, 29(4), 268–276. doi: 10.1097/htr.0000000000000053
- Bertens, D., Fasotti, L., Boelen, D. H., & Kessels, R. P. (2013). A randomized controlled trial on errorless learning in goal management training: Study rationale and protocol. BMC Neurology, 13, 64. doi: 10.1186/1471-2377-13-64
- Bertens, D., Fasotti, L., Boelen, D. H., & Kessels, R. P. (2015). Moderators, mediators, and nonspecific predictors of treatment outcome in an intervention for everyday task improvement in persons with executive deficits after brain injury. Archives of Physical Medicine and Rehabilitation, 97, 97–103. doi: 10.1016/j.apmr.2015.07.021
- Bertens, D., Kessels, R. P., Boelen, D. H., & Fasotti, L. (2016). Transfer effects of errorless goal management training on cognitive function and quality of life in brain-injured persons. NeuroRehabilitation, 38(1), 79–84. doi: 10.3233/nre-151298
- Bertens, D., Kessels, R. P., Fiorenzato, E., Boelen, D. H., & Fasotti, L. (2015). Do old errors always lead to new truths? A randomized controlled trial of errorless goal management training in brain-injured patients. Journal of the International Neuropsychological Society, 21, 639–649. doi: 10.1017/S1355617715000764
- Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009a). Chapter 4: Effect sizes based on means introduction to meta-analysis. West Sussex, UK: John Wiley & Sons, Ltd.
- Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009b). Chapter 16: Indetifying and quantifying heterogeneity introduction to meta-analysis. West Sussex, UK: John Wiley & Sons, Ltd.
- Brown, P. (2009). Improving planning and prospective memory in a virtual reality setting: Investigating the use of periodic auditory alerts in conjunction with goal management training on a complex virtual reality task in individuals with acquired brain injury (PhD.). University of Glasgow.
- Brown, T. E. (2008). ADD/ADHD and impaired executive function in clinical practice. Current Psychiatry Reports, 10(5), 407–411.
- Burgess, P. W., Veitch, E., de Lacy Costello, A., & Shallice, T. (2000). The cognitive and neuroanatomical correlates of multitasking. Neuropsychologia, 38(6), 848–863.
- Cahn-Weiner, D. A., Farias, S. T., Julian, L., Harvey, D. J., Kramer, J. H., Reed, B. R., … Chui, H. (2007). Cognitive and neuroimaging predictors of instrumental activities of daily living. Journal of the International Neuropsychological Society, 13(5), 747–757. doi: 10.1017/S1355617707070853
- Carstens, J. (2011). The effects of goal management training in undergraduate students with problems in attention functioning ( MR76273 M.A.). University of Windsor (Canada), Ann Arbor.
- Carstens, J. (2016). Goal management training in undergraduate students: The effects on executive functioning skills and academic self-efficacy ( 10156420 Ph.D.). University of Windsor (Canada), Ann Arbor.
- Casaletto, K. B. (2016). A metacognition-based approach to improve HIV-associated neurocognitive disorders among substance users ( 10159275 Ph.D.). University of California, San Diego, Ann Arbor.
- Casaletto, K. B., Moore, D. J., Woods, S. P., Umlauf, A., Scott, J. C., & Heaton, R. K. (2016). Abbreviated goal management training shows preliminary evidence as a neurorehabilitation tool for HIV-associated neurocognitive disorders among substance users. The Clinical Neuropsychologist, 30(1), 107–130. doi: 10.1080/13854046.2015.1129437
- Chen, A. J. W., Novakovic-Agopian, T., Nycum, T. J., Song, S., Turner, G. R., Hills, N. K., … D’Esposito, M. (2011). Training of goal-directed attention regulation enhances control over neural processing for individuals with brain injury. Brain, 134(5), 1541–1554. doi: 10.1093/brain/awr067
- Chung, C. S., Pollock, A., Campbell, T., Durward, B. R., & Hagen, S. (2013). Cognitive rehabilitation for executive dysfunction in adults with stroke or other adult non-progressive acquired brain damage. Cochrane Database of Systematic Reviews, 4, CD008391. doi: 10.1002/14651858.CD008391.pub2
- Cicerone, K. D., Dahlberg, C., Kalmar, K., Langenbahn, D. M., Malec, J. F., Bergquist, T. F., … Morse, P. A. (2000). Evidence-based cognitive rehabilitation: Recommendations for clinical practice. Archives of Physical Medicine and Rehabilitation, 81(12), 1596–1615. doi: 10.1053/apmr.2000.19240
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: Erlbaum.
- Corbett, C. (2008). Rehabilitation of executive functioning following paediatric traumatic brain injury: A goal management training intervention. University of Cape Town.
- Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3(3), 201–215. doi: 10.1038/nrn755
- Cuberos-Urbano, G., Caracuel, A., Valls-Serrano, C., Garcia-Mochon, L., Gracey, F., & Verdejo-Garcia, A. (2016). A pilot investigation of the potential for incorporating lifelog technology into executive function rehabilitation for enhanced transfer of self-regulation skills to everyday life. Neuropsychological Rehabilitation, 17, 1–13. doi: 10.1080/09602011.2016.1187630
- Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634.
- Eisenberg, D. P., & Berman, K. F. (2010). Executive function, neural circuitry, and genetic mechanisms in schizophrenia. Neuropsychopharmacology, 35(1), 258–277. doi: 10.1038/npp.2009.111
- Fahmi, H. (2013). Goal management training: A Web-based approach (MS22101 M.S.). Toronto: University of Toronto (Canada).
- Fish, J., Evans, J. J., Nimmo, M., Martin, E., Kersel, D., Bateman, A., … Manly, T. (2007). Rehabilitation of executive dysfunction following brain injury: “Content-free” cueing improves everyday prospective memory performance. Neuropsychologia, 45(6), 1318–1330. doi: 10.1016/j.neuropsychologia.2006.09.015
- Grant, M., Ponsford, J., & Bennett, P. C. (2012). The application of goal management training to aspects of financial management in individuals with traumatic brain injury. Neuropsychological Rehabilitation, 22(6), 852–873. doi: 10.1080/09602011.2012.693455
- Hedden, T., & Gabrieli, J. D. (2004). Insights into the ageing mind: A view from cognitive neuroscience. Nature Reviews Neuroscience, 5(2), 87–96. doi: 10.1038/nrn1323
- In de Braek, D. M., Dijkstra, J. B., Ponds, R. W., & Jolles, J. (2012). Goal management training in adults with ADHD: An intervention study. Journal of Attention Disorders, 21, 1130–1137. doi: 10.1177/1087054712468052
- Jackson, J. C., Ely, E. W., Morey, M. C., Anderson, V. M., Denne, L. B., Clune, J., … Hoenig, H. (2012). Cognitive and physical rehabilitation of intensive care unit survivors: Results of the RETURN randomized controlled pilot investigation. Critical Care Medicine, 40(4), 1088–1097. doi: 10.1097/CCM.0b013e3182373115
- Kennedy, M. R. T., Coelho, C., Turkstra, L., Ylvisaker, M., Moore Sohlberg, M., Yorkston, K., … Kan, P. F. (2008). Intervention for executive functions after traumatic brain injury: A systematic review, meta-analysis and clinical recommendations. Neuropsychological Rehabilitation, 18(3), 257–299. doi: 10.1080/09602010701748644
- Koerts, J., Tucha, L., Leenders, K. L., van Beilen, M., Brouwer, W. H., & Tucha, O. (2011). Subjective and objective assessment of executive functions in Parkinson’s disease. Journal of the Neurological Sciences, 310(1–2), 172–175. doi: 10.1016/j.jns.2011.07.009
- Krasny-Pacini, A., Chevignard, M., & Evans, J. (2014). Goal management training for rehabilitation of executive functions: A systematic review of effectivness in patients with acquired brain injury. Disability and Rehabilitation, 36(2), 105–116. doi: 10.3109/09638288.2013.777807
- Krasny-Pacini, A., Limond, J., Evans, J., Hiebel, J., Bendjelida, K., & Chevignard, M. (2014). Context-sensitive goal management training for everyday executive dysfunction in children after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 29(5), E49–E64. doi: 10.1097/htr.0000000000000015
- Levaux, M. N., Larøi, F., Malmedier, M., Offerlin-Meyer, I., Danion, J. M., & Van der Linden, M. (2012). Rehabilitation of executive functions in a real-life setting: Goal management training applied to a person with schizophrenia. Case Reports in Psychiatry, 2012, Article ID 503023.
- Levine, B., Manly, T., & Robertson, I. H. (2012). Goal management training, trainer’s manual. Toronto, ON, Canada: Baycrest Centre for Geriatric Care.
- Levine, B., Robertson, I. H., Clare, L., Carter, G., Hong, J., Wilson, B. A., … Stuss, D. T. (2000). Rehabilitation of executive functioning: An experimental-clinical validation of goal management training. Journal of the International Neuropsychological Society, 6(3), 299–312.
- Levine, B., Schweizer, T. A., O’Connor, C., Turner, G., Gillingham, S., Stuss, D. T., … Robertson, I. H. (2011). Rehabilitation of executive functioning in patients with frontal lobe brain damage with goal management training. Frontiers Human Neuroscience, 5, 9. doi: 10.3389/fnhum.2011.00009
- Levine, B., Stuss, D. T., Winocur, G., Binns, M. A., Fahy, L., Mandic, M., … Robertson, I. H. (2007). Cognitive rehabilitation in the elderly: Effects on strategic behavior in relation to goal management. Journal of the International Neuropsychological Society, 13(1), 143–152. doi: 10.1017/S1355617707070178
- Lezak, M. D. (1995). Neuropsychological assessment. New York, NY: Oxford Unicversity Press.
- Mahomed, A. (2015). Rehabilitation of executive functioning following pediatric traumatic brain injury: Evaluating a goal management training intervention (Master’s). University of Cape Town, Cape Town.
- McPherson, K. M., Kayes, N., & Weatherall, M. (2009). A pilot study of self-regulation informed goal setting in people with traumatic brain injury. Clinical Rehabilitation, 23(4), 296–309. doi: 10.1177/0269215509102980
- Meltzer, E. P., Kapoor, A., Fogel, J., Elbulok-Charcape, M. M., Roth, R. M., Katz, M. J., … Rabin, L. A. (2017). Association of psychological, cognitive, and functional variables with self-reported executive functioning in a sample of nondemented community-dwelling older adults. Applied Neuropsychology: Adult, 24(4), 364–375. doi: 10.1080/23279095.2016.1185428
- Metzler-Baddeley, C., & Jones, R. W. (2010). Brief communication: Cognitive rehabilitation of executive functioning in a case of craniopharyngioma. Applied Neuropsychology, 17(4), 299–304. doi: 10.1080/09084282.2010.523394
- Miotto, E. C., Evans, J. J., Souza De Lucia, M. C., & Scaff, M. (2009). Rehabilitation of executive dysfunction: A controlled trial of an attention and problem solving treatment group. Neuropsychological Rehabilitation, 19(4), 517–540.
- Novakovic-Agopian, T., Chen, A. J. W., Rome, S., Abrams, G., Castelli, H., Rossi, A., … D’Esposito, M. (2011). Rehabilitation of executive functioning with training in attention regulation applied to individually defined goals: A pilot study bridging theory, assessment, and treatment. Journal of Head Trauma Rehabilitation, 26(5), 325–338.
- O’Connor, C., Kumar, P., Khuu, W., Grady, C., Robertson, I. H., & Levine, B. (2013). Rehabilitation of sustained attention following traumatic brain injury: Effectiveness of goal management training and associated neuroplastic changes. Paper presented at the 23rd Annual Rotman Research Institute, Toronto, ON.
- O’Connor, C., & Levine, B. (in prep). GMT in traumatic brain injury [Data set].
- Pachalska, M., Talar, J., Baranowski, P., & Macqueen, B. D. (2000). The rehabilitation of executive functions in patients with closed-head injuries. Ortopedia,Traumatologia, Rehabilitacja, 2(3), 77–87.
- Pepping, M., Brunings, J., & Goldberg, M. (2013). Cognition, cognitive dysfunction, and cognitive rehabilitation in multiple sclerosis. Physical Medicine and Rehabilitation Clinics of North America, 24(4), 663–672. doi: 10.1016/j.pmr.2013.06.009
- Richard, N. (2013). Rehabilitation of executive dysfunction in multiple sclerosis: Cognitive, behavioural and neurophysiological effects of goal management training (PhD.). University of Toronto, Toronto.
- Robertson, I. H. (1996). Goal management training: A clinical manual. Cambridge: PsyConsult.
- Robertson, I. H., & Garavan, H. (2000). Vigilant attention. In M. Gazzaniga (Ed.), The new cognitive neurosciences (2nd ed., pp. 563–578). Cambridge, Mass: MIT Press.
- Rohling, M. L., Faust, M. E., Beverly, B., & Demakis, G. (2009). Effectiveness of cognitive rehabilitation following acquired brain injury: A meta-analytic re-examination of Cicerone et al.’s (2000, 2005) systematic reviews. Neuropsychology, 23(1), 20–39. doi: 10.1037/a0013659
- Rous, R. S. (2011). Rehabilitation of prospective memory in paediatric acquired brain injury: A preliminary study. University of East Anglia.
- Schweizer, T. A., Levine, B., Rewilak, D., O’Connor, C., Turner, G., Alexander, M. P., … Stuss, D. T. (2008). Rehabilitation of executive functioning after focal damage to the cerebellum. Neurorehabilitation and Neural Repair, 22(1), 72–77. doi: 10.1177/1545968307305303
- Segal, Z. V., Williams, J. M. G., & Teasdale, J. D. (2002). Mindfulness-based cognitive therapy for depression: A new approach to preventing relapse. New York, NY: Guilford.
- Sestito, N. (2010). Improving everyday action through executive training in schizophrenia ( 3415737 Ph.D.). Drexel University, Ann Arbor.
- Spikman, J. M., Boelen, D. H., Lamberts, K. F., Brouwer, W. H., & Fasotti, L. (2010). Effects of a multifaceted treatment program for executive dysfunction after acquired brain injury on indications of executive functioning in daily life. Journal of the International Neuropsychological Society, 16(1), 118–129. doi: 10.1017/S1355617709991020
- Spikman, J. M., Boelen, D. H. E., Pijnenborg, G. H. M., Timmerman, M. E., Van Der Naalt, J., & Fasotti, L. (2013). Who benefits from treatment for executive dysfunction after brain injury? Negative effects of emotion recognition deficits. Neuropsychological Rehabilitation, 23(6), 824–845.
- Spreen, O., & Strauss, E. A. (1998). A compendium of neuropsychological tests: Administration, norms and commentary (2nd ed). New York, NY: Oxford University Press.
- Stubberud, J., Langenbahn, D., Levine, B., Stanghelle, J., & Schanke, A. K. (2013a). Goal management training improves everyday executive functioning for persons with Spina Bifida: Self-and informant reports six months post-training. Neuropsychological Rehabilitation, 24(1), 26–60. doi: 10.1080/09602011.2013.847847
- Stubberud, J., Langenbahn, D., Levine, B., Stanghelle, J., & Schanke, A. K. (2013b). Goal management training of executive functions in patients with Spina Bifida: A randomized controlled trial. Journal of the International Neuropsychological Society, 19(6), 672–685. doi: 10.1017/s1355617713000209
- Stubberud, J., Langenbahn, D., Levine, B., Stanghelle, J., & Schanke, A. K. (2014). Emotional health and coping in spina bifida following goal management training: A randomized controlled trial. Rehabilitation Psychology, 60, 1–16.
- Stuss, D. T. (2011). Functions of the frontal lobes: Relation to executive functions. Journal of the International Neuropsychological Society, 17(5), 759–765. doi: 10.1017/S1355617711000695
- Stuss, D. T., Robertson, I. H., Craik, F. I., Levine, B., Alexander, M. P., Black, S., … Winocur, G. (2007). Cognitive rehabilitation in the elderly: A randomized trial to evaluate a new protocol. Journal of the International Neuropsychological Society, 13(1), 120–131. doi: 10.1017/S1355617707070154
- Tornås, S., Løvstad, M., Solbakk, A. K., Evans, J., Endestad, T., Hol, P. K., … Stubberud, J. (2016). Rehabilitation of executive functions in patients with chronic acquired brain injury with goal management training, external cuing, and emotional regulation: A randomized controlled trial. Journal of the International Neuropsychological Society, 22(4), 436–452. doi: 10.1017/s1355617715001344
- Tornås, S., Løvstad, M., Solbakk, A. K., Schanke, A. K., & Stubberud, J. (2016). Goal management training combined with external cuing as a means to improve emotional regulation, psychological functioning, and quality of life in patients with acquired brain injury: A randomized controlled trial. Archives of Physical Medicine and Rehabilitation, 97(11), 1841–1852. doi: 10.1016/j.apmr.2016.06.014
- Valls-Serrano, C., Caracuel, A., & Verdejo-Garcia, A. (2016). Goal management training and mindfulness meditation improves executive functions and transfers to ecological tasks of daily life in polysubstance users enrolled in therapeutic community treatment. Drug and Alcohol Dependence, 165, 9–14. doi: 10.1016/j.drugalcdep.2016.04.040
- van der Hiele, K., Spliethoff-Kamminga, N. G., Ruimschotel, R. P., Middelkoop, H. A., & Visser, L. H. (2012). The relationship between self-reported executive performance and psychological characteristics in multiple sclerosis. European Journal of Neurology, 19(4), 562–569. doi: 10.1111/j.1468-1331.2011.03538.x
- van Hooren, S. A., Valentijn, S. A., Bosma, H., Ponds, R. W., van Boxtel, M. P., Levine, B., … Jolles, J. (2007). Effect of a structured course involving goal management training in older adults: A randomised controlled trial. Patient Education and Counseling, 65(2), 205–213. doi: 10.1016/j.pec.2006.07.010
- Vasquez, B. P., & Zakzanis, K. K. (2014). The neuropsychological profile of vascular cognitive impairment not demented: A meta-analysis. Journal of Neuropsychology, 9, 109–136. doi: 10.1111/jnp.12039
- Waid-Ebbs, J. K., Bcba, D., Daly, J., Wu, S. S., Berg, W. K., Bauer, R. M., … Crosson, B. (2014). Response to goal management training in veterans with blast-related mild traumatic brain injury. Journal of Rehabilitation Research and Development, 51(10), 1555–1566. doi: 10.1682/JRRD.2013.12.0266