1,970
Views
0
CrossRef citations to date
0
Altmetric
Systematic Review

Systematic review of the effectiveness of innovative, gamified interventions for cognitive training in paediatric acquired brain injury

, , , ORCID Icon &
Pages 268-299 | Received 16 Mar 2022, Accepted 24 Jan 2023, Published online: 12 Mar 2023

ABSTRACT

Effectiveness of innovative, gamified interventions (i.e., Augmented Reality, Computer-Based Cognitive Retraining [CBCR], and Virtual Reality [VR] in conjunction with a Serious Game) for cognitive training in paediatric ABI was evaluated. Studies were identified on PsycINFO, PubMed and Scopus; last searched 4 January 2022. Eligibility criteria were participants diagnosed with ABI and aged ≤ 18 years, experimental intervention to train cognition, cognition assessed pre- and post-intervention at: (1) The level of function, or (2) The level of activity, and written in English. ROB 2 and ROBINS-I were utilised to assess risk of bias. Extracted study characteristics were methods, participants, interventions, outcomes, and results. Seven studies were included, comprising six CBCR studies and one VR study, with 182 participants. Following CBCR: (1) Improvements were observed in several cognitive functions, but there was inconsistent evidence; (2) Improvements were reported in attention and executive functions (EF) at home and at school. Following VR: (1) Improvements were observed in attention and EF; (2) Not evaluated. Due to the small number of included studies with (relatively) small and heterogeneous samples, only a cautious interpretation of the evidence was provided. There is a need for carefully designed studies with more attention to inter-individual differences and generalisation to daily life.

Introduction

Cognitive impairment is highly prevalent following acquired brain injury (ABI) in children (Anderson et al., Citation2006; Babikian & Asarnow, Citation2009; Resch et al., Citation2019; Van Heugten et al., Citation2006). This may be evident across multiple cognitive domains, including attention, executive functions (EF), and memory (Anderson et al., Citation2006; Babikian & Asarnow, Citation2009; Resch et al., Citation2019; Van Heugten et al., Citation2006). These cognitive impairments are considered impaired body functions that may negatively influence the activity and participation domains, as defined in the WHO’s International Classification of Functioning Disability and Health for Children and Youth (ICF-CY, Citation2007). For example, children may have limitations in managing daily routines, restrictions in structured activities in the community, and restrictions in social play with peers at school and in the community (Bedell & Dumas, Citation2004; Van Tol et al., Citation2011). Historical perspectives, such as the “Kennard Principle” suggested children to be more resilient to brain injury and more amenable to recovery than adults (Palanivel & Burrough, Citation2021). More recent literature suggests that long-lasting cognitive impairment in children may be part of a developmental cascade that hampers academic performance and psychosocial outcomes, exacerbating long-term restrictions in daily functioning; the phenomenon known as “growing into deficits” (Lambregts et al., Citation2018). Therefore, early interventions aiming to improve a child’s recovery at the level of body function and the level of activities and participation are essential.

In order to promote recovery, researchers have explored the possibilities of gamified cognitive training in children with ABI. Computer-Based Cognitive Retraining (CBCR) aims to ameliorate cognitive impairment through intensive repetition of specific cognitive tasks (Li et al., Citation2013), typically focusing on attention, EF, information processing speed, working memory (WM) and reasoning skills (Van Heugten et al., Citation2016). CBCR can be performed in the patient’s home environment and provides tailored programs in which difficulty can be adjusted trial-by-trial (e.g., dynamic difficulty progression); therefore, continuously challenging children and maximising cognitive effort (Van Heugten et al., Citation2016). More recently, innovative (i.e., a new approach that has emerged in the published literature over the last two decades) technologies, such as Augmented Reality (AR) and immersive Virtual Reality (iVR), have gained increased attention. VR is typically defined as a “computer-generated interactive virtual world that simulates the real world” (Huygelier et al., Citation2021). VR encompasses a wide variety of technologies, from semi-iVR featuring 2D displays with a limited field of view (i.e., window on the world displays) to iVR (i.e., head-mounted 3D displays), which provides a 360° field of view (Huygelier et al., Citation2021). In AR, virtual 3D objects can be projected on the real world (Huygelier et al., Citation2021), which offers the opportunity to interact with the real environment. AR and VR can deliver enhanced feelings of engagement, transport and presence (Cipresso et al., Citation2018; Laver et al., Citation2017; Spreij et al., Citation2020), which may facilitate treatment motivation (Huygelier et al., Citation2020). Enhancing treatment motivation may also be achieved by introducing game aspects into cognitive rehabilitation (Huygelier et al., Citation2020; Vermeir et al., Citation2020). Serious Gaming refers to the gamification of tasks (e.g., goal setting, narrative, feedback and rewards) used in nonentertainment settings (Vermeir et al., Citation2020). However, there is still no consensus on the definition of a “game” or a clear perspective on the main characteristics that make a game enjoyable, motivating or even addictive (Bedwell et al., Citation2012).

In contrast to CBCR, VR allows the development of more ecologically valid environments (i.e., complex, dynamic, interactive 3D scenarios), offering the possibility to intensively train complex cognitive skills in an enriched, multisensory environment which elicits more natural behaviour and cognition (Bohil et al., Citation2011; Huygelier et al., Citation2020; Krohn et al., Citation2020; Parsons, Citation2015; Rizzo et al., Citation2004; Spreij et al., Citation2020). VR training can target an array of cognitive domains including attention, EF, and memory. Similar to CBCR, VR can be performed in the patient’s home and tailored to a patient’s individual level of functioning. VR allows more control over stimulus presentation (i.e., audio-visual-tactile stimuli; Foerster et al., Citation2016), whilst capturing precise and detailed performance measures due to continuous data acquisition; therefore, offering real-time feedback (Rizzo et al., Citation2004; Spreij et al., Citation2020).

Reviews on cognitive interventions in paediatric ABI and practice recommendations for effective intervention in paediatric ABI are limited (Laatsch et al., Citation2007; Resch et al., Citation2018). A previous review included six studies evaluating CBCR (i.e., Captain’s Log, Cogmed™ WM training, Foramen Rehab®, Nintendo Wii™, and TEACHware™; Resch et al., Citation2018). Following CBCR, improvements were demonstrated in trained tasks or cognitive tasks similar to the trained tasks in the intervention. VR was not included.

The majority of meta-analyses and systematic reviews on VR and Serious Games in a range of neurodevelopmental conditions, published between 2009 and 2015, focused on motor outcomes (i.e., nine of 12 published systematic reviews; Deutsch & McCoy, Citation2017). Children with cerebral palsy represented the most prevalent participants (i.e., 70% of participants); the remaining participants included Down’s syndrome and coordination disorders (Deutsch & McCoy, Citation2017). A recent “umbrella” review evaluated the effectiveness and quality of evidence of VR-based cognitive and physical interventions in patients with cerebral palsy, stroke, and traumatic brain injury (TBI; Voinescu et al., Citation2021). Six of the 41 included meta-analyses included children, specifically with cerebral palsy (Chen et al., Citation2014; Citation2018; Ghai & Ghai, Citation2019; Johansen et al., Citation2020; Mekbib et al., Citation2020; Rutkowski et al., Citation2020; Warnier et al., Citation2020; Wiley et al., Citation2020; Wu et al., Citation2019). Therefore, no meta-analysis included children with ABI. Moreover, the effectiveness of VR-based interventions on improving cognitive functioning (e.g., overall cognition) was not evaluated.

To the best of our knowledge, there is no state-of-the-art review specifically examining the effectiveness of innovative, gamified interventions for cognitive training in paediatric ABI. The overarching aim of the present review was; therefore, to comprehensively evaluate the effectiveness of innovative, gamified interventions (i.e., AR, CBCR, and VR in conjunction with a Serious Game), for cognitive training in paediatric ABI. The sub-aims were three-fold: (1) To describe the technology-based interventions most frequently reported, (2) To evaluate the effectiveness of gamified technology-based interventions at the level of body function (i.e., cognition), and (3) To evaluate the effectiveness of gamified technology-based interventions at the level of activities, such as acquiring skills, carrying out daily routines, and undertaking multiple tasks.

Methods

Eligibility criteria

Studies were included when the following criteria were met: (1) Participantcharacteristics: participants were diagnosed with ABI and aged ≤ 18 years. If an article reported results from participants with different neurological conditions (i.e., non-ABI) or participants older than 18 years, it was included only if the data from the ABI participants or the participants younger than 18 years were reported separately or if the majority of the participants had been diagnosed with ABI or younger than 18 years; (2) Intervention characteristics: experimental intervention (i.e., AR, CBCR, or VR in conjunction with a Serious Game) to train cognitive function vs. a comparator (e.g., no intervention, wait-list control, usual care); (3) Outcomes: assessed cognition pre- and post-intervention at the level of function (i.e., academic- and cognitive capacity – what an individual can do in a standardised, controlled environment; Holsbeeke et al., Citation2009) and, if included, at the level of activities (i.e., academic- and cognitive capability – what an individual can do in their daily environment; Holsbeeke et al., Citation2009); (4) Study design: no specific criteria; (5) Written in English.

Search methods for identification of studies

Electronic searches

Innovative, gamified intervention studies for cognitive training in children with ABI were identified on PsycINFO, PubMed and Scopus, using relevant keywords appropriate to each resource. For an overview of the relevant keywords utilised for the search string, see Appendix 1. Searches were initially conducted by two authors (C.L.W and M.V) on 23 December 2020, updated 20 April 2021, and updated 4 January 2022. Restrictions were applied on language (English only), year (2000 – current) and age (0-18 years).

Reference lists

Reference lists of all included studies were manually checked to identify additional studies missed from the original electronic searches.

Data collection and analysis

Selection of studies

Two authors (C.L.W and M.V) screened the titles and abstracts of all studies identified as a result of the search, coding them as “retrieve” (eligible or potentially eligible/unclear) or “do not retrieve.” Full-text study reports/publications were retrieved, and the two authors independently screened the full texts to identify studies for inclusion, identifying and recording reasons for exclusion of the ineligible studies. Disagreements were resolved through discussion. Duplicate records were identified and collated. Selection process is documented in the PRISMA flow diagram (Moher et al., Citation2015).

Data extraction and management

C.L.W extracted the following study characteristics: (1) Methods: date of study, study design, recorded time points, and type of study (i.e., efficacy or feasibility); (2) Participants: aetiology, age, age ofonset/time post-injury, number of participants, and sex; (3) Interventions: comparator, intervention, setting, and targeted cognitive domain; (4) Outcomes: academic- and cognitive capacity (i.e., neuropsychological assessment [NPA]; a conventional set of paper-and-pencil tests used in a clinical setting to estimate the maximum capacity of a patient’s cognitive functions), academic- and cognitive capability (i.e., self/parent reported outcomes on cognition in daily life); (5) Results: effect size , statistical significance (p < 0.05 or p < 0.001), and training completion rate; (6) Funding. Following this, M.V checked that data were entered correctly by comparing data presented in the systematic review with all study reports. Study characteristics were spot-checked for accuracy against the study report. For an overview of the characteristics of the included studies, see and .

Table 1. Summary of study characteristics (level of function).

Table 2. Summary of study characteristics (level of activities).

Assessment of risk of bias

C.L.W and M.V independently assessed the risk of bias for each study using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., Citation2019). The Revised Cochrane risk-of-bias for randomised trials (ROB 2) was utilised to judge the risk of bias in the findings of the randomised trials. The Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I) was utilised to judge the risk of bias in findings of non-randomised trials. Any disagreements were resolved by discussion or by involving another review author (T.C.W.N). Risk of bias was judged according to the following domains: (1) Bias arising from the randomisation process; (2) Bias due to deviations from the intended interventions (effect of assignment to intervention); (3) Bias due to missing outcome data; (4) Bias in measurement of the outcome; (5) Bias in selection of the reported result. Signalling questions within each domain provided the basis for domain-level judgements about the risk of bias. According to the ROB-2, the possible risk of bias judgements are: low, some concerns and high. According to the ROBINS-I, the possible risk of bias judgements are: low, moderate and serious. In the present review, the risk of bias was referred to as: low, moderate and severe.

Results

A total of 473 studies were identified as a result of the electronic search; a further 29 studies were identified through manual searches of references (n = 28) or Google Scholar (n = 1). After removing duplicates, 464 studies were screened on the basis of titles and abstracts, excluding 410 studies. A total of 54 studies were analysed on the basis of full text. Subsequently, seven studies with a total of 182 participants met the eligibility criteria and were included in the present review, consisting of six CBCR studies identified as a result of the electronic search and one iVR study identified via Google Scholar. No studies were identified that evaluated the effectiveness of AR. For an overview of the study selection process and reasons for exclusion, see the PRISMA flow diagram in (Moher et al., Citation2015).

Figure 1. Flow diagram of study selection in accordance with the PRISMA statement (Moher et al., Citation2015).

Flow diagram portraying study selection (i.e., identification, screening, eligibility and included).
Figure 1. Flow diagram of study selection in accordance with the PRISMA statement (Moher et al., Citation2015).

Description of included studies

For an overview of the study characteristics of the included studies, see and .

Design

Three studies were RCTs (Individually randomised parallel-group trials), and four studies were non-RCT (Observational [case study and case-series] or single-group experimental design). Regarding the type of study, four focused on efficacy (n = 149 participants), two studies focused on efficacy and feasibility (n = 28 participants), and one study focused on feasibility only (n = 5 participants). An efficacy study evaluated iVR.

Six of the seven studies in the present review criticised their (relatively) small sample (Carlson-Green et al., Citation2017; Corti et al., Citation2020; De Luca et al., Citation2020; Eve et al., Citation2016; Phillips et al., Citation2016; Verhelst et al., Citation2017), with three studies discussing the limited power (Corti et al., Citation2020; Eve et al., Citation2016; Verhelst et al., Citation2017). Therefore, the chance of a Type II error (i.e., no effect in the population, whilst in fact an effect is present) increased.

Participants

Included studies involved participants of varied ages (8–18 years) with ABI due to the following underlying conditions: trauma (n = 5), cardiovascular injury (n = 3), brain tumour (n = 1), hypoxic-anoxic brain injury (n = 1), and other non-traumatic ABI (i.e., no further information provided; n = 1). Participants were recruited from: hospital (n = 4), rehabilitation setting (n = 3), outreach facilities (n = 1), or the Brain Damage Registry of Scientific Institute (n = 1). The CBCR studies involved participants in the chronic phase of rehabilitation (i.e., at least one-year post-ABI), whereas the iVR study involved one participant in the subacute phase of rehabilitation (i.e., at least one-month post-ABI).

Outcome measures

With regard to the level of function, NPA was employed to assess individual cognitive domains, employing conventional paper-and-pencil tests (n = 7) or the Luminosity Performance index (i.e., age-adjusted measure of improvement for Lumosity Cognitive Training™; n = 1). The most commonly assessed cognitive domain was memory (n = 5). Other cognitive domains assessed were attention (n = 3), EF (n = 3), and processing speed (n = 1). Four studies assessed academic capacity. In some cases, the same tasks were considered to measure attention of EF by different authors (e.g., the TEA-Ch).

With regard to the level of activities, three studies employed the Behaviour Rating Inventory of EF (the BRIEF) to assess cognition in daily life (i.e., at home and school), utilising parent-reported outcomes (n = 2) or self- and parent-reported outcomes (n = 1).

Interventions

The most frequently used intervention was CBCR (n = 6), specifically Cogmed™ WM Training (2D computer; n = 3). Other CBCR interventions used were BrainGames (2D computer; n = 1), Lumosity Cognitive Training™ (2D computer; n = 1) and Move it to improve it (Mitii™ - 2D computer and Microsoft Kinect®; n = 1). One study used a CAVE iVR system (3D), specifically Computer Assisted Rehabilitation Environment (CAREN).

Duration (and the number of sessions) of the interventions varied considerably in the included studies, ranging from four weeks (20 sessions; n = 1), ≥ five to ≤ eight weeks (25–40 sessions; n = 4), ≤ 12 weeks (35 sessions; n = 1) and 20 weeks (120 sessions; n = 1). In the majority of the included studies (n = 5), participants in the intervention group had a mean training completion of > 90% (n = 5). In another study, participants completed a mean of 29%; the proposed training dose was 60 hours over 20 weeks, whereas children in the intervention group completed a mean of 17.6 (SD = 14.9) hours. One study did not report training completion rate. For an overview of the completion rate, see and .

Table 3. Summary of findings (level of function).

Table 4. Summary of findings (level of activities).

As comparator, the RCTs utilised either an active control (non-adaptive: n = 1) or passive control (wait-list: n = 1; waiting-first/training-first: n = 1). The non-adaptive training (i.e., Placebo WM training) provided the same exercises as the intervention, but the difficulty never increased. The observational case study utilised an active control (adaptive); conventional cognitive therapy delivered in a clinical setting (i.e., face-to-face and paper-and-pencil approach) and adapted over time to the needs and performance of the participant. The active control was presented first, followed by a three-week stay in which the participant participated in conventional physiotherapy, before the active intervention was presented.

Personnel were blinded in two RCTs (Corti et al., Citation2020 [specifically outcome assessors]; Phillips et al., Citation2016) and one single case study (De Luca et al., Citation2020), whilst two RCTs (Corti et al., Citation2020; Phillips et al., Citation2016) reported that the participants were blinded. No study checked the success of the blindness manipulation.

Funding

Funding The majority of included studies were financially supported (n = 5), one study stated no financial support (De Luca et al., Citation2020) and the other did not state anything (Eve et al., Citation2016). For an overview of the acquisition/support of funding, see See Appendix 2 for an overview of the acquisition/support of funding.

Effect of interventions

For an overview of the findings of the included studies, see and .

Computer-based cognitive retraining

Working memory training. One RCT evaluated the efficacy of Cogmed™ WM training in TBI (Phillips et al., Citation2016). A significant improvement was demonstrated on a single measure of visual WM (i.e., Dot Matrix of the Automated WM Assessment [AWMA]), when compared with the active control group (i.e., modified, non-adaptive Cogmed WM training). At the three-month follow-up, improvements were sustained. No significant group differences were observed on measures of attention and other measures of WM (i.e., Counting Recall [verbal WM], Digit Recall [verbal WM], and Mr. X [visuospatial WM] of the AWMA). Significant improvements were observed on select academic capacity measures post-training (i.e., Reading Comprehension of the Wechsler Individual Achievement Test, 2nd Edition [WIAT-II]) and at the three-month follow-up (i.e., Word Reading of the WIAT-II). Limitations of this study concerned the design (i.e., small sample size [<16 per group; McIntosh & Rittmo, Citation2020]) and the results, namely the internal validity (i.e., repeated measurements; the testing effect). Phillips et al. (Citation2016) was considered a moderate risk of bias.

Two single-group, pilot studies evaluated the efficacy and feasibility of Cogmed™ WM training in childhood arterial ischemic stroke (AIS; Eve et al., Citation2016) and survivors of paediatric cancer (Carlson-Green et al., Citation2017); participants were diagnosed with brain tumours (i.e., medulloblastoma [n = 11], ependymoma [n = 4], germinoma [n = 4], and other tumour types [n = 2]).

Considering children with AIS, significant improvements were observed on most measures of verbal WM (i.e., Digit Recall, Word List Matching, Word List Recall, and Non-word list Recall of the WM Test Battery for Children; Eve et al., Citation2016). However, improvements were not sustained at the 12-month follow-up. There were no significant improvements on measures of attention, academic capacity or other measures of WM (i.e., Block Recall [visuospatial WM], Listening Recall [verbal WM], and Backward Digit Recall [verbal WM] of the WM Test Battery for Children). Cogmed™ WM training was not recommended in its current form as an effective intervention for children with AIS. Limitations of this study concerned the design (e.g., single group and small sample size) and the results, namely the internal validity (e.g., no control group, maturation and repeated measurements; the testing effect) and the external validity (e.g., generalisability to the larger, target population). Eve et al. (Citation2016) was considered a severe risk of bias.

Regarding paediatric cancer survivors, although reported to be measured post-training (i.e., after 8–12 weeks), no results were reported for cognition at this time point (Carlson-Green et al., Citation2017). Nevertheless, significant improvements were reported at the six-month follow-up on measures of verbal and visual WM (i.e., Digit Recall, Word Recall, Dot Matrix, Block Recall, Mr. X, Spatial Recall of the AWMA), as well as select academic measures (i.e., Applied Problems of the WJ-III). With regard to the level of activities, parents reported significant improvements in their child’s attentional problems and EF at home (i.e., the BRIEF); for example, ability to inhibit responses, monitor behaviour, shift between activities, WM, and planning and organisational skills. Limitations of this study concerned the design (e.g., single-group) and the results, namely the internal validity (e.g., no control group and use of parent-reported outcomes) and the external validity (e.g., selection of participants with a WM impairment; generalisation to the larger, target population). Carlson-Green et al. (Citation2017) was considered a severe risk of bias.

In summary, Cogmed™ WM training demonstrated a significant medium-term improvement (i.e., 7–23 weeks; Ashton et al., Citation2020) on a single measure of visual WM (i.e., near-transfer effect) and on select measures of academic capacity (i.e., far-transfer effects) in paediatric TBI. However, the training did not generalise to performance on untrained tasks; therefore, it provided very weak evidence that Cogmed™ WM training improves WM. As for the efficacy of Cogmed™ WM training in paediatric AIS, significant short-term improvements (i.e., ≤6 weeks; Ashton et al., Citation2020) were demonstrated on verbal WM tasks dissimilar to the trained WM tasks (i.e., near-transfer effects). There were no significant improvements at the 12-month follow-up, therefore, Cogmed™ WM training was not recommended as an effective intervention for this patient group. In paediatric cancer survivors, significant long-term improvements (i.e., ≥24 weeks; Ashton et al., Citation2020) were demonstrated on a verbal WM task that differed from the trained tasks and on more similar visual WM tasks (i.e., near-transfer effects), as well as select academic capacity measures (i.e., far-transfer effects). With regard to the level of activities, parents reported significant improvements in their child’s attentional problems and EF at home. Although improvements in academic capacity and WM were demonstrated, strong inferences regarding the efficacy of Cogmed™ WM training could not be drawn due to the limited number of studies and the moderate- to severe risk of bias.

Executive functions training. One RCT evaluated the effectiveness of Mitii™, a web-based, multi-modal rehabilitation program to improve EF in children and adolescents with ABI (Piovesana et al., Citation2017). No significant differences in EF were observed post-training compared to the waitlist control group. With regard to the level of activities, there were no group differences observed for daily EF (i.e., the BRIEF). However, program adherence was poor. Mitii™ was not recommended in its current form as an alternative to traditional cognitive rehabilitation for children with ABI. The main limitation of this study concerned the results, namely the internal validity (e.g., no active control group, use of parent-reported outcomes and repeated measurements; the testing effect). Piovesana et al. (Citation2017) was considered a moderate risk of bias.

General cognitive functions training. Other training programs did not target a specific cognitive domain, such as BrainGames and Lumosity Cognitive Training™.

One RCT evaluated the efficacy of Lumosity Cognitive Training™, a multi-domain, computerised cognitive training program in children with ABI (Corti et al., Citation2020). A significant improvement in visual WM (i.e., Corsi Block-Tapping test; near-transfer effect) was observed post-training and at the two-month follow-up for the training-first group, compared to the control group (i.e., waiting-first). In addition, a significant improvement on an academic measure of arithmetic calculation speed (i.e., Arithmetic Calculation task of the Italian battery AC-MT; near-transfer effect) was observed post-training, but not maintained at the two-month follow-up. No other significant effects were demonstrated (i.e., arithmetic calculation accuracy, cognitive flexibility, and mathematic problem-solving). Limitations of this study concerned the design (e.g., small control sample size) and the results, namely the internal validity (e.g., no active control group and repeated measurements; testing effect). Corti et al. (Citation2020) was considered a moderate risk of bias.

A longitudinal, pilot study evaluated the feasibility of BrainGames, a home-based, computerised cognitive training program, in adolescents with TBI (Verhelst et al., Citation2017). Large effect sizes were demonstrated post-training in measures of attention (i.e., Continuous Performance Test and Flanker task of the Psychology Experiment Building Language battery; near-transfer effects) and a single measure of EF (i.e., Stockings of Cambridge of the Cambridge Automated Neuropsychological Test Battery [CANTAB]; near-transfer effect). A small effect size was reported for verbal WM (i.e., near-transfer effect). At the six-month follow-up, large effect sizes were demonstrated in attention (i.e., Continuous Performance Test and Flanker task; near-transfer effects), EF (i.e., Stockings of Cambridge and Intra-Extradimensional Set Shift of the CANTAB; near-transfer effects), verbal WM (i.e., Digit Span of the Dutch Wechsler Intelligence Scale for Children, 3rd Edition [WISC-III-NL]; near-transfer effect), and visual WM (i.e., Spatial Span of the WISC-III-NL; near-transfer effect). Note that, whilst superior to baseline, effect sizes decreased for the attentional measures. Moreover, only a small effect size was reported for processing speed (i.e., Digit Symbol Coding of the WISC-III-NL; near-transfer effect). With regard to the level of activities, self- and parent-reported outcomes on the BRIEF revealed small post-training effects, with the exception of parent-reported Behavioural Regulation Index (e.g., cognitive flexibility, inhibitory control, and self-monitoring) and Global Executive Composite (i.e., summary score). Scores on the BRIEF improved at the six-month follow-up, which is reflected in medium to large effect sizes. Limitations of this study concerned the design (e.g., single-group and small sample size) and results, namely the internal validity (e.g., no control group, use of self- and parent-reported outcomes, and repeated measurements; testing effect) and the external validity (e.g., generalisability to the larger, target population). Verhelst et al. (Citation2017) was considered a moderate risk of bias.

In summary, Lumosity Cognitive Training™ demonstrated a significant medium-term improvement on a single measure of visual WM (i.e., near-transfer effect) in children with ABI. In adolescents with TBI, BrainGames demonstrated medium- and long-term improvements in attention, EF and WM (i.e., near-transfer effects). With regard to the level of activities, children and their parents reported improvements in attentional problems and EF at home and school. Although the two studies demonstrated medium- and long-term improvements in general cognitive functioning, strong conclusions cannot be drawn due to the limited number of studies and the moderate risk of bias.

Virtual Reality training

One case study evaluated the efficacy of two rehabilitation training programs, including conventional cognitive therapy (CCT) and CCT in a Computer Assisted Rehabilitation Environment (CAREN) in a 15-year-old boy with severe TBI (De Luca et al., Citation2020). Following four weeks of CCT (20 sessions), improvements in attentional (i.e., the Modified Bell Cancellation Test) and executive processes (i.e., the WEIGL test) were observed, however were non-significant. Following four weeks of CCT in CAREN (20 sessions), improvements in attention and executive processes were significant. De Luca et al. (Citation2020) employed the reliable change index to evaluate significance (i.e., RCI ≥ 1.96). The limitations of this study concerned the design (e.g., single-case A-B-A-B design, repeated-measures) and results, namely the internal validity (e.g., no control group and repeated measurements; the testing effect) and external validity (e.g., generalisability to the larger, target population and multiple treatment inferences). De Luca et al. (Citation2020) was judged to be at severe risk of bias.

In summary, CCT in an iVR environment may hold promise for improving attention and executive processes at the ICF level of body function in paediatric TBI. However, to prevent erroneous conclusions, well-designed clinical studies with a low- to moderate risk of bias are needed to corroborate these results. Refer to and for an overview of the findings.

Risk of bias in included studies

The included studies were rated as a moderate (n = 4) and severe (n = 3) risk of bias, the latter attributable to: (1) Selection bias and confounding, namely the potential for confounding of the effect of intervention (i.e., severity of disease); (2) Reporting bias, namely that the outcome measurements and analyses were defined in different ways in the methods and results sections. See and for an overview and summary of risk of bias for the included studies.

Figure 2. Overview of risk of bias for the randomised controlled trials (ROB 2).

Note: D2 (Bias due to deviations from the intended intervention) – intervention effect of interest to the review author was the effect of assignment to the intervention at baseline (i.e., the “intention-to-treat effect”).

A 7 × 4 table portraying the risk of bias for the three randomized controlled trials. Risk of bias rated as some concerns.
Figure 2. Overview of risk of bias for the randomised controlled trials (ROB 2).Note: D2 (Bias due to deviations from the intended intervention) – intervention effect of interest to the review author was the effect of assignment to the intervention at baseline (i.e., the “intention-to-treat effect”).

Figure 3. Overview of risk of bias for the non-randomised controlled trials (ROBINS).

Note: D3 (Bias in classification of interventions) not included due to single condition; D4 (Bias due to deviations from the intended intervention) – intervention effect of interest to the review author was the effect of assignment to the intervention at baseline (i.e., the “intention-to-treat effect”).

A 9 × 5 table portraying the risk of bias for the four non-randomized controlled trials. Three studies rated the risk of bias rated as serious. One study rated the risk of bias as moderate.
Figure 3. Overview of risk of bias for the non-randomised controlled trials (ROBINS).Note: D3 (Bias in classification of interventions) not included due to single condition; D4 (Bias due to deviations from the intended intervention) – intervention effect of interest to the review author was the effect of assignment to the intervention at baseline (i.e., the “intention-to-treat effect”).

Discussion

The overarching aim of the present review was to comprehensively evaluate the effectiveness of innovative, gamified interventions (i.e., AR, CBCR, and VR in conjunction with a Serious Game) for cognitive training in paediatric ABI at the level of function (i.e., NPA) and then at the level of activities (i.e., self/parent-reported outcomes on cognition in daily life). Findings were based on seven studies, published between 2016 and 2020. Training focused on remediation of general cognitive functioning (three studies), WM (three studies), or EF (one study).

Level of function

Computer-based cognitive retraining

The present review revealed no consistent evidence that CBCR was effective at the level of function; therefore, there was no clear pattern to explain the inconsistencies in results between studies. Despite the lack of improvement on most measures in individual studies, general improvements were observed in academic capacity, EF, general cognitive functioning (i.e., attention and processing speed), and WM. Importantly, there was evidence demonstrating that CBCR can have medium to long-term training-related effects, as the majority of studies that reported post-training improvements also reported improvements at follow-up (Corti et al., Citation2020; Phillips et al., Citation2016; Verhelst et al., Citation2017). Moreover, there was evidence that CBCR targeting a specific cognitive domain can positively influence performance in tasks dissimilar to the trained tasks, suggesting that far-transfer is possible. That is, improvements in academic capacity (i.e., mathematics and reading) were observed following Cogmed™ WM Training.

Lastly, training targeting general cognitive functioning appeared more promising than training targeting a specific cognitive domain. However, this was strongly driven by one study examining the feasibility of BrainGames (Verhelst et al., Citation2017), which demonstrated more widespread improvements in different cognitive domains. The improvements evidenced on the NPA may be due to task-specific practice effects, as the majority of minigames in BrainGames were adaptations of well-known cognitive tasks or existing cognitive training material (Verhelst et al., Citation2017); therefore, resulting in near-transfer effects.

Strong conclusions could not be drawn regarding the effectiveness of CBCR in improving cognitive functioning at the level of function, as there were no studies with a low risk of bias. Four studies (Corti et al., Citation2020; Phillips et al., Citation2016; Piovesana et al., Citation2017; Verhelst et al., Citation2017) were considered a moderate risk of bias, whilst two studies (Carlson-Green et al., Citation2017; Eve et al., Citation2016) were considered severe.

Virtual Reality

One single case study explored the efficacy of a CAVE iVR system (i.e., CAREN) in contrast to CCT in an adolescent with severe TBI (De Luca et al., Citation2020). At the level of function, significant improvements were demonstrated in attention and executive processes following iVR training, but not following CCT. De Luca et al. (Citation2020) was considered at severe risk of bias. One of the methodological limitations concerned the fact that iVR training followed CCT; therefore, it was difficult to disentangle the effect of the active control condition from the benefit of the intervention (i.e., the carry-over effect). Therefore, the effect of iVR training for children with ABI remains unclear.

Level of activities

Three studies included an outcome measure at the level of activities (i.e., the BRIEF; Carlson-Green et al., Citation2017; Piovesana et al., Citation2017; Verhelst et al., Citation2017). Two studies revealed long-term improvements in attention problems and EF at home or school (i.e., the BRIEF: initiation, planning/organisation, task-monitoring, and WM; far-transfer effects) following CBCR training. There did not appear to be discernible differences between training targeting general cognitive functioning (i.e., BrainGames) or training targeting a specific cognitive domain (i.e., Memory; Cogmed™ WM Training). Despite the encouraging findings, the BRIEF focuses on subjective cognitive impairment, which may be biased as a result of the consequences of brain injury, or influenced by environmental and personal factors (Domensino et al., Citation2022). Underreporting of symptoms may be explained by a lack of knowledge about the invisible sequelae of brain injury among family members (Hochstenbach et al., Citation2005; Spreij et al., Citation2020). In addition, mild consequences of brain injury and the impact of brain injury on daily life may not be recognised or understood, leading to an overestimation of patients’ abilities (Fordyce & Roueche, Citation1986; Hochstenbach et al., Citation2005; Spreij et al., Citation2020). Therefore, findings should be interpreted with caution. No study evaluated the effectiveness of VR training at the level of activities.

Considerations for interpretation

Outcome measures

A critical comparison of results between studies was difficult due to the choice of outcome measure. For example, the most commonly measured cognitive domain was visual WM (five studies), however only a few studies utilised similar tasks to measure the same construct. The majority of included studies (Carlson-Green et al., Citation2017; Corti et al., Citation2020; De Luca et al., Citation2020; Verhelst et al., Citation2017) utilised a limited number of outcome measures (i.e., one or two tasks) per cognitive domain. For example, a RCT and a case series study employed only one or two tasks to assess EF (Corti et al., Citation2020; Verhelst et al., Citation2017; respectively). EF is a broad concept, encompassing a multitude of cognitive functions (e.g., cognitive flexibility, inhibition, and WM; Miyake et al., Citation2000); therefore, the limited number of outcome measures may be insufficient to identify true training-related effects (Van de Ven et al., Citation2016). In addition, an improvement observed on a single outcome measure tapping into a cognitive domain is less meaningful than improvements demonstrated on multiple outcome measures tapping into the same cognitive domain (Slagter, Citation2012; Van de Ven et al., Citation2017).

Conversely, three studies may have exposed their clinical cohort to an extended range of outcome measures (Corti et al., Citation2020; Eve et al., Citation2016; Verhelst et al., Citation2017). For example, the single-group study aimed to investigate whether there would be measurable and sustained changes in untrained WM tasks in children with AIS (Eve et al., Citation2016). Therefore, the hypothesis (i.e., improvements in “cognitive function”) was not justified and an extended range of outcome measures were included (e.g., academic capacity- and attention measures). The RCT explored the training effects on a myriad of secondary outcome measures (Corti et al., Citation2020). Moreover, the case series study aimed to investigate the feasibility of the intervention (Verhelst et al., Citation2017); therefore, the inclusion of an extended range of cognitive outcome measures was not justified. No study corrected for multiple comparisons. An extended range of outcome measures are subject to observer bias, increasing the likelihood of Type I error (i.e., an effect in the population, whilst in fact no effect is present) through use of multiple comparisons across a range of outcomes (Mascha & Vetter, Citation2018 ).

Lack of (active) control group

A critical comparison of results between studies was difficult due to the lack of (active) control groups. In total, four studies included a control group; only one study included an active control condition (Phillips et al., Citation2016). However, Phillips et al. (Citation2016) employed a low-level, non-adaptive variant of the WM intervention. This may not have appropriately controlled for the effects of expectancy and motivation (Jones et al., Citation2020); children in the intervention group were provided with feedback and continuously challenged, which may have biased their performance at assessment (Jones et al., Citation2020). Nevertheless, the children in the active control group were supported with training aids and received the same support from coaches as children in the intervention group (Phillips et al., Citation2016). In order to achieve similar expectations and motivation at assessment (i.e., minimising potential biases), children in the active control group should be appropriately challenged and observe improvements (Jones et al., Citation2020).

The likelihood that findings were merely due to practice effects increases when there is no active control group. With the exception of two studies (Carlson-Green et al., Citation2017; Eve et al., Citation2016), all studies included in this review utilised the same task pre- and post-intervention. A case series study reported that improvements at follow-up were somewhat unexpected and may be attributable to maturation or test-retest effects (Verhelst et al., Citation2017). Conversely, maturation may explain the lack of improvement at follow-up. For example, a single-group study reported no significant improvements at 12-month follow-up (Eve et al., Citation2016). It is plausible that the CBCR did not have the same benefit at this stage of the child’s developmental trajectory and, therefore, reflects a plateau in cognitive capacity as other cognitive functions developed (Limond et al., Citation2014). Multiple baseline testing (i.e., sequentially introducing the intervention) may provide insight into the effectiveness of the intervention and minimise the effects of spontaneous recovery (Krasny-Pacini et al., Citation2014).

Study population

The majority of participants were at least one-year post-ABI (i.e., the chronic phase); therefore, it was not possible to ascertain whether positive training-related effects were dependent on the phase post-ABI. The effectiveness of an intervention may vary depending on the rehabilitation phase the participants were in at the time of intervention (Resch et al., Citation2020). This highlighted the general issue within the field; there is a paucity of studies systematically evaluating the effectiveness of cognitive training depending on the post-ABI phase, and the clinical question of when to train and what dosage to use.

Future research

In order to make strong inferences regarding the effectiveness of an intervention, it is essential that an active control group is employed (Van de Ven et al., Citation2016). Therefore, future studies should include both an active (e.g., adaptive visual search training; Jones et al., Citation2020) and passive (e.g., wait list) control group to control for placebo effects and the Hawthorne effect (i.e., possible expectation bias; Van de Ven et al., Citation2016). Moreover, an active control group would facilitate blinding of participants and personnel, minimisingpotential biases due to deviations from intended interventions and ensuring reliable assessment of outcomes (Van de Ven et al., Citation2016).

To facilitate a more meaningful comparison of results between studies, future research should include an agreed common core set of outcome measures to critically compare the effectiveness of different cognitive interventions (Reijnders et al., Citation2013). As a first step towards agreement on outcome measures, a recent study reviewed the most commonly used outcome measures with reference to the ICF framework in studies evaluating the outcome of neuropsychological interventions in adults with ABI over the past 20 years (Van Heugten et al., Citation2020). This study indicated the need to evaluate the quality of outcome measures, employ Delphi or consensus procedures, and involve rehabilitation professionals in the choice of outcome measures. However, the results from the aforementioned study in adults with ABI cannot be directly applied to paediatric ABI, as different outcome measures are required for this population. Therefore, future research should review the most commonly used outcome measures with reference to the ICF framework in studies evaluating the outcome of cognitive training in children with ABI over the past 20 years.

To bridge the gap between estimating cognitive functioning at the level of function and at the level of activities, future research should consider other, more ecologically valid, outcome measures (Green et al., Citation2019 ; e.g., a digital NPA; Spreij et al., Citation2021a or questionnaire; Spreij et al., Citation2021b). For example, the BRIEF (i.e., the questionnaire employed by three studies in the present review); a parent/guardian or self-reported outcome measure that focuses on a child’s behaviour in relation to daily EF in a natural setting (Hypher et al., Citation2019; Van Heugten et al., Citation2020; Vriezen & Pigott, Citation2010). Compared to performance-based measures (e.g., a NPA), which aim to estimate underlying cognitive skills, the BRIEF evaluates the application of cognitive skills at home and at school (McAuley et al., Citation2010). Therefore, the BRIEF is considered an ecologically valid measure of the subjective impact of cognitive impairment on performing activities of daily living (e.g., “trouble concentrating during play”).

Future research should; therefore, include a sufficiently large and varied battery of tasks to extract the cognitive domain (Green et al., Citation2019), whilst also considering the potential problem of multiple comparisons. Moreover, the use of composite scores could be considered, as they may be more sensitive in revealing training effects that consistently improve a specific domain, not just task-specific skills (Van de Ven et al., Citation2017).

Clinical implementation

The present review revealed insufficient evidence to provide clear recommendations for clinicians on whether to implement innovative gamified technology in paediatric clinical practice. That is, there were few studies that addressed this and the majority of studies were considered at moderate to severe risk of bias. Moreover, it is too early to formulate very clear recommendations for direct clinical implementation due to the unknown answers to clinical questions (e.g., when to train and what dosage to use). To enhance the effectiveness of interventions, it is important to determine when (i.e., phase of rehabilitation) and for whom (i.e., subpopulation) the intervention is most appropriate (Resch et al., Citation2020). In this domain, case-series methodology has attracted considerable interest (Ralph et al., Citation2011; Schwartz & Dell, Citation2010).

Innovative gamified interventions appeared feasible for training cognition (i.e., six of the seven included studies reported completion rates of more than 92%). Therefore, as a first step towards implementing innovative gamified interventions, pilot-, single case- and case series studies should be conducted in a clinical setting (i.e., the so-called “hybrid” design; Curran et al., Citation2012; Eldh et al., Citation2017). An advantage of single case methodology is the opportunity to determine whether a specific treatment works for a specific individual (i.e., identifying individual patients’ performance patterns), which may be important given patient heterogeneity. Clinicians may observe the effects of the innovative gamified intervention on relevant outcome measures whilst collecting information on implementation, or evaluate implementation whilst observing the effect on relevant outcome measures (Eldh et al., Citation2017). Nevertheless, in studies with a single case experimental design, three direct replications of one successful trial are required for external validity (Barlow et al., Citation2009). Researchers and clinicians should consider this before implementing an intervention into clinical practice.

Strengths and limitations

A potential limitation of the present review concerns the interpretation of results, which was limited by the small sample size and heterogenous sample of the included studies.

It is important to regard the findings from this review with caution when applying findings from the individual subpopulations (e.g., AIS) to the overall ABI cohort, due to differences in aetiology and consequences of brain injury (e.g., focal lesions of ischemia vs. focal and diffuse lesions caused by TBI). For example, the present review included three studies evaluating Cogmed™ WM Training, but each study focused on a different aetiology: AIS, cancerous brain tumour and TBI. Therefore, it was difficult to draw conclusions on the effectiveness of innovative, gamified interventions for cognitive training in children with ABI.

A second potential limitation is that the present review evaluated the effectiveness of innovative, gamified interventions for cognitive training at group level; therefore, individual differences were not considered – a potential limitation of the field.

Despite these limitations, the present review contributed to a better understanding of the available evidence base for innovative, gamified interventions for cognitive training in paediatric ABI; there is currently no evidence-based recommended cognitive intervention for paediatric ABI (Resch, Citation2019).

Conclusion

The present review included six CBCR studies and one iVR study, which highlighted the paucity of research investigating the effectiveness of innovative, gamified interventions for cognitive training in children with ABI, specifically mixed reality (e.g., AR and VR). Due to the small number of included studies with (relatively) small and heterogeneous samples, only a cautious interpretation of the evidence was provided.

CBCR targeting a specific cognitive domain (i.e., WM) only demonstrated improvements in WM (i.e., near-transfer effect) and in academic capacity (i.e., far-transfer effect). CBCR targeting general cognitive functioning appeared more promising, however, this was strongly driven by one study influenced by task-specific practice effects and multiple comparisons. A iVR study demonstrated improvements in EF and general cognitive functioning (i.e., attention), but the evidence was very weak. In two studies, CBCR demonstrated an improvement in cognition in daily life at home or at school. No study evaluated the effectiveness of iVR training at the level of activities.

Considering the investment of “time and effort,” stand-alone CBCR may not provide incremental benefits for children with ABI that may outweigh other more useful interventions (e.g., advice on managing daily routines). Metacognitive strategy training (e.g., MetaCogmed; Jones et al., Citation2020) may be an effective addition to CBCR (Jones et al., Citation2020; Resch et al., Citation2018). There is a need for more carefully designed studies, consensus on a core set of outcome measures per cognitive domain, more attention to inter-individual differences, and for the generalisation to daily life (Van Heugten et al., Citation2020).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Stichting Hartekind (20210406); Kinderrevalidatiefonds Adriaanstichting (20210045); SVLG (JeT1923.2); Special Interest Group Social & Cognitive Modeling; Research Foundation Flanders (FWO; 1249923N).

References

  • Anderson, V. A., Catroppa, C., Dudgeon, P., Morse, S. A., Haritou, F., & Rosenfeld, J. V. (2006). Understanding predictors of functional recovery and outcome 30 months following early childhood head injury. Neuropsychology, 20(1), 42. https://doi.org/10.1037/0894-4105.20.1.42
  • Ashton, R. E., Tew, G. A., Aning, J. J., Gilbert, S. E., Lewis, L., & Saxton, J. M. (2020). Effects of short-term, medium-term and long-term resistance exercise training on cardiometabolic health outcomes in adults: Systematic review with meta-analysis. British Journal of Sports Medicine, 54(6), 341–348. https://doi.org/10.1136/bjsports-2017-098970.
  • Babikian, T., & Asarnow, R. (2009). Neurocognitive outcomes and recovery after pediatric TBI: Meta-analytic review of the literature. Neuropsychology, 23(3), 283. https://doi.org/10.1037/a0015268
  • Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single-Case Experimental Designs: Strategies for Studying Behaviour Change. Boston. MA: Pearson.
  • Bedell, G. M., & Dumas, H. M. (2004). Social participation of children and youth with acquired brain injuries discharged from inpatient rehabilitation: A follow-up study. Brain Injury, 18(1), 65–82. https://doi.org/10.1080/0269905031000110517
  • Bedwell, W. L., Pavlas, D., Heyne, K., Lazzara, E. H., & Salas, E. (2012). Toward a taxonomy linking game attributes to learning. Simulation & Gaming, 43(6), 729–760. http://doi.org/10.1177/1046878112439444
  • Bohil, C. J., Alicea, B., & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12(12), 752–762. https://doi.org/10.1038/nrn3122
  • Carlson-Green, B., Puig, J., & Bendel, A. (2017). Feasibility and efficacy of an extended trial of home-based working memory training for pediatric brain tumor survivors: A pilot study. Neuro-Oncology Practice, 4(2), 111–120. https://doi.org/10.1093/nop/npw015
  • Chen, Y., Fanchiang, H. D., & Howard, A. (2018). Effectiveness of virtual reality in children with cerebral palsy: A systematic review and meta-analysis of randomized controlled trials. Physical Therapy, 98(1), 63–77. https://doi.org/10.1093/ptj/pzx107
  • Chen, Y. P., Lee, S. Y., & Howard, A. M. (2014). Effect of virtual reality on upper extremity function in children with cerebral palsy: A meta-analysis. Pediatric Physical Therapy, 26(3), 289–300. https://doi.org/10.1097/PEP.0000000000000046
  • Cipresso, P., Giglioli, I. A. C., Raya, M. A., & Riva, G. (2018). The past, present, and future of virtual and augmented reality research: A network and cluster analysis of the literature. Frontiers in Psychology, 9, 2086. https://doi.org/10.3389/fpsyg.2018.02086
  • Corti, C., Urgesi, C., Poggi, G., Strazzer, S., Borgatti, R., & Bardoni, A. (2020). Home-based cognitive training in pediatric patients with acquired brain injury: Preliminary results on efficacy of a randomized clinical trial. Scientific Reports, 10(1), 1–15. https://doi.org/10.1038/s41598-020-57952-5
  • Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation Hybrid Designs. Medical Care, 50(3), 217–226. http://doi.org/10.1097/MLR.0b013e3182408812
  • De Luca, R., Portaro, S., Le Cause, M., De Domenico, C., Maggio, M. G., Cristina Ferrera, M., & Calabrò, R. S. (2020). Cognitive rehabilitation using immersive virtual reality at young age: A case report on traumatic brain injury. Applied Neuropsychology: Child, 9(3), 282–287. https://doi.org/10.1080/21622965.2019.1576525
  • Deutsch, J., & McCoy, S. W. (2017). Virtual reality and serious games in neurorehabilitation of children and adults: Prevention, plasticity and participation. Pediatric Physical Therapy, 29(Suppl. 3), S23. https://doi.org/10.1097/PEP.0000000000000387
  • Domensino, A. F., Evans, J., & van Heugten, C. (2022). From word list learning to successful shopping: The neuropsychological assessment continuum from cognitive tests to cognition in everyday life. Applied Neuropsychology: Adult, 1–8. https://doi.org/10.1080/23279095.2022.2079087
  • Eldh, A. C., Almost, J., DeCorby-Watson, K., Gifford, W., Harvey, G., Hasson, H., Kenny, D., Moodie, S., Wallin, L., & Yost, J. (2017). Clinical interventions, implementation interventions, and the potential greyness in between -a discussion paper. BMC Health Services Research, 17(1http://doi.org/10.1186/s12913-016-1958-5.
  • Eve, M., O’Keeffe, F., Jhuty, S., Ganesan, V., Brown, G., & Murphy, T. (2016). Computerized working-memory training for children following arterial ischemic stroke: A pilot study with long-term follow-up. Applied Neuropsychology: Child, 5(4), 273–282. https://doi.org/10.1080/21622965.2015.1055563
  • Foerster, R. M., Poth, C. H., Behler, C., Botsch, M., & Schneider, W. X. (2016). Using the virtual reality device Oculus Rift for neuropsychological assessment of visual processing capabilities. Scientific Reports, 6(1), 1–10. https://doi.org/10.1038/s41598-016-0001-8
  • Fordyce, D. J., & Roueche, J. R. (1986). Changes in perspectives of disability among patients, staff, and relatives during rehabilitation of brain injury. Rehabilitation Psychology, 31(4), 217. https://doi.org/10.1037/h0091549
  • Ghai, S., & Ghai, I. (2019). Virtual reality enhances gait in cerebral palsy: A training dose-response meta-analysis. Frontiers in Neurology, 10, 236. https://doi.org/10.3389/fneur.2019.00236
  • Green, C. S., Bavelier, D., Kramer, A. F., Vinogradov, S., Ansorge, U., Ball, K. K., Bingel, U., Chein, J. M., Colzato, L. S., Edwards, J. D., Facoetti, A., Gazzaley, A., Gathercole, S. E., Ghisletta, P., Gori, S., Granic, I., Hillman, C. H., Hommel, B., Jaeggi, S. M., … Witt, C. M. (2019). Improving Methodological Standards in Behavioral Interventions for Cognitive Enhancement. Journal of Cognitive Enhancement, 3(1), 2–29. http://doi.org/10.1007/s41465-018-0115-y
  • Higgins, J. P., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2019). Cochrane handbook for systematic reviews of interventions. John Wiley & Sons.
  • Hochstenbach, J., Prigatano, G., & Mulder, T. (2005). Patients’ and relatives’ reports of disturbances 9 months after stroke: Subjective changes in physical functioning, cognition, emotion, and behavior. Archives of Physical Medicine and Rehabilitation, 86(8), 1587–1593. https://doi.org/10.1016/j.apmr.2004.11.050
  • Holsbeeke, L., Ketelaar, M., Schoemaker, M. M., & Gorter, J. W. (2009). Capacity, capability, and performance: Different constructs or three of a kind? Archives of Physical Medicine and Rehabilitation, 90(5), 849–855. http://doi.org/10.1016/j.apmr.2008.11.015
  • Huygelier, H., Mattheus, E., Abeele, V. V., van Ee, R., & Gillebert, C. R. (2021). The use of the term virtual reality in post-stroke rehabilitation: A scoping review and commentary. Psychologica Belgica, 61(1), 145. https://doi.org/10.5334/pb.1033
  • Huygelier, H., Schraepen, B., Lafosse, C., Vaes, N., Schillebeeckx, F., Michiels, K., & Gillebert, C. R. (2020). An immersive virtual reality game to train spatial attention orientation after stroke: A feasibility study. Applied Neuropsychology: Adult, 29(5), 1–21. https://doi.org/10.1080/23279095.2020.1821030.
  • Hypher, R. E., Brandt, A. E., Risnes, K., Rø, T. B., Skovlund, E., Andersson, S., & Stubberud, J. (2019). Paediatric goal management training in patients with acquired brain injury: Study protocol for a randomised controlled trial. BMJ Open, 9(8), e029273. https://doi.org/10.1136/bmjopen-2019-029273
  • Johansen, T. E. B., Strøm, V., Simic, J., & Rike, P. O. (2020). Effectiveness of training with motion-controlled commercial video games for hand and arm function in people with cerebral palsy: A systematic review and meta-analysis. Journal of Rehabilitation Medicine, 52(1), 0–10. https://doi.org/10.2340/16501977-2633
  • Jones, J. S., Milton, F., Mostazir, M., & Adlam, A. R. (2020). The academic outcomes of working memory and metacognitive strategy training in children: A double-blind randomized controlled trial. Developmental Science, 23(4), e12870. https://doi.org/10.1111/desc.12870
  • Krasny-Pacini, A., Chevignard, M., & Evans, J. (2014). Goal management training for rehabilitation of executive functions: A systematic review of effectivness in patients with acquired brain injury. Disability and Rehabilitation, 36(2), 105–116. https://doi.org/10.3109/09638288.2013.777807
  • Krohn, S., Tromp, J., Quinque, E. M., Belger, J., Klotzsche, F., Rekers, S., & Thöne-Otto, A. (2020). Multidimensional evaluation of virtual reality paradigms in clinical neuropsychology: Application of the VR-check framework. Journal of Medical Internet Research, 22(4), e16724. https://doi.org/10.2196/16724
  • Laatsch, L., Harrington, D., Hotz, G., Marcantuono, J., Mozzoni, M. P., Walsh, V., & Hersey, K. P. (2007). An evidence-based review of cognitive and behavioral rehabilitation treatment studies in children with acquired brain injury. The Journal of Head Trauma Rehabilitation, 22(4), 248–256. https://doi.org/10.1097/01.HTR.0000281841.92720.0a
  • Lambregts, S. A., Smetsers, J. E., Verhoeven, I. M., de Kloet, A. J., van de Port, I. G., Ribbers, G. M., & Catsman-Berrevoets, C. E. (2018). Cognitive function and participation in children and youth with mild traumatic brain injury two years after injury. Brain Injury, 32(2), 230–241. https://doi.org/10.1080/02699052.2017.1406990
  • Laver, K. E., Lange, B., George, S., Deutsch, J. E., Saposnik, G., & Crotty, M. (2017). Virtual reality for stroke rehabilitation. Cochrane Database of Systematic Reviews, (11). https://doi.org/10.1002/14651858.CD008349.pub4
  • Lenhard, W., & Lenhard, A. (2016). Calculation of effect sizes. Psychometrica.
  • Li, K., Robertson, J., Ramos, J., & Gella, S. (2013). Computer-based cognitive retraining for adults with chronic acquired brain injury: A pilot study. Occupational Therapy in Health Care, 27(4), 333–344. https://doi.org/10.3109/07380577.2013.844877
  • Limond, J., Adlam, A. R., & Cormack, M. (2014). A Model for Pediatric Neurocognitive Interventions: Considering the Role of Development and Maturation in Rehabilitation Planning. The Clinical Neuropsychologist, 28(2), 181–198. http://doi.org/10.1080/13854046.2013.873083
  • Mascha, E. J., & Vetter, T. R. (2018). Significance, errors, power, and sample size: the blocking and tackling of statistics. Anesthesia & Analgesia, 126(2), 691–698. https://doi.org/10.1213/ANE.0000000000002741
  • McAuley, T., Chen, S., Goos, L., Schachar, R., & Crosbie, J. (2010). Is the behavior rating inventory of executive function more strongly associated with measures of impairment or executive function? Journal of the International Neuropsychological Society, 16(3), 495–505. https://doi.org/10.1017/S1355617710000093
  • McIntosh, R. D., & Rittmo, J. Ö. (2020). Power calculations in single case neuropsychology. PsyArXiv. https://doi.org/10.31234/osf.io/fxz49
  • Mekbib, D. B., Han, J., Zhang, L., Fang, S., Jiang, H., Zhu, J., & Xu, D. (2020). Virtual reality therapy for upper limb rehabilitation in patients with stroke: A meta-analysis of randomized clinical trials. Brain Injury, 34(4), 456–465. https://doi.org/10.1080/02699052.2020.1725126
  • Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734
  • Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1–9. https://doi.org/10.1186/2046-4053-4-1
  • Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational research methods, 11(2), 364–386.
  • Palanivel, V., & Burrough, M. (2021). Acquired brain injury in children, and their rehabilitation: Where we are now? Paediatrics and Child Health, 31(5). https://doi.org/10.1016/j.paed.2021.02.001
  • Parsons, T. D. (2015). Ecological validity in virtual reality-based neuropsychological assessment. In M Khosrow-Pour (Editor-in-Chief), Encyclopedia of Information Science and Technology (pp. 1006–1015). DBA Contemporary Research in Information Science and Technology, Book Series. IGI Global.
  • Phillips, N. L., Mandalis, A., Benson, S., Parry, L., Epps, A., Morrow, A., & Lah, S. (2016). Computerized working memory training for children with moderate to severe traumatic brain injury: A double-blind, randomized, placebo-controlled trial. Journal of Neurotrauma, 33(23), 2097–2104. https://doi.org/10.1089/neu.2015.4358
  • Piovesana, A., Ross, S., Lloyd, O., Whittingham, K., Ziviani, J., Ware, R. S., & Boyd, R. N. (2017). A randomised controlled trial of a web-based multi-modal therapy program to improve executive functioning in children and adolescents with acquired brain injury. Clinical Rehabilitation, 31(10), 1351–1363. https://doi.org/10.1177/0269215517695373
  • Ralph, M. L., Patterson, K., & Plaut, D. C. (2011). Finite case series or infinite single-case studies? Comments on “Case series investigations in cognitive neuropsychology” by Schwartz and Dell (2010). Cognitive Neuropsychology, 28(7), 466–474. https://doi.org/10.1080/02643294.2012.671765
  • Reijnders, J., van Heugten, C., & van Boxtel, M. (2013). Cognitive interventions in healthy older adults and people with mild cognitive impairment: A systematic review. Ageing Research Reviews, 12(1), 263–275. https://doi.org/10.1016/j.arr.2012.07.003
  • Resch, C. (2019). Executive functions in children and adolescents: novel perspectives on assessment and intervention. Ridderprint. https://doi.org/10.26481/dis.20191016cr
  • Resch, C., Anderson, V. A., Beauchamp, M. H., Crossley, L., Hearps, S. J., van Heugten, C. M., & Catroppa, C. (2019). Age-dependent differences in the impact of paediatric traumatic brain injury on executive functions: A prospective study using susceptibility-weighted imaging. Neuropsychologia, 124, 236–245. https://doi.org/10.1016/j.neuropsychologia.2018.12.004
  • Resch, C., Hurks, P. P., & van Heugten, C. M. (2020). Cognitieve revalidatie voor kinderen en jongeren met niet-aangeboren hersenletsel: Wat zijn de effectieve componenten? Neuropraxis, 24(3), 66–73. https://doi.org/10.1007/s12474-020-00254-3
  • Resch, C., Rosema, S., Hurks, P., de Kloet, A., & van Heugten, C. (2018). Searching for effective components of cognitive rehabilitation for children and adolescents with acquired brain injury: A systematic review. Brain Injury, 32(6), 679–692. https://doi.org/10.1080/02699052.2018.1458335
  • Rizzo, A. A., Schultheis, M., Kerns, K. A., & Mateer, C. (2004). Analysis of assets for virtual reality applications in neuropsychology. Neuropsychological Rehabilitation, 14(1-2), 207–239. https://doi.org/10.1080/09602010343000183
  • Rutkowski, S., Kiper, P., Cacciante, L., Cieślik, B., Mazurek, J., Turolla, A., & Szczepańska-Gieracha, J. (2020). Use of virtual reality-based training in different fields of rehabilitation: A systematic review and meta-analysis. Journal of Rehabilitation Medicine, 52(11). https://doi.org/10.2340/16501977-2755
  • Schwartz, M. F., & Dell, G. S. (2010). Case series investigations in cognitive neuropsychology. Cognitive Neuropsychology, 27(6), 477–494. https://doi.org/10.1080/02643294.2011.574111
  • Slagter, H. A. (2012). Conventional working memory training may not improve intelligence. Trends in Cognitive Sciences, 16(12), 582–583. https://doi.org/10.1016/j.tics.2012.10.001
  • Spreij, L. A., Gosselt, I. K., Visser-Meily, J. M. A., Hoogerbrugge, A. J., Kootstra, T. M., Nijboer, Tanja C. W., & Lavorgna, L. (2021a). The journey is just as important as the destination—Digital neuropsychological assessment provides performance stability measures in patients with acquired brain injury. PLOS ONE, 16(7), e0249886. http://doi.org/10.1371/journal.pone.0249886
  • Spreij, L. A., Sluiter, D., Gosselt, I. K., Visser-Meily, J. M. A., & Nijboer, T. C.W. (2021b). CoCo - participation: The development and clinical use of a novel inventory measuring cognitive complaints in daily life. Neuropsychological Rehabilitation, 31(2), 255–277. http://doi.org/10.1080/09602011.2019.1691017
  • Spreij, L. A., Visser-Meily, J. M., Sibbel, J., Gosselt, I. K., & Nijboer, T. C. (2020). Feasibility and user-experience of virtual reality in neuropsychological assessment following stroke. Neuropsychological Rehabilitation, 32(34), 1–21. https://doi.org/10.1080/09602011.2020.1831935
  • Van de Ven, R. M., Buitenweg, J. I., Schmand, B., Veltman, D. J., Aaronson, J. A., Nijboer, T. C., & Murre, J. M. (2017). Brain training improves recovery after stroke but waiting list improves equally: A multicenter randomized controlled trial of a computer-based cognitive flexibility training. PLoS One, 12(3), e0172993. https://doi.org/10.1371/journal.pone.0172993
  • Van de Ven, R. M., Murre, J. M., Veltman, D. J., & Schmand, B. A. (2016). Computer-based cognitive training for executive functions after stroke: A systematic review. Frontiers in Human Neuroscience, 10, 150. https://doi.org/10.3389/fnhum.2016.00150
  • Van Heugten, C., Caldenhove, S., Crutsen, J., & Winkens, I. (2020). An overview of outcome measures used in neuropsychological rehabilitation research on adults with acquired brain injury. Neuropsychological Rehabilitation, 30(8), 1598–1623. https://doi.org/10.1080/09602011.2019.1589533
  • Van Heugten, C. M., Hendriksen, J., Rasquin, S., Dijcks, B., Jaeken, D., & Vles, J. H. S. (2006). Long-term neuropsychological performance in a cohort of children and adolescents after severe paediatric traumatic brain injury. Brain Injury, 20(9), 895–903. https://doi.org/10.1080/02699050600832015
  • Van Heugten, C. M., Ponds, R. W., & Kessels, R. P. (2016). Brain training: Hype or hope? Neuropsychological Rehabilitation, 26(5-6), 639–644. https://doi.org/10.1080/09602011.2016.1186101
  • Van Tol, E., Gorter, J. W., DeMatteo, C., & Meester-Delver, A. (2011). Participation outcomes for children with acquired brain injury: A narrative review. Brain Injury, 25(13-14), 1279–1287. https://doi.org/10.3109/02699052.2011.613089
  • Verhelst, H., Vander Linden, C., Vingerhoets, G., & Caeyenberghs, K. (2017). How to train an injured brain? A pilot feasibility study of home-based computerized cognitive training. Games for Health Journal, 6(1), 28–38. https://doi.org/10.1089/g4h.2016.0043
  • Vermeir, J. F., White, M. J., Johnson, D., Crombez, G., & Van Ryckeghem, D. M. (2020). The effects of gamification on computerized cognitive training: Systematic review and meta-analysis. JMIR Serious Games, 8(3), e18644. https://doi.org/10.2196/18644
  • Voinescu, A., Sui, J., & Stanton Fraser, D. (2021). Virtual reality in neurorehabilitation: An umbrella review of meta-analyses. Journal of Clinical Medicine, 10(7), 1478. https://doi.org/10.3390/jcm10071478
  • Vriezen, E. R., & Pigott, S. E. (2010). The Relationship Between Parental Report on the BRIEF and Performance-Based Measures of Executive Function in Children with Moderate to Severe Traumatic Brain Injury. Child Neuropsychology, 8(4), 296–303. http://doi.org/10.1076/chin.8.4.296.13505
  • Warnier, N., Lambregts, S., & Port, I. V. D. (2020). Effect of virtual reality therapy on balance and walking in children with cerebral palsy: A systematic review. Developmental Neurorehabilitation, 23(8), 502–518. https://doi.org/10.1080/17518423.2019.1683907
  • Wiley, E., Khattab, S., & Tang, A. (2020). Examining the effect of virtual reality therapy on cognition post-stroke: A systematic review and meta-analysis. Disability and Rehabilitation: Assistive Technology, 17(1), 1–11. https://doi.org/10.1080/17483107.2020.1755376
  • World Health Organization. (2007). International classification of functioning, disability, and health: Children & youth version: ICF-CY.
  • Wu, J., Loprinzi, P. D., & Ren, Z. (2019). The rehabilitative effects of virtual reality games on balance performance among children with cerebral palsy: A meta-analysis of randomized controlled trials. International Journal of Environmental Research and Public Health, 16(21), 4161. https://doi.org/10.3390/ijerph16214161