Systematic review of the effectiveness of innovative, gamified interventions for cognitive training in paediatric acquired brain injury

ABSTRACT

Effectiveness of innovative, gamified interventions (i.e., Augmented Reality, Computer-Based Cognitive Retraining [CBCR], and Virtual Reality [VR] in conjunction with a Serious Game) for cognitive training in paediatric ABI was evaluated. Studies were identified on PsycINFO, PubMed and Scopus; last searched 4 January 2022. Eligibility criteria were participants diagnosed with ABI and aged ≤ 18 years, experimental intervention to train cognition, cognition assessed pre- and post-intervention at: (1) The level of function, or (2) The level of activity, and written in English. ROB 2 and ROBINS-I were utilised to assess risk of bias. Extracted study characteristics were methods, participants, interventions, outcomes, and results. Seven studies were included, comprising six CBCR studies and one VR study, with 182 participants. Following CBCR: (1) Improvements were observed in several cognitive functions, but there was inconsistent evidence; (2) Improvements were reported in attention and executive functions (EF) at home and at school. Following VR: (1) Improvements were observed in attention and EF; (2) Not evaluated. Due to the small number of included studies with (relatively) small and heterogeneous samples, only a cautious interpretation of the evidence was provided. There is a need for carefully designed studies with more attention to inter-individual differences and generalisation to daily life.

KEYWORDS:

• Brain injury
• Cognition
• Computer-Based Cognitive Retraining
• Paediatric
• Virtual Reality

Introduction

Cognitive impairment is highly prevalent following acquired brain injury (ABI) in children (Anderson et al., 2006; Babikian & Asarnow, 2009; Resch et al., 2019; Van Heugten et al., 2006). This may be evident across multiple cognitive domains, including attention, executive functions (EF), and memory (Anderson et al., 2006; Babikian & Asarnow, 2009; Resch et al., 2019; Van Heugten et al., 2006). These cognitive impairments are considered impaired body functions that may negatively influence the activity and participation domains, as defined in the WHO’s International Classification of Functioning Disability and Health for Children and Youth (ICF-CY, 2007). For example, children may have limitations in managing daily routines, restrictions in structured activities in the community, and restrictions in social play with peers at school and in the community (Bedell & Dumas, 2004; Van Tol et al., 2011). Historical perspectives, such as the “Kennard Principle” suggested children to be more resilient to brain injury and more amenable to recovery than adults (Palanivel & Burrough, 2021). More recent literature suggests that long-lasting cognitive impairment in children may be part of a developmental cascade that hampers academic performance and psychosocial outcomes, exacerbating long-term restrictions in daily functioning; the phenomenon known as “growing into deficits” (Lambregts et al., 2018). Therefore, early interventions aiming to improve a child’s recovery at the level of body function and the level of activities and participation are essential.

In order to promote recovery, researchers have explored the possibilities of gamified cognitive training in children with ABI. Computer-Based Cognitive Retraining (CBCR) aims to ameliorate cognitive impairment through intensive repetition of specific cognitive tasks (Li et al., 2013), typically focusing on attention, EF, information processing speed, working memory (WM) and reasoning skills (Van Heugten et al., 2016). CBCR can be performed in the patient’s home environment and provides tailored programs in which difficulty can be adjusted trial-by-trial (e.g., dynamic difficulty progression); therefore, continuously challenging children and maximising cognitive effort (Van Heugten et al., 2016). More recently, innovative (i.e., a new approach that has emerged in the published literature over the last two decades) technologies, such as Augmented Reality (AR) and immersive Virtual Reality (iVR), have gained increased attention. VR is typically defined as a “computer-generated interactive virtual world that simulates the real world” (Huygelier et al., 2021). VR encompasses a wide variety of technologies, from semi-iVR featuring 2D displays with a limited field of view (i.e., window on the world displays) to iVR (i.e., head-mounted 3D displays), which provides a 360° field of view (Huygelier et al., 2021). In AR, virtual 3D objects can be projected on the real world (Huygelier et al., 2021), which offers the opportunity to interact with the real environment. AR and VR can deliver enhanced feelings of engagement, transport and presence (Cipresso et al., 2018; Laver et al., 2017; Spreij et al., 2020), which may facilitate treatment motivation (Huygelier et al., 2020). Enhancing treatment motivation may also be achieved by introducing game aspects into cognitive rehabilitation (Huygelier et al., 2020; Vermeir et al., 2020). Serious Gaming refers to the gamification of tasks (e.g., goal setting, narrative, feedback and rewards) used in nonentertainment settings (Vermeir et al., 2020). However, there is still no consensus on the definition of a “game” or a clear perspective on the main characteristics that make a game enjoyable, motivating or even addictive (Bedwell et al., 2012).

In contrast to CBCR, VR allows the development of more ecologically valid environments (i.e., complex, dynamic, interactive 3D scenarios), offering the possibility to intensively train complex cognitive skills in an enriched, multisensory environment which elicits more natural behaviour and cognition (Bohil et al., 2011; Huygelier et al., 2020; Krohn et al., 2020; Parsons, 2015; Rizzo et al., 2004; Spreij et al., 2020). VR training can target an array of cognitive domains including attention, EF, and memory. Similar to CBCR, VR can be performed in the patient’s home and tailored to a patient’s individual level of functioning. VR allows more control over stimulus presentation (i.e., audio-visual-tactile stimuli; Foerster et al., 2016), whilst capturing precise and detailed performance measures due to continuous data acquisition; therefore, offering real-time feedback (Rizzo et al., 2004; Spreij et al., 2020).

Reviews on cognitive interventions in paediatric ABI and practice recommendations for effective intervention in paediatric ABI are limited (Laatsch et al., 2007; Resch et al., 2018). A previous review included six studies evaluating CBCR (i.e., Captain’s Log, Cogmed™ WM training, Foramen Rehab®, Nintendo Wii™, and TEACHware™; Resch et al., 2018). Following CBCR, improvements were demonstrated in trained tasks or cognitive tasks similar to the trained tasks in the intervention. VR was not included.

The majority of meta-analyses and systematic reviews on VR and Serious Games in a range of neurodevelopmental conditions, published between 2009 and 2015, focused on motor outcomes (i.e., nine of 12 published systematic reviews; Deutsch & McCoy, 2017). Children with cerebral palsy represented the most prevalent participants (i.e., 70% of participants); the remaining participants included Down’s syndrome and coordination disorders (Deutsch & McCoy, 2017). A recent “umbrella” review evaluated the effectiveness and quality of evidence of VR-based cognitive and physical interventions in patients with cerebral palsy, stroke, and traumatic brain injury (TBI; Voinescu et al., 2021). Six of the 41 included meta-analyses included children, specifically with cerebral palsy (Chen et al., 2014; 2018; Ghai & Ghai, 2019; Johansen et al., 2020; Mekbib et al., 2020; Rutkowski et al., 2020; Warnier et al., 2020; Wiley et al., 2020; Wu et al., 2019). Therefore, no meta-analysis included children with ABI. Moreover, the effectiveness of VR-based interventions on improving cognitive functioning (e.g., overall cognition) was not evaluated.

To the best of our knowledge, there is no state-of-the-art review specifically examining the effectiveness of innovative, gamified interventions for cognitive training in paediatric ABI. The overarching aim of the present review was; therefore, to comprehensively evaluate the effectiveness of innovative, gamified interventions (i.e., AR, CBCR, and VR in conjunction with a Serious Game), for cognitive training in paediatric ABI. The sub-aims were three-fold: (1) To describe the technology-based interventions most frequently reported, (2) To evaluate the effectiveness of gamified technology-based interventions at the level of body function (i.e., cognition), and (3) To evaluate the effectiveness of gamified technology-based interventions at the level of activities, such as acquiring skills, carrying out daily routines, and undertaking multiple tasks.

Methods

Eligibility criteria

Studies were included when the following criteria were met: (1) Participantcharacteristics: participants were diagnosed with ABI and aged ≤ 18 years. If an article reported results from participants with different neurological conditions (i.e., non-ABI) or participants older than 18 years, it was included only if the data from the ABI participants or the participants younger than 18 years were reported separately or if the majority of the participants had been diagnosed with ABI or younger than 18 years; (2) Intervention characteristics: experimental intervention (i.e., AR, CBCR, or VR in conjunction with a Serious Game) to train cognitive function vs. a comparator (e.g., no intervention, wait-list control, usual care); (3) Outcomes: assessed cognition pre- and post-intervention at the level of function (i.e., academic- and cognitive capacity – what an individual can do in a standardised, controlled environment; Holsbeeke et al., 2009) and, if included, at the level of activities (i.e., academic- and cognitive capability – what an individual can do in their daily environment; Holsbeeke et al., 2009); (4) Study design: no specific criteria; (5) Written in English.

Search methods for identification of studies

Electronic searches

Innovative, gamified intervention studies for cognitive training in children with ABI were identified on PsycINFO, PubMed and Scopus, using relevant keywords appropriate to each resource. For an overview of the relevant keywords utilised for the search string, see Appendix 1. Searches were initially conducted by two authors (C.L.W and M.V) on 23 December 2020, updated 20 April 2021, and updated 4 January 2022. Restrictions were applied on language (English only), year (2000 – current) and age (0-18 years).

Reference lists

Reference lists of all included studies were manually checked to identify additional studies missed from the original electronic searches.

Data collection and analysis

Selection of studies

Two authors (C.L.W and M.V) screened the titles and abstracts of all studies identified as a result of the search, coding them as “retrieve” (eligible or potentially eligible/unclear) or “do not retrieve.” Full-text study reports/publications were retrieved, and the two authors independently screened the full texts to identify studies for inclusion, identifying and recording reasons for exclusion of the ineligible studies. Disagreements were resolved through discussion. Duplicate records were identified and collated. Selection process is documented in the PRISMA flow diagram (Moher et al., 2015).

Data extraction and management

C.L.W extracted the following study characteristics: (1) Methods: date of study, study design, recorded time points, and type of study (i.e., efficacy or feasibility); (2) Participants: aetiology, age, age ofonset/time post-injury, number of participants, and sex; (3) Interventions: comparator, intervention, setting, and targeted cognitive domain; (4) Outcomes: academic- and cognitive capacity (i.e., neuropsychological assessment [NPA]; a conventional set of paper-and-pencil tests used in a clinical setting to estimate the maximum capacity of a patient’s cognitive functions), academic- and cognitive capability (i.e., self/parent reported outcomes on cognition in daily life); (5) Results: effect size , statistical significance (p < 0.05 or p < 0.001), and training completion rate; (6) Funding. Following this, M.V checked that data were entered correctly by comparing data presented in the systematic review with all study reports. Study characteristics were spot-checked for accuracy against the study report. For an overview of the characteristics of the included studies, see Tables 1 and 2.

Table 1. Summary of study characteristics (level of function).

Authors	Design (methodology) & recorded time points	Participant characteristics	Intervention characteristics			Cognition at level of function
			Training (active/passive control)	Duration (sessions)	Setting	Cognitive domain	NPA
Carlson-Green et al. (2017)	Single-group (pilot; efficacy & feasibility) Pre-post (NI) 6 months	n = 21 Aetiology: BT Age (y): 8.0–18.0 Age of onset (y): m = 6.0, 1.0–14.0 range Sex: 12F/9M	Cogmed™ WM training (no control)	8–12 w (35)	Home; contacted 1/w by phone + emails (adherence, feedback, motivation, & support)	Academic capacity VWM Visual WM	WJ-III: Applied Problems Passage Comprehension AWMA: Backward Digit Recall Counting Recall Digit Recall Word Recall Dot Matrix Mr. X Spatial Span
Corti et al. (2020)	RCT (stepped-wedge; efficacy) Pre-post 2 months 6 months (NI)	n = 32 (E: 18, C: 14) Aetiology: CVA (E: 5, C: 4), encephalitis (E: 0, C: 1), HAI (E: 2, C: 0), TBI (E: 11, C: 9) Age (y): E: 13.83 ± 1.65 & C: 13.50 ± 1.99 Sex: E: 7F/12M & C: 3F/11M Time post-injury (y­): ≥1	Lumosity Cognitive Training™ (active)	5 × 20 mins pw, 8 w (40)	Home; contacted 1/w by phone (no feedback – adherence & motivation)	Academic capacity EF Visual WM	AC-MT: Arithmetic Calculation Problem-solving Wisconsin Card Sorting test Corsi Block-Tapping test
De Luca et al. (2020)	Case study (efficacy) Pre-post	n = 1 Aetiology: TBI Age (y): 15.0 Sex: 1M Time post-injury (y): ≥0.8	CAREN (active)	5 × 45 mins pw, 4 w (20)	Rehabilitation	Attention EF	Modified Bell Cancellation test WEIGL test
Eve et al. (2016)	Single-group, longitudinal (pilot; efficacy & feasibility) Pre-post 12 months	n = 7 (12 months: n = 5) Aetiology: AIS Age (y): 10.0–16.0 Age of onset (y): 1.0–10.0 Sex: 3F/4M	Cogmed™ WM training (no control)	30–40 mins, 5–7 w (25)	Home	Academic capacity Attention VWM Visual WM	WRAT-4: Mathematics TEA-Ch: Score! Score! DT Sky-Search Sky Search DT WMTB-C: Backward Digit Recall Digit Recall Listening Recall Non-word List Recall Word Listening Matching Word List Recall Block Recall
Phillips et al. (2016)	RCT (efficacy) Pre-post 3 months	n = 27 Aetiology: TBI Age (y): E: 12.75 & C: 11. 82 (median) Age of onset (y): E: 4.01 & C: 8.45 (median) Sex: NI	Cogmed™ WM training (active)	5 × 30–40 mins pw, 5 w (25)	Home	Academic capacity Attention VWM Visual WM	WIAT-II: Numerical Operations Reading Comprehension Word Reading TEA-Ch: Creature Counting Score! Sky Search Sky Search DT Walk/Don’t Walk AWMA: Counting Recall Digit Recall Dot Matrix Mr. X
Piovesana et al. (2017)	RCT (efficacy) Pre-post	n = 58 Aetiology: CVA (E: 12, C: 10), non-TBI (E: 9, C: 8), TBI (E: 8, C: 11) Age (y): E: 11.11 ± 2.60 & C: 11.10 ± 2.60 Age of onset (y): E: 5.20 ± 4.0 & C: 5.90 ± 3.50 Sex: E: 12F/17M & C:14F/15M	Mitii™ (passive)	6 × 30 mins pw, 20 w (120)	Home; frequently contacted by phone/email (feedback & support)	EF	D-KEFS: Colour-Word Inference TEA-Ch: Score! Sky Search Sky Search DT Tower of London-DX 2nd Edition Trail Making Test WISC-IV: Coding, Digit Span Backward, Symbol Search
Verhelst et al. (2017)	Case series, observational, longitudinal (pilot; feasibility) Pre-post 6 months	n = 5 Aetiology: TBI Age (y): 15.0–17.0 Sex: 1F/4M Time post-injury (y): 2.0–4.0	BrainGames (no control)	5 × pw, 8 w (40)	Home	Attention EF PSI VWM Visual WM	PEBL: Continuous Performance Test (reaction time) Flanker task CANTAB: Stockings of Cambridge Intra–Extradimensional Set Shift WISC-III-NL: Digit Symbol Coding Digit Span Backward Digit Span Forward Spatial Span

Abbreviations: AC-MT = Italian battery AC-MT; AIS = Arterial ischemic stroke; AWMA = Automated Working Memory Assessment; BT = Brain tumour; C = Control group; CANTAB = Cambridge Neuropsychological Test Automated Battery; CVA = Cerebrovascular accident; D-KEFS = Delis–Kaplan Executive Function System; E = Experimental group; EF = Executive functions; F = Female; HAI = Hypoxic-anoxic brain injury; M = Male; m = Mean; n = Number of participants; NI = No information; PEBL = Psychology Experiment Building Language; PSI = Information processing speed; pw = Per week; RCT = Randomised controlled trial; TBI = Traumatic brain injury; TEA-Ch = Test of Everyday Attention for Children; VWM = Verbal working memory; w = Week(s); WIAT-II = Wechsler Individual Achievement Tests, Second Edition Australian standardised edition; WISC-III-NL = Weschler Intelligence Scale for Children Third Edition (Dutch); WJ-III = Woodcock-Johnson Tests of Achievement, Third Edition; WM = Working memory; WMTB-C = Working Memory Test Battery for Children; WRAT-4= Wide Ranging Achievement Test 4; y = Year(s). Trailing Making Test: WISC-IV:, Coding, Digit Span Backward, Symbol Search.

Table 2. Summary of study characteristics (level of activities).

Authors	Design (methodology) & recorded time points	Participant characteristics	Intervention characteristics			Cognition at level of activities
			Training (active/passive control)	Duration (sessions)	Setting	Cognitive domain	Subjective: cognition in daily life
Carlson-Green et al. (2017)	Single-group (pilot; efficacy & feasibility) Pre-post (NI) 6 months	n = 21 Aetiology: BT Age (y): 8.0–18.0 Age of onset (y): m = 6.0, 1.0–14.0 range Sex: 12F/9M	Cogmed™ WM training (no control)	8–12 w (35)	Home; contacted 1/w by phone + emails (adherence, feedback, motivation, & support)	EF	BRIEF: Inhibit (parent-reported) Shift (parent-reported) Initiate (parent-reported) Emotional control (parent-reported) WM (parent-reported) Plan/organisation (parent-reported) organisation of materials (parent-reported) Monitor (parent-reported)
Piovesana et al. (2017)	RCT (efficacy) Pre-post	n = 58 Aetiology: CVA (E: 12, C: 10), non-TBI (E: 9, C: 8), TBI (E: 8, C: 11) Age (y): E: 11.11 ± 2.60 & C: 11.10 ± 2.60 Age of onset (y): E: 5.20 ± 4.0 & C: 5.90 ± 3.50 Sex: E: 12F/17M & C: 14F/15M	Mitii™ (passive)	6 × 30 mins pw, 20 w (120)	Home; frequently contacted by phone/email (feedback & support)	EF	BRIEF: BRI (parent-reported) MI (parent-reported) GEC (parent-reported)
Verhelst et al. (2017)	Case-series, observational, longitudinal (pilot; feasibility) Pre-post 6 months	n = 5 Aetiology: TBI Age (y): 15.0–17.0 Sex: 1F/4M Time post-injury (y): 2.0–4.0	BrainGames (no control)	5 × pw, 8 w (40)	Home	EF	BRIEF: BRI (self- & parent-reported) MI (self- & parent-reported) GEC (self- & parent-reported)

Abbreviations: BRIEF = Behaviour Rating Inventory of Executive Function; BT = Brain tumour; BRI = Behavioural Regulation Index (e.g., cognitive flexibility, inhibitory control, self-monitoring, shifting); C = Control group; EF = Executive functions; E = Experimental group; F = Female; GEC = Global Executive Composite; M = Male; MI = Metacognition Index (e.g., initiation, planning/organisation, task-monitoring, and working memory); m = Mean; n = Number of participants; NI = No information; pw = Per week; RCT = randomised controlled trial; TBI = Traumatic brain injury; w = Week(s); WM = Working memory; y = Year(s).

Assessment of risk of bias

C.L.W and M.V independently assessed the risk of bias for each study using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2019). The Revised Cochrane risk-of-bias for randomised trials (ROB 2) was utilised to judge the risk of bias in the findings of the randomised trials. The Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I) was utilised to judge the risk of bias in findings of non-randomised trials. Any disagreements were resolved by discussion or by involving another review author (T.C.W.N). Risk of bias was judged according to the following domains: (1) Bias arising from the randomisation process; (2) Bias due to deviations from the intended interventions (effect of assignment to intervention); (3) Bias due to missing outcome data; (4) Bias in measurement of the outcome; (5) Bias in selection of the reported result. Signalling questions within each domain provided the basis for domain-level judgements about the risk of bias. According to the ROB-2, the possible risk of bias judgements are: low, some concerns and high. According to the ROBINS-I, the possible risk of bias judgements are: low, moderate and serious. In the present review, the risk of bias was referred to as: low, moderate and severe.

Results

A total of 473 studies were identified as a result of the electronic search; a further 29 studies were identified through manual searches of references (n = 28) or Google Scholar (n = 1). After removing duplicates, 464 studies were screened on the basis of titles and abstracts, excluding 410 studies. A total of 54 studies were analysed on the basis of full text. Subsequently, seven studies with a total of 182 participants met the eligibility criteria and were included in the present review, consisting of six CBCR studies identified as a result of the electronic search and one iVR study identified via Google Scholar. No studies were identified that evaluated the effectiveness of AR. For an overview of the study selection process and reasons for exclusion, see the PRISMA flow diagram in Figure 1 (Moher et al., 2015).

Figure 1. Flow diagram of study selection in accordance with the PRISMA statement (Moher et al., 2015).

Flow diagram portraying study selection (i.e., identification, screening, eligibility and included).

Description of included studies

For an overview of the study characteristics of the included studies, see Tables 1 and 2.

Design

Three studies were RCTs (Individually randomised parallel-group trials), and four studies were non-RCT (Observational [case study and case-series] or single-group experimental design). Regarding the type of study, four focused on efficacy (n = 149 participants), two studies focused on efficacy and feasibility (n = 28 participants), and one study focused on feasibility only (n = 5 participants). An efficacy study evaluated iVR.

Six of the seven studies in the present review criticised their (relatively) small sample (Carlson-Green et al., 2017; Corti et al., 2020; De Luca et al., 2020; Eve et al., 2016; Phillips et al., 2016; Verhelst et al., 2017), with three studies discussing the limited power (Corti et al., 2020; Eve et al., 2016; Verhelst et al., 2017). Therefore, the chance of a Type II error (i.e., no effect in the population, whilst in fact an effect is present) increased.

Participants

Included studies involved participants of varied ages (8–18 years) with ABI due to the following underlying conditions: trauma (n = 5), cardiovascular injury (n = 3), brain tumour (n = 1), hypoxic-anoxic brain injury (n = 1), and other non-traumatic ABI (i.e., no further information provided; n = 1). Participants were recruited from: hospital (n = 4), rehabilitation setting (n = 3), outreach facilities (n = 1), or the Brain Damage Registry of Scientific Institute (n = 1). The CBCR studies involved participants in the chronic phase of rehabilitation (i.e., at least one-year post-ABI), whereas the iVR study involved one participant in the subacute phase of rehabilitation (i.e., at least one-month post-ABI).

Outcome measures

With regard to the level of function, NPA was employed to assess individual cognitive domains, employing conventional paper-and-pencil tests (n = 7) or the Luminosity Performance index (i.e., age-adjusted measure of improvement for Lumosity Cognitive Training™; n = 1). The most commonly assessed cognitive domain was memory (n = 5). Other cognitive domains assessed were attention (n = 3), EF (n = 3), and processing speed (n = 1). Four studies assessed academic capacity. In some cases, the same tasks were considered to measure attention of EF by different authors (e.g., the TEA-Ch).

With regard to the level of activities, three studies employed the Behaviour Rating Inventory of EF (the BRIEF) to assess cognition in daily life (i.e., at home and school), utilising parent-reported outcomes (n = 2) or self- and parent-reported outcomes (n = 1).

Interventions

The most frequently used intervention was CBCR (n = 6), specifically Cogmed™ WM Training (2D computer; n = 3). Other CBCR interventions used were BrainGames (2D computer; n = 1), Lumosity Cognitive Training™ (2D computer; n = 1) and Move it to improve it (Mitii™ - 2D computer and Microsoft Kinect®; n = 1). One study used a CAVE iVR system (3D), specifically Computer Assisted Rehabilitation Environment (CAREN).

Duration (and the number of sessions) of the interventions varied considerably in the included studies, ranging from four weeks (20 sessions; n = 1), ≥ five to ≤ eight weeks (25–40 sessions; n = 4), ≤ 12 weeks (35 sessions; n = 1) and 20 weeks (120 sessions; n = 1). In the majority of the included studies (n = 5), participants in the intervention group had a mean training completion of > 90% (n = 5). In another study, participants completed a mean of 29%; the proposed training dose was 60 hours over 20 weeks, whereas children in the intervention group completed a mean of 17.6 (SD = 14.9) hours. One study did not report training completion rate. For an overview of the completion rate, see Tables 3 and 4.

Table 3. Summary of findings (level of function).

Authors	Training completion (%)	NPA	Significance of comparisons between pre-training (t0), post-training (t1) & follow-up (t2)			Effect size (Cohen’s d, dppc2, r, SCS)			Comments
			t2-t1	t2-t0	t1-t0	t2-t1	t2-t0	t1-t0
Carlson-Green et al. (2017)	95.00	WJ-III:
		Applied Problems		+*	x
		Passage Comprehension		-	x
		AWMA:
		Backward Digit Recall		-	x
		Block Recall		+**	x
		Counting Recall		-	x
		Digit Recall		+**	x
		Dot Matrix		+**	x
		Mr. X		+*	x
		Spatial Span		+*	x
		Word Recall		+*	x
Corti et al. (2020)	98.30	AC-MT:							dppc2
		Arithmetic Calculation (accuracy)		-	-	-0.29	-0.06	0.24
		Arithmetic Calculation (speed)		-	+**	-0.33	-0.22	0.12
		Problem-solving		-	-	-0.02	0.13	0.15
		Wisconsin Card Sorting test		-	-	-0.40	-0.29	0.09
		Corsi Block-Tapping test		+**	+**	-0.47	0.32	0.92
De Luca et al. (2020)	100.00	Modified Bell Cancellation test			+*				RCI > 1.96
		WEIGL test			+*
Eve et al. (2016)	100.00	WRAT-4:							SCS
		Mathematics		-	-		-0.82	0.25
		TEA-Ch:
		Score!		-	-		0.03	0.17
		Score! DT		-	-		0.28	0.16
		Sky-Search		x	x		x	x
		Sky Search DT		-	-		0.28	0.16
		Walk/Don’t Walk		-	-		-0.82	0.25
		WMTB-C:
		Backward Digit Recall		-	-		0.03	0.48
		Digit Recall		-	+*		-0.15	0.46
		Listening Recall		-	-		0.03	0.48
		Non-word List Recall		-	+*		-0.15	0.46
		Word List Matching		-	+*		-0.15	0.46
		Word List Recall		-	+*		-0.15	0.46
		Block Recall		-	-		0.10	0.54
Phillips et al. (2016)	93.04	WIAT-II:							r
		Numerical Operations		-	-		-0.20	-0.15
		Reading Compreh.		-	+*		-0.10	-0.40
		Word Reading		+*	-		-0.46	-0.01
		TEA-Ch:
		Creature Counting		-	-		-0.07	-0.15
		Score!		-	-		-0.26	0.00
		Sky Search		-	-		-0.16	-0.14
		Sky Search DT		-	-		-0.07	-0.02
		Walk/Don’t Walk		-	-		0.00	-0.09
		AWMA:
		Counting Recall		-	-		-0.26	-0.07
		Digit Recall		-	-		-0.07	-0.21
		Dot Matrix		+*	+**		-0.41	-0.50
		Mr. X		-	-		-0.03	-0.34
Piovesana et al. (2017)	17.57	D-KEFS:							dppc2
		Colour Naming			-			-0.49
		Word Reading			-			-0.20
		TEA-Ch:
		Score!			-			-0.08
		Sky Search			-			-0.08
		Sky Search DT			-			-0.30
		Tower of London-DX 2nd Edition			-			0.54
		CTMT Trail 2			-			0.10
		CTMT Trail 3			-			0.25
		CTMT Trail 4			-			0.08
		CTMT Trail 5			-			0.12
		WISC-IV:
		Coding			-			0.18
		Digit Span Backward			-			-0.02
		Symbol Search			-			0.96
Verhelst et al. (2017)	92.13	PEBL:							Cohen’s d
		Continuous Performance Test (reaction time)					0.93	0.99
		Flanker task					0.98	3.18
		CANTAB:
		Stockings of Cambridge					1.89	1.26
		Intra–Extradimensional Set Shift					0.97	0.08
		WISC-III-NL:
		Digit Symbol Coding					0.35	0.30
		Digit Span Backward					1.98	0.55
		Digit Span Forward					1.07	0.48
		Spatial Span Backward					1.06	0.27
		Spatial Span Forward					1.73	0.78

Abbreviations: − = Not significant; +* = Significant at p ≤ .05; +** = Significant at p ≤ .01; AC-MT = Italian battery AC-MT; AWMA = Automated Working Memory Assessment; CANTAB = Cambridge Neuropsychological Test Automated Battery;?CTMT = Comprehensive Trail Making Test; D-KEFS = Delis–Kaplan Executive Function System; d_ppc2 =?Effect Size?d_ppc2?sensu Morris (2008) C.V.W calculated the effect size ‘d_ppc2’ (Morris, 2008), using an online calculator (Lenhard & Lenhard, 2016); n = Number of participants; PEBL = Psychology Experiment Building Language; TEA-Ch = Test of Everyday Attention for Children; WIAT-II = Wechsler Individual Achievement Tests; SCS =?Standardised change score;?Second Edition Australian standardised edition; WISC-III-NL = Weschler Intelligence Scale for Children Third Edition (Dutch); WJ-III = Woodcock-Johnson Tests of Achievement, Third Edition; WMTB-C = Working Memory Test Battery for Children; WRAT-4 = Wide Ranging Achievement Test 4

Table 4. Summary of findings (level of activities).

Authors	Training completion (%)	Subjective: cognition in daily life	Significance of comparisons between pre-training (t0), post-training (t1) & follow-up (t2)			Effect size (Cohen’s d, dppc2)			Comments
			t2-t1	t2-t0	t1-t0	t2-t1	t2-t0	t1-t0
Carlson-Green et al. (2017)	95.00	BRIEF:
		Inhibit (parent-reported)		+**
		Shift (parent-reported)		+*
		Initiate (parent-reported)		+*
		Emotional control (parent-reported)		-
		WM (parent-reported)		+**
		Plan/organisation (parent-reported)		+*
		Organisation of materials (parent-reported)		-
		Monitor (parent-reported)		+**
Piovesana et al. (2017)	17.57	BRIEF:							dppc2
		BRI (parent-reported)			-			-0.11
		MI (parent-reported)			-			-0.12
		GEC (parent-reported)			-			-0.09
Verhelst et al. (2017)	92.13	BRIEF:							Cohen’s d
		BRI
		(self-reported)					0.84	0.27
		(parent-reported)					-0.70	-0.24
		MI
		(self-reported)					0.50	0.25
		(parent-reported)					0.73	0.30
		GEC
		(self-reported)					1.00	0.34
		(parent-reported)					0.64	0.14

Abbreviations: − = Not significant; +* = Significant at p ≤ .05; +** = Significant at p ≤ .01; BRIEF: = Behaviour Rating Inventory of Executive Function; BRI = Behavioural Regulation Index (e.g., cognitive flexibility, inhibitory control, self-monitoring, shifting); GEC = Global Executive Composite; MI = Metacognition Index (e.g., initiation, planning/organisation, task-monitoring, and working memory); WM = Working memory.

As comparator, the RCTs utilised either an active control (non-adaptive: n = 1) or passive control (wait-list: n = 1; waiting-first/training-first: n = 1). The non-adaptive training (i.e., Placebo WM training) provided the same exercises as the intervention, but the difficulty never increased. The observational case study utilised an active control (adaptive); conventional cognitive therapy delivered in a clinical setting (i.e., face-to-face and paper-and-pencil approach) and adapted over time to the needs and performance of the participant. The active control was presented first, followed by a three-week stay in which the participant participated in conventional physiotherapy, before the active intervention was presented.

Personnel were blinded in two RCTs (Corti et al., 2020 [specifically outcome assessors]; Phillips et al., 2016) and one single case study (De Luca et al., 2020), whilst two RCTs (Corti et al., 2020; Phillips et al., 2016) reported that the participants were blinded. No study checked the success of the blindness manipulation.

Funding

Funding The majority of included studies were financially supported (n = 5), one study stated no financial support (De Luca et al., 2020) and the other did not state anything (Eve et al., 2016). For an overview of the acquisition/support of funding, see See Appendix 2 for an overview of the acquisition/support of funding.

Effect of interventions

For an overview of the findings of the included studies, see Tables 3 and 4.

Computer-based cognitive retraining

Working memory training. One RCT evaluated the efficacy of Cogmed™ WM training in TBI (Phillips et al., 2016). A significant improvement was demonstrated on a single measure of visual WM (i.e., Dot Matrix of the Automated WM Assessment [AWMA]), when compared with the active control group (i.e., modified, non-adaptive Cogmed WM training). At the three-month follow-up, improvements were sustained. No significant group differences were observed on measures of attention and other measures of WM (i.e., Counting Recall [verbal WM], Digit Recall [verbal WM], and Mr. X [visuospatial WM] of the AWMA). Significant improvements were observed on select academic capacity measures post-training (i.e., Reading Comprehension of the Wechsler Individual Achievement Test, 2nd Edition [WIAT-II]) and at the three-month follow-up (i.e., Word Reading of the WIAT-II). Limitations of this study concerned the design (i.e., small sample size [<16 per group; McIntosh & Rittmo, 2020]) and the results, namely the internal validity (i.e., repeated measurements; the testing effect). Phillips et al. (2016) was considered a moderate risk of bias.

Two single-group, pilot studies evaluated the efficacy and feasibility of Cogmed™ WM training in childhood arterial ischemic stroke (AIS; Eve et al., 2016) and survivors of paediatric cancer (Carlson-Green et al., 2017); participants were diagnosed with brain tumours (i.e., medulloblastoma [n = 11], ependymoma [n = 4], germinoma [n = 4], and other tumour types [n = 2]).

Considering children with AIS, significant improvements were observed on most measures of verbal WM (i.e., Digit Recall, Word List Matching, Word List Recall, and Non-word list Recall of the WM Test Battery for Children; Eve et al., 2016). However, improvements were not sustained at the 12-month follow-up. There were no significant improvements on measures of attention, academic capacity or other measures of WM (i.e., Block Recall [visuospatial WM], Listening Recall [verbal WM], and Backward Digit Recall [verbal WM] of the WM Test Battery for Children). Cogmed™ WM training was not recommended in its current form as an effective intervention for children with AIS. Limitations of this study concerned the design (e.g., single group and small sample size) and the results, namely the internal validity (e.g., no control group, maturation and repeated measurements; the testing effect) and the external validity (e.g., generalisability to the larger, target population). Eve et al. (2016) was considered a severe risk of bias.

Regarding paediatric cancer survivors, although reported to be measured post-training (i.e., after 8–12 weeks), no results were reported for cognition at this time point (Carlson-Green et al., 2017). Nevertheless, significant improvements were reported at the six-month follow-up on measures of verbal and visual WM (i.e., Digit Recall, Word Recall, Dot Matrix, Block Recall, Mr. X, Spatial Recall of the AWMA), as well as select academic measures (i.e., Applied Problems of the WJ-III). With regard to the level of activities, parents reported significant improvements in their child’s attentional problems and EF at home (i.e., the BRIEF); for example, ability to inhibit responses, monitor behaviour, shift between activities, WM, and planning and organisational skills. Limitations of this study concerned the design (e.g., single-group) and the results, namely the internal validity (e.g., no control group and use of parent-reported outcomes) and the external validity (e.g., selection of participants with a WM impairment; generalisation to the larger, target population). Carlson-Green et al. (2017) was considered a severe risk of bias.

In summary, Cogmed™ WM training demonstrated a significant medium-term improvement (i.e., 7–23 weeks; Ashton et al., 2020) on a single measure of visual WM (i.e., near-transfer effect) and on select measures of academic capacity (i.e., far-transfer effects) in paediatric TBI. However, the training did not generalise to performance on untrained tasks; therefore, it provided very weak evidence that Cogmed™ WM training improves WM. As for the efficacy of Cogmed™ WM training in paediatric AIS, significant short-term improvements (i.e., ≤6 weeks; Ashton et al., 2020) were demonstrated on verbal WM tasks dissimilar to the trained WM tasks (i.e., near-transfer effects). There were no significant improvements at the 12-month follow-up, therefore, Cogmed™ WM training was not recommended as an effective intervention for this patient group. In paediatric cancer survivors, significant long-term improvements (i.e., ≥24 weeks; Ashton et al., 2020) were demonstrated on a verbal WM task that differed from the trained tasks and on more similar visual WM tasks (i.e., near-transfer effects), as well as select academic capacity measures (i.e., far-transfer effects). With regard to the level of activities, parents reported significant improvements in their child’s attentional problems and EF at home. Although improvements in academic capacity and WM were demonstrated, strong inferences regarding the efficacy of Cogmed™ WM training could not be drawn due to the limited number of studies and the moderate- to severe risk of bias.

Executive functions training. One RCT evaluated the effectiveness of Mitii™, a web-based, multi-modal rehabilitation program to improve EF in children and adolescents with ABI (Piovesana et al., 2017). No significant differences in EF were observed post-training compared to the waitlist control group. With regard to the level of activities, there were no group differences observed for daily EF (i.e., the BRIEF). However, program adherence was poor. Mitii™ was not recommended in its current form as an alternative to traditional cognitive rehabilitation for children with ABI. The main limitation of this study concerned the results, namely the internal validity (e.g., no active control group, use of parent-reported outcomes and repeated measurements; the testing effect). Piovesana et al. (2017) was considered a moderate risk of bias.

General cognitive functions training. Other training programs did not target a specific cognitive domain, such as BrainGames and Lumosity Cognitive Training™.

One RCT evaluated the efficacy of Lumosity Cognitive Training™, a multi-domain, computerised cognitive training program in children with ABI (Corti et al., 2020). A significant improvement in visual WM (i.e., Corsi Block-Tapping test; near-transfer effect) was observed post-training and at the two-month follow-up for the training-first group, compared to the control group (i.e., waiting-first). In addition, a significant improvement on an academic measure of arithmetic calculation speed (i.e., Arithmetic Calculation task of the Italian battery AC-MT; near-transfer effect) was observed post-training, but not maintained at the two-month follow-up. No other significant effects were demonstrated (i.e., arithmetic calculation accuracy, cognitive flexibility, and mathematic problem-solving). Limitations of this study concerned the design (e.g., small control sample size) and the results, namely the internal validity (e.g., no active control group and repeated measurements; testing effect). Corti et al. (2020) was considered a moderate risk of bias.

A longitudinal, pilot study evaluated the feasibility of BrainGames, a home-based, computerised cognitive training program, in adolescents with TBI (Verhelst et al., 2017). Large effect sizes were demonstrated post-training in measures of attention (i.e., Continuous Performance Test and Flanker task of the Psychology Experiment Building Language battery; near-transfer effects) and a single measure of EF (i.e., Stockings of Cambridge of the Cambridge Automated Neuropsychological Test Battery [CANTAB]; near-transfer effect). A small effect size was reported for verbal WM (i.e., near-transfer effect). At the six-month follow-up, large effect sizes were demonstrated in attention (i.e., Continuous Performance Test and Flanker task; near-transfer effects), EF (i.e., Stockings of Cambridge and Intra-Extradimensional Set Shift of the CANTAB; near-transfer effects), verbal WM (i.e., Digit Span of the Dutch Wechsler Intelligence Scale for Children, 3^rd Edition [WISC-III-NL]; near-transfer effect), and visual WM (i.e., Spatial Span of the WISC-III-NL; near-transfer effect). Note that, whilst superior to baseline, effect sizes decreased for the attentional measures. Moreover, only a small effect size was reported for processing speed (i.e., Digit Symbol Coding of the WISC-III-NL; near-transfer effect). With regard to the level of activities, self- and parent-reported outcomes on the BRIEF revealed small post-training effects, with the exception of parent-reported Behavioural Regulation Index (e.g., cognitive flexibility, inhibitory control, and self-monitoring) and Global Executive Composite (i.e., summary score). Scores on the BRIEF improved at the six-month follow-up, which is reflected in medium to large effect sizes. Limitations of this study concerned the design (e.g., single-group and small sample size) and results, namely the internal validity (e.g., no control group, use of self- and parent-reported outcomes, and repeated measurements; testing effect) and the external validity (e.g., generalisability to the larger, target population). Verhelst et al. (2017) was considered a moderate risk of bias.

In summary, Lumosity Cognitive Training™ demonstrated a significant medium-term improvement on a single measure of visual WM (i.e., near-transfer effect) in children with ABI. In adolescents with TBI, BrainGames demonstrated medium- and long-term improvements in attention, EF and WM (i.e., near-transfer effects). With regard to the level of activities, children and their parents reported improvements in attentional problems and EF at home and school. Although the two studies demonstrated medium- and long-term improvements in general cognitive functioning, strong conclusions cannot be drawn due to the limited number of studies and the moderate risk of bias.

Virtual Reality training

One case study evaluated the efficacy of two rehabilitation training programs, including conventional cognitive therapy (CCT) and CCT in a Computer Assisted Rehabilitation Environment (CAREN) in a 15-year-old boy with severe TBI (De Luca et al., 2020). Following four weeks of CCT (20 sessions), improvements in attentional (i.e., the Modified Bell Cancellation Test) and executive processes (i.e., the WEIGL test) were observed, however were non-significant. Following four weeks of CCT in CAREN (20 sessions), improvements in attention and executive processes were significant. De Luca et al. (2020) employed the reliable change index to evaluate significance (i.e., RCI ≥ 1.96). The limitations of this study concerned the design (e.g., single-case A-B-A-B design, repeated-measures) and results, namely the internal validity (e.g., no control group and repeated measurements; the testing effect) and external validity (e.g., generalisability to the larger, target population and multiple treatment inferences). De Luca et al. (2020) was judged to be at severe risk of bias.

In summary, CCT in an iVR environment may hold promise for improving attention and executive processes at the ICF level of body function in paediatric TBI. However, to prevent erroneous conclusions, well-designed clinical studies with a low- to moderate risk of bias are needed to corroborate these results. Refer to Tables 3 and 4 for an overview of the findings.

Risk of bias in included studies

The included studies were rated as a moderate (n = 4) and severe (n = 3) risk of bias, the latter attributable to: (1) Selection bias and confounding, namely the potential for confounding of the effect of intervention (i.e., severity of disease); (2) Reporting bias, namely that the outcome measurements and analyses were defined in different ways in the methods and results sections. See Figures 2 and 3 for an overview and summary of risk of bias for the included studies.

Figure 2. Overview of risk of bias for the randomised controlled trials (ROB 2).

Note: D2 (Bias due to deviations from the intended intervention) – intervention effect of interest to the review author was the effect of assignment to the intervention at baseline (i.e., the “intention-to-treat effect”).

A 7 × 4 table portraying the risk of bias for the three randomized controlled trials. Risk of bias rated as some concerns.

Figure 3. Overview of risk of bias for the non-randomised controlled trials (ROBINS).

Note: D3 (Bias in classification of interventions) not included due to single condition; D4 (Bias due to deviations from the intended intervention) – intervention effect of interest to the review author was the effect of assignment to the intervention at baseline (i.e., the “intention-to-treat effect”).

A 9 × 5 table portraying the risk of bias for the four non-randomized controlled trials. Three studies rated the risk of bias rated as serious. One study rated the risk of bias as moderate.

Discussion

The overarching aim of the present review was to comprehensively evaluate the effectiveness of innovative, gamified interventions (i.e., AR, CBCR, and VR in conjunction with a Serious Game) for cognitive training in paediatric ABI at the level of function (i.e., NPA) and then at the level of activities (i.e., self/parent-reported outcomes on cognition in daily life). Findings were based on seven studies, published between 2016 and 2020. Training focused on remediation of general cognitive functioning (three studies), WM (three studies), or EF (one study).

Level of function

Computer-based cognitive retraining

The present review revealed no consistent evidence that CBCR was effective at the level of function; therefore, there was no clear pattern to explain the inconsistencies in results between studies. Despite the lack of improvement on most measures in individual studies, general improvements were observed in academic capacity, EF, general cognitive functioning (i.e., attention and processing speed), and WM. Importantly, there was evidence demonstrating that CBCR can have medium to long-term training-related effects, as the majority of studies that reported post-training improvements also reported improvements at follow-up (Corti et al., 2020; Phillips et al., 2016; Verhelst et al., 2017). Moreover, there was evidence that CBCR targeting a specific cognitive domain can positively influence performance in tasks dissimilar to the trained tasks, suggesting that far-transfer is possible. That is, improvements in academic capacity (i.e., mathematics and reading) were observed following Cogmed™ WM Training.

Lastly, training targeting general cognitive functioning appeared more promising than training targeting a specific cognitive domain. However, this was strongly driven by one study examining the feasibility of BrainGames (Verhelst et al., 2017), which demonstrated more widespread improvements in different cognitive domains. The improvements evidenced on the NPA may be due to task-specific practice effects, as the majority of minigames in BrainGames were adaptations of well-known cognitive tasks or existing cognitive training material (Verhelst et al., 2017); therefore, resulting in near-transfer effects.

Strong conclusions could not be drawn regarding the effectiveness of CBCR in improving cognitive functioning at the level of function, as there were no studies with a low risk of bias. Four studies (Corti et al., 2020; Phillips et al., 2016; Piovesana et al., 2017; Verhelst et al., 2017) were considered a moderate risk of bias, whilst two studies (Carlson-Green et al., 2017; Eve et al., 2016) were considered severe.

Virtual Reality

One single case study explored the efficacy of a CAVE iVR system (i.e., CAREN) in contrast to CCT in an adolescent with severe TBI (De Luca et al., 2020). At the level of function, significant improvements were demonstrated in attention and executive processes following iVR training, but not following CCT. De Luca et al. (2020) was considered at severe risk of bias. One of the methodological limitations concerned the fact that iVR training followed CCT; therefore, it was difficult to disentangle the effect of the active control condition from the benefit of the intervention (i.e., the carry-over effect). Therefore, the effect of iVR training for children with ABI remains unclear.

Level of activities

Three studies included an outcome measure at the level of activities (i.e., the BRIEF; Carlson-Green et al., 2017; Piovesana et al., 2017; Verhelst et al., 2017). Two studies revealed long-term improvements in attention problems and EF at home or school (i.e., the BRIEF: initiation, planning/organisation, task-monitoring, and WM; far-transfer effects) following CBCR training. There did not appear to be discernible differences between training targeting general cognitive functioning (i.e., BrainGames) or training targeting a specific cognitive domain (i.e., Memory; Cogmed™ WM Training). Despite the encouraging findings, the BRIEF focuses on subjective cognitive impairment, which may be biased as a result of the consequences of brain injury, or influenced by environmental and personal factors (Domensino et al., 2022). Underreporting of symptoms may be explained by a lack of knowledge about the invisible sequelae of brain injury among family members (Hochstenbach et al., 2005; Spreij et al., 2020). In addition, mild consequences of brain injury and the impact of brain injury on daily life may not be recognised or understood, leading to an overestimation of patients’ abilities (Fordyce & Roueche, 1986; Hochstenbach et al., 2005; Spreij et al., 2020). Therefore, findings should be interpreted with caution. No study evaluated the effectiveness of VR training at the level of activities.

Considerations for interpretation

Outcome measures

A critical comparison of results between studies was difficult due to the choice of outcome measure. For example, the most commonly measured cognitive domain was visual WM (five studies), however only a few studies utilised similar tasks to measure the same construct. The majority of included studies (Carlson-Green et al., 2017; Corti et al., 2020; De Luca et al., 2020; Verhelst et al., 2017) utilised a limited number of outcome measures (i.e., one or two tasks) per cognitive domain. For example, a RCT and a case series study employed only one or two tasks to assess EF (Corti et al., 2020; Verhelst et al., 2017; respectively). EF is a broad concept, encompassing a multitude of cognitive functions (e.g., cognitive flexibility, inhibition, and WM; Miyake et al., 2000); therefore, the limited number of outcome measures may be insufficient to identify true training-related effects (Van de Ven et al., 2016). In addition, an improvement observed on a single outcome measure tapping into a cognitive domain is less meaningful than improvements demonstrated on multiple outcome measures tapping into the same cognitive domain (Slagter, 2012; Van de Ven et al., 2017).

Conversely, three studies may have exposed their clinical cohort to an extended range of outcome measures (Corti et al., 2020; Eve et al., 2016; Verhelst et al., 2017). For example, the single-group study aimed to investigate whether there would be measurable and sustained changes in untrained WM tasks in children with AIS (Eve et al., 2016). Therefore, the hypothesis (i.e., improvements in “cognitive function”) was not justified and an extended range of outcome measures were included (e.g., academic capacity- and attention measures). The RCT explored the training effects on a myriad of secondary outcome measures (Corti et al., 2020). Moreover, the case series study aimed to investigate the feasibility of the intervention (Verhelst et al., 2017); therefore, the inclusion of an extended range of cognitive outcome measures was not justified. No study corrected for multiple comparisons. An extended range of outcome measures are subject to observer bias, increasing the likelihood of Type I error (i.e., an effect in the population, whilst in fact no effect is present) through use of multiple comparisons across a range of outcomes (Mascha & Vetter, 2018 ).

Lack of (active) control group

A critical comparison of results between studies was difficult due to the lack of (active) control groups. In total, four studies included a control group; only one study included an active control condition (Phillips et al., 2016). However, Phillips et al. (2016) employed a low-level, non-adaptive variant of the WM intervention. This may not have appropriately controlled for the effects of expectancy and motivation (Jones et al., 2020); children in the intervention group were provided with feedback and continuously challenged, which may have biased their performance at assessment (Jones et al., 2020). Nevertheless, the children in the active control group were supported with training aids and received the same support from coaches as children in the intervention group (Phillips et al., 2016). In order to achieve similar expectations and motivation at assessment (i.e., minimising potential biases), children in the active control group should be appropriately challenged and observe improvements (Jones et al., 2020).

The likelihood that findings were merely due to practice effects increases when there is no active control group. With the exception of two studies (Carlson-Green et al., 2017; Eve et al., 2016), all studies included in this review utilised the same task pre- and post-intervention. A case series study reported that improvements at follow-up were somewhat unexpected and may be attributable to maturation or test-retest effects (Verhelst et al., 2017). Conversely, maturation may explain the lack of improvement at follow-up. For example, a single-group study reported no significant improvements at 12-month follow-up (Eve et al., 2016). It is plausible that the CBCR did not have the same benefit at this stage of the child’s developmental trajectory and, therefore, reflects a plateau in cognitive capacity as other cognitive functions developed (Limond et al., 2014). Multiple baseline testing (i.e., sequentially introducing the intervention) may provide insight into the effectiveness of the intervention and minimise the effects of spontaneous recovery (Krasny-Pacini et al., 2014).

Study population

The majority of participants were at least one-year post-ABI (i.e., the chronic phase); therefore, it was not possible to ascertain whether positive training-related effects were dependent on the phase post-ABI. The effectiveness of an intervention may vary depending on the rehabilitation phase the participants were in at the time of intervention (Resch et al., 2020). This highlighted the general issue within the field; there is a paucity of studies systematically evaluating the effectiveness of cognitive training depending on the post-ABI phase, and the clinical question of when to train and what dosage to use.

Future research

In order to make strong inferences regarding the effectiveness of an intervention, it is essential that an active control group is employed (Van de Ven et al., 2016). Therefore, future studies should include both an active (e.g., adaptive visual search training; Jones et al., 2020) and passive (e.g., wait list) control group to control for placebo effects and the Hawthorne effect (i.e., possible expectation bias; Van de Ven et al., 2016). Moreover, an active control group would facilitate blinding of participants and personnel, minimisingpotential biases due to deviations from intended interventions and ensuring reliable assessment of outcomes (Van de Ven et al., 2016).

To facilitate a more meaningful comparison of results between studies, future research should include an agreed common core set of outcome measures to critically compare the effectiveness of different cognitive interventions (Reijnders et al., 2013). As a first step towards agreement on outcome measures, a recent study reviewed the most commonly used outcome measures with reference to the ICF framework in studies evaluating the outcome of neuropsychological interventions in adults with ABI over the past 20 years (Van Heugten et al., 2020). This study indicated the need to evaluate the quality of outcome measures, employ Delphi or consensus procedures, and involve rehabilitation professionals in the choice of outcome measures. However, the results from the aforementioned study in adults with ABI cannot be directly applied to paediatric ABI, as different outcome measures are required for this population. Therefore, future research should review the most commonly used outcome measures with reference to the ICF framework in studies evaluating the outcome of cognitive training in children with ABI over the past 20 years.

To bridge the gap between estimating cognitive functioning at the level of function and at the level of activities, future research should consider other, more ecologically valid, outcome measures (Green et al., 2019 ; e.g., a digital NPA; Spreij et al., 2021a or questionnaire; Spreij et al., 2021b). For example, the BRIEF (i.e., the questionnaire employed by three studies in the present review); a parent/guardian or self-reported outcome measure that focuses on a child’s behaviour in relation to daily EF in a natural setting (Hypher et al., 2019; Van Heugten et al., 2020; Vriezen & Pigott, 2010). Compared to performance-based measures (e.g., a NPA), which aim to estimate underlying cognitive skills, the BRIEF evaluates the application of cognitive skills at home and at school (McAuley et al., 2010). Therefore, the BRIEF is considered an ecologically valid measure of the subjective impact of cognitive impairment on performing activities of daily living (e.g., “trouble concentrating during play”).

Future research should; therefore, include a sufficiently large and varied battery of tasks to extract the cognitive domain (Green et al., 2019), whilst also considering the potential problem of multiple comparisons. Moreover, the use of composite scores could be considered, as they may be more sensitive in revealing training effects that consistently improve a specific domain, not just task-specific skills (Van de Ven et al., 2017).

Clinical implementation

The present review revealed insufficient evidence to provide clear recommendations for clinicians on whether to implement innovative gamified technology in paediatric clinical practice. That is, there were few studies that addressed this and the majority of studies were considered at moderate to severe risk of bias. Moreover, it is too early to formulate very clear recommendations for direct clinical implementation due to the unknown answers to clinical questions (e.g., when to train and what dosage to use). To enhance the effectiveness of interventions, it is important to determine when (i.e., phase of rehabilitation) and for whom (i.e., subpopulation) the intervention is most appropriate (Resch et al., 2020). In this domain, case-series methodology has attracted considerable interest (Ralph et al., 2011; Schwartz & Dell, 2010).

Innovative gamified interventions appeared feasible for training cognition (i.e., six of the seven included studies reported completion rates of more than 92%). Therefore, as a first step towards implementing innovative gamified interventions, pilot-, single case- and case series studies should be conducted in a clinical setting (i.e., the so-called “hybrid” design; Curran et al., 2012; Eldh et al., 2017). An advantage of single case methodology is the opportunity to determine whether a specific treatment works for a specific individual (i.e., identifying individual patients’ performance patterns), which may be important given patient heterogeneity. Clinicians may observe the effects of the innovative gamified intervention on relevant outcome measures whilst collecting information on implementation, or evaluate implementation whilst observing the effect on relevant outcome measures (Eldh et al., 2017). Nevertheless, in studies with a single case experimental design, three direct replications of one successful trial are required for external validity (Barlow et al., 2009). Researchers and clinicians should consider this before implementing an intervention into clinical practice.

Strengths and limitations

A potential limitation of the present review concerns the interpretation of results, which was limited by the small sample size and heterogenous sample of the included studies.

It is important to regard the findings from this review with caution when applying findings from the individual subpopulations (e.g., AIS) to the overall ABI cohort, due to differences in aetiology and consequences of brain injury (e.g., focal lesions of ischemia vs. focal and diffuse lesions caused by TBI). For example, the present review included three studies evaluating Cogmed™ WM Training, but each study focused on a different aetiology: AIS, cancerous brain tumour and TBI. Therefore, it was difficult to draw conclusions on the effectiveness of innovative, gamified interventions for cognitive training in children with ABI.

A second potential limitation is that the present review evaluated the effectiveness of innovative, gamified interventions for cognitive training at group level; therefore, individual differences were not considered – a potential limitation of the field.

Despite these limitations, the present review contributed to a better understanding of the available evidence base for innovative, gamified interventions for cognitive training in paediatric ABI; there is currently no evidence-based recommended cognitive intervention for paediatric ABI (Resch, 2019).

Conclusion

The present review included six CBCR studies and one iVR study, which highlighted the paucity of research investigating the effectiveness of innovative, gamified interventions for cognitive training in children with ABI, specifically mixed reality (e.g., AR and VR). Due to the small number of included studies with (relatively) small and heterogeneous samples, only a cautious interpretation of the evidence was provided.

CBCR targeting a specific cognitive domain (i.e., WM) only demonstrated improvements in WM (i.e., near-transfer effect) and in academic capacity (i.e., far-transfer effect). CBCR targeting general cognitive functioning appeared more promising, however, this was strongly driven by one study influenced by task-specific practice effects and multiple comparisons. A iVR study demonstrated improvements in EF and general cognitive functioning (i.e., attention), but the evidence was very weak. In two studies, CBCR demonstrated an improvement in cognition in daily life at home or at school. No study evaluated the effectiveness of iVR training at the level of activities.

Considering the investment of “time and effort,” stand-alone CBCR may not provide incremental benefits for children with ABI that may outweigh other more useful interventions (e.g., advice on managing daily routines). Metacognitive strategy training (e.g., MetaCogmed; Jones et al., 2020) may be an effective addition to CBCR (Jones et al., 2020; Resch et al., 2018). There is a need for more carefully designed studies, consensus on a core set of outcome measures per cognitive domain, more attention to inter-individual differences, and for the generalisation to daily life (Van Heugten et al., 2020).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Stichting Hartekind (20210406); Kinderrevalidatiefonds Adriaanstichting (20210045); SVLG (JeT1923.2); Special Interest Group Social & Cognitive Modeling; Research Foundation Flanders (FWO; 1249923N).