A longitudinal investigation of parental ratings and performance metrics for executive functioning and symptom severity in clinically referred youth with ADHD

Abstract

There remains a knowledge gap concerning the persistence of attention deficit hyperactivity disorder (ADHD) over the longer term. The current study aimed to investigate the change in parent-rated, and performance-based metrics of executive functioning (EF) and the relationship between these EF metrics and ADHD symptoms in individuals with ADHD from childhood/adolescence to young adulthood. This was done by examining possible improvements in parent-rated EF and performance-based measures of inattention and inhibition over a three-year interval and their relationship to ADHD outcomes in 137 clinically referred youth with ADHD (mean age = 12.4 years, SD = 3.1). Participants’ parents completed the Behavior Rating Inventory of Executive Function (BRIEF) and the Swanson-Nolan-Pelham Scale at baseline and follow-up. Participants completed the Conners’ Continuous Performance Test, Version II (CPT II) at baseline and follow-up. Statistical analyses were performed with Linear Mixed Models. The sex- and age-standardized measures Commission and Hit reaction time (RT) subscales of the CPT II and parent-rated metacognitive, and behavior regulation composites of the BRIEF were largely stable between measuring points. CPT Omissions, Hit RT standard error (reaction time variability), and parent-rated ADHD symptom scores improved slightly. BRIEF composites and reaction time variability were related to ADHD symptoms using longitudinal data. Overall, behavioral aspects of EF, as observed by parents in the home context, appear to play a significant role in the trajectory of childhood ADHD.

Keywords:

• ADHD
• assessment
• BRIEF
• CPT II
• executive functioning
• follow-up

Introduction

Attention deficit hyperactivity disorder (ADHD) is a highly heterogeneous condition (Silk et al., 2019; Steinhausen, 2009) with a heterogeneous course during childhood to adulthood (Hechtman et al., 2016; Ramos-Olazagasti et al., 2018). High levels of childhood ADHD symptoms, together with aggressive and/or defiant behavior, are associated with the persistence of the ADHD diagnosis and greater impairments in overall functioning (Lahey et al., 2016; Roy et al., 2017; Sasser et al., 2016). Even if the severity of the ADHD symptoms declines over time, the relative rank of symptom severity may remain and the negative impact of earlier ADHD symptoms may still be seen at follow-up (Hechtman et al., 2016; Molina et al., 2009; Owens & Hinshaw, 2016; Ramtekkar et al., 2010). Recent follow-up in the Multimodal Treatment of Attention Deficit Hyperactivity Disorder (MTA) study has shown that ADHD problems vary over time in most individuals who had ADHD as children, with only about 10% achieving full remission and about 10% experiencing persistent ADHD over time (Sibley et al., 2022). There remains an important knowledge gap in the literature concerning the persistence of ADHD over the longer term, and concerning the role played in these outcomes by variables other than the baseline ADHD severity and comorbidity (Lin & Gau, 2019; McAuley et al., 2017; Molina et al., 2009; van Lieshout et al., 2019).

Executive functioning

The term executive functioning (EF) refers to higher-order neurocognitive processes necessary for adaptive management of daily life, solving problems in new situations, and regulating thoughts and behaviors to attain goals (Andersson et al., 2008). Andersson et al. (2008) clarified that attentional control, i.e., selective attention, sustained attention, and response inhibition, is the first subcomponent of EF to emerge during childhood, also influencing performance on the other two EF components, i.e., goal setting and cognitive flexibility (Andersson et al., 2008). EF impairments underpin several neurodevelopmental disorders and have also been found to be associated with a general psychopathology factor (i.e., person’s susceptibility to a mental disorder) (Bloemen et al., 2018; Martel et al., 2017; Shanmugan et al., 2016). Impairments in attention control, working memory capacity, and planning ability have repeatedly been shown in people with ADHD in cross-sectional studies (Tarle et al., 2017; Willcutt et al., 2005). Although working memory “capacity” may not necessarily be affected, the ability to manipulate information via the central executive may be involved in ADHD (Kasper et al., 2012; Martinussen et al., 2005; Tarle et al., 2017). Likewise, task complexity also appears to affect planning ability more in children with ADHD than in controls (Patros et al., 2016, 2019). The longitudinal relationship between inattention and other EF deficits and the trajectory of ADHD symptoms remains under-investigated and based on the available studies, unclear (Agnew-Blais et al., 2020; Biederman et al., 2009; Coghill et al., 2014; Gordon & Hinshaw, 2020; Karalunas et al., 2017; Lin & Gau, 2019; Murray et al., 2017; van Lieshout et al., 2013, 2019; Vaughn et al., 2011; Wang et al., 2015).

There are two complementary ways of measuring attention and other EF in young people with ADHD: performance-based tests and rating scales (usually parent-rated). There is considerable evidence that scores on the performance-based measures are weakly correlated with the rating scales, and the use of both has been recommended to provide incremental information (non-overlapping information) and a multidimensional perspective on EF and related behaviors in people with ADHD (Barkley & Fischer, 2011; Biederman et al., 2008; Gerst et al., 2017; Häger et al., 2020; Krieger & Amador-Campos, 2018; Soto et al., 2020; Tan et al., 2018; Toplak et al., 2009, 2013). In both clinical work and research, EF is often examined using various standardized neuropsychological tests designed to measure the cognitive components of EF with a high degree of internal validity, controlling the environmental context in which these components are measured (i.e., via standardized tasks carried out in a laboratory or clinical setting) (Toplak et al., 2013). Continuous Performance Tests (CPTs) are computer-based test batteries widely used in ADHD assessing sustained attention, vigilance, inhibition, reaction time (RT), and RT variability, all of which are considered measurements of attention or attention control as subcomponents of EF (Hall et al., 2016; Huang-Pollock et al., 2012; Kofler et al., 2013; Willcutt et al., 2005). Meta-analytic investigations of studies employing CPTs find moderately strong cross-sectional correlations between EF deficits and ADHD severity (Frazier et al., 2004; Pievsky & McGrath, 2018; Willcutt et al., 2005). At least moderate effect sizes in reaction time variability (.66), vigilance (.56), and response inhibition (.52) were found in a systematic review and quantitative summary of 34 meta-analyses examining the most significant neurocognitive impairments in individuals with ADHD compared with typically developed control groups (Pievsky & McGrath, 2018). Age moderated the associations, because children and adults displayed greater impairments than did adolescents (Pievsky & McGrath, 2018). Despite the good validity of CPTs in terms of their ability to identify neuropsychological difficulties in young people with ADHD, these measures have been criticized for limited ecological validity as an index of how these measures relate to behavior in more naturalistic settings (Barkley, 2016; Barkley & Fischer, 2011; Kofler et al., 2013; Munkvold et al., 2014; Toplak et al., 2013).

Contrary to CPTs, behavioral rating scales of EF are accepted as an ecologically valid method for assessing the behavioral manifestation of EF (Andersson et al., 2008; Davidson et al., 2016; Gioia et al., 2000, 2002; McAuley et al., 2010; McCandless & O’ Laughlin, 2007; Tan et al., 2018). Scores on EF rating scales, in parent-format for youth and self-report format for adults, show moderate to high correspondence with ADHD across the age range (Barkley & Fischer, 2011; Dehili et al., 2017; Silverstein et al., 2020; Tan et al., 2018; Toplak et al., 2009, 2013). These scales have been proposed to identify people with high rates of ADHD symptoms and psychiatric comorbidity, compared with performance-based tests that identify people with cognitive and achievement difficulties (Biederman et al., 2008). Behavioral EF ratings' strong relationship with ADHD and simultaneously weak relationship with EF tests have raised the concern that parental ratings are merely proxy measures of ADHD symptoms (McAuley et al., 2010). Again, like other EF metrics, parental ratings of EF are also aligned with impairment in other developmental and acquired conditions, all having both similar and discriminating patterns of impairment (Hovik et al., 2017; Jacobson et al., 2020; Leung & Zakzanis, 2014; Mahone et al., 2002; Semrud-Clikeman et al., 2010; Weyandt et al., 2017). Also, it is important to consider the overlap between the symptoms of these conditions and the high degree of comorbidity between them (Chang et al., 2020; Chen et al., 2018; Jensen & Steinhausen, 2015).

Longitudinal studies of EF and ADHD

The longitudinal relationship between EF, when measured using CPTs, and ADHD symptom severity during childhood, adolescence, and young adulthood remains unclear. In general, both persisters and remitters of childhood ADHD tend to perform more poorly on CPTs over time than do typically developing controls, which implies that these metrics are long-lasting in ADHD (Gordon & Hinshaw, 2020; Lin & Gau, 2019; van Lieshout et al., 2019; Wang et al., 2015). There is some support that individuals with ADHD may experience minor gains in measures of age-adjusted CPT scores over time (i.e., omission and commission errors) (Murray et al., 2017; Wang et al., 2015). However, changes in CPT measures over time do not appear to be reliable predictors of symptomatic outcomes in persons with ADHD (Coghill et al., 2014; Gordon & Hinshaw, 2020; Karalunas et al., 2017; Lin & Gau, 2019; McAuley et al., 2014; Murray et al., 2017; van Lieshout et al., 2019; Vaughn et al., 2011; Wang et al., 2015).

The World Federation of ADHD is addressing a knowledge gap by seeking the causal biological or psychological mechanisms that contribute to symptom reduction or improved function in children, adolescents, and young adults with ADHD. Such research is necessary in order to achieve better treatment effects by finding new or optimizing current treatment methods (Faraone et al., 2021). To our knowledge, no published longitudinal studies have administered an EF rating scale and measures of ADHD symptoms at more than one time point to participants reaching adulthood.

Aims

The primary aim of this study was to investigate the change in parent-rated EF and the relationship between parent-rated EF and ADHD symptoms in individuals with ADHD from childhood, through adolescence, and up to young adulthood. Given the absence of studies employing parental ratings of EF in youth with ADHD measured at two or more time points during their childhood years, this study is an exploratory prospective observational study aiming to explore the change in parent-rated age- and sex-standardized measures of EF and their possible long-term associations with parent-rated ADHD symptoms. We also studied the change in parent-rated ADHD symptoms and child-completed CPTs between baseline and three-year follow-up and their long-term associations. Participants were 7–17 years old at baseline and were reassessed three years later. Parent-rated ADHD symptoms (measured using summary scores of the Swanson-Nolan-Pelham Scale, fourth edition [SNAP-IV] (Bussing et al., 2008)) and parent-rated EF (measured using age- and sex-standardized T scores of the Behavior Rating Inventory of Executive Function [BRIEF] (Gioia et al., 2000)) were collected at baseline and three-year follow-up. The youth completed performance-based inattention measures (using age- and sex-standardized T scores of Conners’ CPT II (Conners, 2002)) at the same time points.

Material and methods

Study group

The participants were clinically referred children and adolescents with ADHD: n = 137 (males = 96, females = 41), mean age = 12.4 years (SD = 3.1; range = 6.7–17.9). Participants were recruited between January 1, 2011 and December 31, 2012 from consecutive diagnostic assessments carried out at the Neuropsychiatric Unit of the Child and Adolescent Psychiatry Clinic (CAP) in Lund, Sweden. The study group has been described previously (Tallberg et al., 2021). The inclusion criteria were as follows: (1) a current diagnosis of ADHD according to the criteria outlined in the Fourth Edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (American Psychiatric Association, 1994); (2) fluency in Swedish, and (3) absence of a known intellectual disability. Written informed consent was obtained from all caregivers and participants aged 15 years and older. This study was approved by the Research Ethics Committee at Lund University, Lund, Sweden (Reg. No. 2012/88) and registered in ClinicalTrials.gov (ID: NCT04201509, protocol ID: 2012/88).

Rating scales

Behavior Rating Inventory of Executive Function

The BRIEF is an 86-item rating scale measuring everyday behaviors associated with EF in children aged 5–18 years, hereafter referred to as parent-rated EF (Gioia et al., 2000). In the current study, the parental version of the BRIEF (Gioia et al., 2000) was administered at baseline and follow-up. Individual items are rated on a three-point frequency scale (1 = never, 2 = sometimes, and 3 = often), summed to yield an overall General Executive Composite (GEC) and two index scores: Behavior Regulation Index (BRI), based on the subscales: Inhibit, Shift, Emotional Control, and Monitor and Metacognitive Index (MI), based on the subscales: Initiate, Working Memory, Plan/Organize, and Organization of Materials. Metacognition is about processes that serve the regulation of cognition (Efklides, 2008). Raw scores are converted to age- and sex-specific T scores with a mean of 50 and standard deviation (SD) equal to 10, based on a U.S. national standardization sample (n = 1419; 815 girls, 604 boys, aged 5–18 years). Higher T scores indicate greater levels of EF impairment. The composite and index scores have been found to have good psychometric properties including high internal consistency (0.89–0.98), test–retest reliability (0.76–0.91), and expected correspondence with other respondent-based ratings of executive function (Gioia et al., 2000).

At follow-up, some participants were more than 18 years old, and their parents completed the 75-item informant version of the BRIEF-Adult (BRIEF-A) (Roth et al., 2005). Scoring is similar to the BRIEF with the exception that the BRI index also includes a self-monitor subscale. Age-specific T scores are based on an older U.S. national standardization sample (N = 1200). Psychometric support for the composite and index scores for the BRIEF-A includes high internal consistency (0.95–0.98), test–retest reliability (0.96), and expected correspondence with other respondent-based ratings of executive function (Roth et al., 2005).

Swanson-Nolan-Pelham Scale

The Swanson-Nolan-Pelham Scale, Fourth Edition (SNAP-IV) is a DSM-IV-based ADHD rating scale in a parent and teacher report format consisting of 26 items rated on a 4-point scale (0 = not at all, 1 = just a little, 2 = quite a bit, 3 = very much) (Bussing et al., 2008; Hall et al., 2020). The SNAP-IV consists of three subscales with subscales scores based on the mean score for the particular scale: inattention symptoms (nine items), hyperactivity/impulsivity symptoms (nine items), and the ADHD-combined score (Bussing et al., 2008). The internal reliability coefficient for the three subscales is in the acceptable range (alpha = .90, .79, and .89, respectively) (Bussing et al., 2008). The SNAP-IV has been found to be valid for use in identifying children with varying levels of ADHD but not as a diagnostic instrument (Bussing et al., 2008; Hall et al., 2020). In the present study, the continuous values of the raw scores of parental SNAP-IV ratings were used as a measure of baseline and follow-up levels of ADHD symptoms, which are the same for children and adults (DSM-IV). Parents completed SNAP-IV even for children over the age of 18 years, as all participants remained with their families.

Conners’ Continuous Performance Test, version II

Version II of the Conners’ Continuous Performance Test (CPT II) is a computerized, visual task of attention, vigilance, and response inhibition that is widely used in the assessment of ADHD in children and adults aged six years and above (Conners, 2002; Conners et al., 2003). The participant is instructed to press the space bar whenever any letter (target) other than the letter “X” (the “non-target”) appears on the screen. Targeted and non-targeted stimuli (letters) are randomly shown for 250 ms, with one, two, and four inter-stimuli intervals between presentations and with the overall test lasting 14 minutes. The CPT II yields twelve performance measures, of which four are used in this study: Omissions, Commissions, Hit RT (mean), and Hit RT standard error (SE). Raw scores are converted to age- and sex-specific T scores with a mean of 50 (SD = 10) based on a standardization sample including 1920 healthy individuals from the general population and 378 individuals with ADHD (Conners, 2002). The omission score is the number of times the participant failed to press the space bar when a target letter was presented. The commission score is the number of space bar presses in response to a non-target letter. The Hit RT (mean) is the average speed of correct responses for the entire test; higher values indicate slow response time. The Hit RT standard error (SE) is a measure of reaction time variability. Psychometric support for the individual CPT II measures is provided in the manual: split-half reliability for Omissions (0.94), Commissions (0.83), Hit RT (0.95), and Hit RT SE (0.87); three-month test–retest reliability coefficients for Omissions (0.84), Commissions (0.65), Hit RT (0.55), and Hit RT SE (0.65) (Conners, 2002).

Procedure

Baseline assessments

The diagnosis of ADHD was based on the DSM-IV criteria according to international guidelines (NICE, 2008) using information from multiple sources, including: a comprehensive psychiatric interview by a child and adolescent psychiatrist with the child and his or her parents; semi-structured telephone interviews with the child’s teachers; parent-rated SNAP-IV and BRIEF questionnaires; and the child-completed CPT II, administered by a clinical psychologist. The baseline data consists of the assessment results of the SNAP-IV, the BRIEF, and the Conners’ CPT II collected 2011–2012. Figure 1 presents descriptive information including the year when the assessments were completed, the number of participants who completed each of the measures. All participants were medicine-naïve at the baseline assessment.

Figure 1. Flow-chart. BRI: behavioral regulation index; BRIEF: Behavior Rating of Executive Function; CPT: Continuous Performance Test; HI: hyperactivity impulsivity; I: inattention; MI: metacognition index; SNAP-IV: Swanson-Nolan-Pelham Scale, fourth edition. *Seven parents used BRIEF (parent version) for children older than 18 years old.

Follow-up assessments

Approximately three years after the baseline assessment, participants were invited for reassessment; one participant was under 10 years old, 50 were 10–14 years old, 37 were 14–18 years old, and 23 were over 18 years old. For descriptive information about the number of participants that completed the tests, see Figure 1. All participants were asked about the current use of ADHD medications (yes/no) and were asked to stop taking any ADHD medication 24 h prior to the follow-up assessment. All follow-up assessments were carried out at the CAP clinic 2014–2015. No diagnostic assessments were conducted at follow-up; however, symptoms of ADHD were measured using parental ratings of SNAP-IV. Furthermore, the CPT II was re-administered under the supervision of a clinical psychologist/trainee trained in administering this test, and the parental versions of BRIEF/BRIEF-A were collected.

Statistical analyses

Group comparisons with Student’s t-test and chi-square were carried out using version 25 of SPSS (IBM Corp., Armonk, NY, United States). Linear mixed model regression analyses (LMM) were carried out using version 9.4 of SAS (SAS Institute Inc., Cary, NC, United States). The current study is an exploratory prospective observational study in which we used LMM to find possible associations, while not aiming to establish causal relationships. Results yielding an alpha level <.05 were defined as significant concerning the attrition analyses. Because of the exploratory nature of the current study, an alpha level of <.01 was used since we wanted to balance the risk of type 1, and type 2 errors (Rubin, 2017). Unstandardized b-values with corresponding 99% confidence intervals (CIs) and p-values are reported.

Changes in performance-based inattention and parent-rated EF and ADHD symptom severity across time points

To evaluate change over time, LMM with an autoregressive covariance structure were used. By using LMM the effect of the attrition is reduced since all available data is included in the analyses. An autoregressive covariance matrix structure is used to fit a model when the time between assessments is equal for all subjects, and the resulting parameter estimates assume that the variances in scores are constant across all measurement times. When an autoregressive covariance matrix is used to fit a model, there is generally no need to fit a random intercept (Vonesh, 2014). Dependent variable was the follow-up time between baseline and follow-up. The independent variables were baseline and follow-up measures of parent-rated ADHD symptoms assessed using the SNAP-IV (unstandardized summary scores) for the inattention symptoms and hyperactivity/impulsivity symptoms, age- and sex-standardized T scores for parent-rated EF: MI and BRI from the BRIEF and BRIEF-A (parent reported), and performance-based age- and sex-standardized T scores of inattention: Omissions, Commissions, Hit RT (mean), and HIT RT SE from the CPT II.

Relationships between performance-based inattention and parent-rated EF and ADHD symptoms across time points

We analyzed the effect of parent-rated BRIEF composites and CPT II subscales on ADHD symptoms using LMM, including baseline and follow-up data for all measures simultaneously. An example of the equation, using matrix notation, of the regression analyses in this study is $$\mathrm{Y}=\mathrm{X}\beta +\text{Ku}+\mathrm{\epsilon .}$$

In this study, Y is a vector of SNAP results, X is the matrix of fixed effects (e.g. BRIEF composites), β the vector of fixed effects parameters and ε is the vector of errors. ε is assumed to be normally distributed with expected value 0 and, due to the longitudinal design of this study, its covariance matrix having an autoregressive structure. K and u is usually the matrix of random effects and the random effects parameters vector, respectively, but no random effects (intercept or slope) are included in the model. In theory, a random intercept and a random slope model could have been used, but since there was a maximum of two observations per subject this model would have overfitted the data (Bates et al., 2015). The associations between EF and ADHD symptoms were analyzed including all non-missing observations from both baseline and follow-up, i.e., baseline data were included in the models and affected the results even though there were no follow-up data in some cases. The analyses were performed as non-adjusted analyses, i.e., only the actual parameter was included as the independent variable.

Adjustment analyses

Since the results may be affected by medication treatment, all analyses were controlled forADHD medication status (yes/no) to reduce the effect of possible medication (Losier et al., 1996) (not all participants were treated with ADHD medication, and all participants were asked not to take their medication 24 h before the follow-up assessments). Since socioeconomic factors may affect the results, we adjusted for parental education level (Rieppi et al., 2002). Except for the SNAP-IV, all measures were standardized according to age and sex, so we did not control for age and sex. We adjusted for age and sex in the analyses of how SNAP-IV changed between baseline and follow-up. If there were any discrepancies between the results from non-adjusted and adjusted model, we performed the non-adjusted model with the same number of observations as the adjusted to verify that the discrepancies were not due to subjects excluded due to missing values of variables included in the adjusted model.

Results

Sociodemographic characteristics, attrition, and clinical data

Of the 137 participants included at baseline, 96 (70%) were boys. The sex ratio at the follow-up assessment was similar. The parents’ level of education at baseline were: primary school only (to age 16) = 11%; high school (age 16–19 years) = 46%; University or above = 43%. Altogether, 111 of the 137 participants (81.0%) took part in one or more assessment methods at both the baseline and follow-up assessments. There was partial attrition concerning different methods (see Figure 1 and Table 1). Participants lost to follow-up scored higher at baseline than those who completed the follow-up on the SNAP-IV inattention scale, t(126) = 2.09; mean difference (M_diff) = 2.36; 95% CI [0.13, 4.60], p = .038, and the BRIEF MI, t(123) = 2.15; M_diff = 4.60; 95% CI [0.36, 8.35], p = .034. No other significant baseline differences were found between completers and non-completers on the BRIEF, SNAP-IV, or any of the CPT II indexes/scales. Table 1 presents the means and standard deviations at baseline and follow-up, all age- and sex-standardized T Scores, for the CPT-IICPT II, and BRIEF, and summary scores for the SNAP-IV, also shown in the figure in Supplement Material. Reliability for collected data of the SNAP-IV inattention at baseline was α = .80, and for hyperactivity/impulsivity was α = .90.

Table 1. Demographic data, mean values at baseline and follow-up, and standard deviations (SDs) for the SNAP-IV, BRIEF, and CPT II at baseline and follow-up.

	Baseline scores (2011–2012)			Follow-up scores (2014–2015)
Measures	n	%		n	%
Sex: boys	96	70		77	69
Older than 18	–	–		23	21
Medication, yes	–	–		82	81
Parental education:
Primary school	12	9		11	10
High school	50	36		38	34
University	46	34		39	35
	n	Mean	SD	n	Mean	SD
SNAP inattention	128	18.04	5.05	108	15.11	6.30
SNAP hyperactivity/impulsivity	127	12.40	13.14	108	9.20	6.23
BRIEF BRI	127	66.72	13.67	108	63.94	12.61
BRIEF MI	125	68.65	9.09	108	65.75	10.71
CPT Omissions	133	58.08	15.36	101	53.16	15.62
CPT Commissions	133	53.60	9.06	101	55.70	12.46
CPT RT (mean)	133	53.42	12.54	101	52.62	10.88
CPT RT (SE)	133	58.15	10.49	101	54.47	12.15

Notes. ADHD: attention deficit hyperactivity disorder; BRI: behavioral regulation index; BRIEF: Behavior Rating Inventory of Executive Function; CPT: Continuous Performance Test; MI: metacognition index; RT: reaction time; SE: standard error; SNAP-IV: Swanson-Nolan-Pelham Scale, fourth edition. The number of observations varied due to partial attrition. The CPT II and BRIEF were age- and sex-standardized and raw scores were converted to T scores (mean = 50; SD = 10), BRIEF-A, age standardized. Parent-rated SNAP-IV was measured as a summary score. Higher scores indicate greater impairment in all measures except CPT Hit RT (mean). Lower scores in Hit RT (mean) indicate faster RT.

Improvements in ADHD symptoms, performance-based inattention, and parent-rated EF from baseline to follow-up

Figure 2 shows the slopes of the LMM analyses to examine the change in ADHD symptoms and EF between baseline and follow-up. Significant improvements between baseline and follow-up were found for inattention symptoms (SNAP-IV), a score decrease of 1.80, omissions (CPT II), a score decrease of 4.50, and the Hit RT (SE) (CPT II), a score decrease of 3.45 in non-adjusted analyses, and when adjusting for the control variables (ADHD medication use and parents’ level of education, and for SNAP-IV we also controlled for age and sex). Non-adjusted analyses showed statistically significant changes from baseline to follow-up for hyperactivity/impulsivity symptoms from the SNAP-IV, and the MI scale from the BRIEF. When adjusting for the control variables (age, sex, ADHD medication use and parents’ level of education), the change in hyperactivity/impulsivity symptoms did not remain significant (p = .216). The controlling variable age, but no other controlling variables, was significantly associated with hyperactivity/impulsivity at baseline and follow-up (p = .007). When adjusting for the controlling variables (ADHD medication use and parents’ level of education) the decrease BRIEF MI (p = .313) did not remain significant. No significant baseline to follow-up change was found for commission scores, Hit RT (mean) from the CPT II, or the BRI index from the parent-rated BRIEF.

Figure 2. The estimates (b), corresponding 99% confidence intervals, and a forest plot for the slopes of change between baseline and follow-up of the SNAP-IV, the BRIEF, and the Conners’ CPT II.

Notes. ADHD: attention deficit hyperactivity disorder; BRI: behavioral regulation index; BRIEF: Behavior Rating Inventory of Executive Function; CI: confidence interval; CPT: Continuous Performance Test; MI: metacognition index; RT: reaction time; SE: standard error; SNAP-IV: Swanson-Nolan-Pelham Scale, fourth edition. The number of observations varied due to partial attrition. The CPT II and BRIEF were age- and sex-standardized and raw scores were converted to T scores (mean = 50; standard deviation (SD) = 10), BRIEF-A standardized to age. Parent-rated SNAP-IV was measured in summary scores. Higher scores indicate greater impairment in all measures. In CPT II, lower scores for Hit RT (mean) indicate faster RT.

Relationships between parent-rated EF, and performance-based inattention and ADHD symptoms

Figure 3 presents the results of LMM analyses, examining the effects of BRIEF and CPT II on ADHD symptom severity (SNAP-IV) using baseline and follow-up data simultaneously. Inattention and hyperactivity/impulsivity scores were significantly associated with sex- and age-standardized T scores of the MI and BRI composites from the BRIEF, and the Hit RT (SE) subscale from the Conners’ CPT II in non-adjusted analyses, and after adjusting for the control variables (ADHD medication use and parents’ level of education at baseline). Inattention symptoms were significantly associated with Hit RT (mean), and hyperactivity/impulsivity symptoms were significantly associated with omission scores in non-adjusted analyses. When adjusting for the control variables (ADHD medication use and parents’ level of education at baseline) the associations between Hit RT (mean) and inattention symptoms (p = .027), and between Omissions, (p = .012) and hyperactivity/impulsivity symptoms did not remain significant.

Figure 3. Non-adjusted linear mixed models analyzing associations between the SNAP-IV subscales and BRIEF indexes, and CPT II variables measured at baseline and follow-up.

Notes: ADHD: attention deficit hyperactivity disorder; b: unstandardized value of the estimate; BRI: behavioral regulation index; BRIEF: Behavior Rating of Executive Function; CPT: Continuous Performance Test, Version II; CI: confidence interval; MI: metacognition index; RT: reaction time; SE: standard error; SNAP-IV: Swanson-Nolan-Pelham Scale, Fourth Edition. The number of observations varied due to partial attrition. The CPT II and BRIEF were age- and sex-standardized and raw scores were converted to T scores (mean = 50; standard deviation (SD) = 10), BRIEF-A standardized to age. Parent-rated SNAP-IV was measured in summary scores. Higher scores indicate greater impairment in all measures. In CPT II, lower scores for Hit RT (mean) indicate faster RT.

Discussion

The present study aimed to fill a gap in the literature with respect to the longitudinal relationships between parent-rated EF, performance-based inattention and ADHD symptoms, in a clinical sample of youth diagnosed and treated for ADHD. One aim was to examine the change in ADHD symptoms and standardized measures of parent-rated EF (BRIEF) and performance-based inattention (CPT II), between baseline and three-year follow-up. Another aim was to investigate the relationships between parent-rated EF and ADHD symptoms, and performance-based inattention and ADHD symptoms, using longitudinal data.

We observed a clinically small, but statistically significant, improvements in the severity of parent-rated ADHD symptoms between the baseline and three-year follow-up assessments. In contrast to the improvements in inattention symptoms, the improvements in hyperactivity/impulsivity were mainly explained by older age at baseline and follow-up (not by the change between follow-up and baseline scores). Research results concerning how ADHD symptoms change over time from childhood to young adulthood remain unclear, since studies report various remission rates of 30–95% (Agnew-Blais et al., 2016; Faraone et al., 2006; Hechtman et al., 2016; Lahey et al., 2016; Moffitt et al., 2015; Sasser et al., 2016; Swanson et al., 2017). The MTA study recently reported that the ADHD symptoms in most people with ADHD seem to vary between periods of remission and persistence (Sibley et al., 2022). Furthermore, some studies have shown that different symptoms relevant to ADHD seem to vary between different ages (Holbrook et al., 2016; Larsson et al., 2011; Martel et al., 2016; Sasser et al., 2016). Barkley (2016) and Ramtekkar et al. (2010) suggested that using standardized measures of ADHD symptoms may improve the assessed ADHD at different ages and provide guidance for how to interpret improvements in symptoms (Barkley, 2016; Ramtekkar et al., 2010). While ADHD symptoms are not standardized by age, clinicians are supposed to weigh the clinical impairment against the child’s level of development (American Psychiatric Association, 1994), which seems to be difficult given the pattern of results identified in previous studies.

When using age- and sex-standardized measures of parent-rated EF (BRIEF) we found largely stable composite scores across baseline and three-year follow-up. Similarly, standardized scores on the child-completed, performance-based measure of inattention were largely stable between baseline and follow-up concerning commission scores and mean reaction time (Hit RT (mean)). Significant improvements for omission scores, and reaction time variability (Hit RT (SE)) were found. The pattern of our results is partly consistent with previous literature noting that the differences between children and young people with ADHD and normative groups on performance-based measures of EF, including the CPT II, have been found to be largely stable over time, with both persisters and remitters from childhood ADHD tending to perform more poorly compared to typically developing controls (Gordon & Hinshaw, 2020; Lin & Gau, 2019; van Lieshout et al., 2019; Vaughn et al., 2011). Age-related impairments in EF may be largely persistent, underlying impairments across ages and disorders, and not specific to people with ADHD.

Parent-rated EF composite scores were associated with ADHD symptoms using longitudinal data collected at baseline and three-year follow-up. While helping to fill a gap in the ADHD literature, this is in accordance with previous cross-sectional findings that behavioral ratings of EF are associated with ADHD (Barkley & Fischer, 2011; Dehili et al., 2017; Silverstein et al., 2020; Tan et al., 2018; Toplak et al., 2009, 2013). However, behavioral EF is also associated with other conditions, often comorbid with ADHD (Hovik et al., 2017; Jacobson et al., 2020; Leung & Zakzanis, 2014; Mahone et al., 2002; Semrud-Clikeman et al., 2010; Weyandt et al., 2017). The strong relationship between EF and ADHD can partly be explained by the fact that the core symptoms of ADHD, i.e., impairments in attention control and inhibition, at the same time constitute basic EF, influencing the development of more advanced EF from childhood to young adulthood (American Psychiatric Association, 1994; Andersson et al., 2008). Furthermore, Biederman (2008) found that performance-based EF identified individuals with cognitive impairments while EF questionnaires largely identified individuals with higher levels of ADHD symptoms, psychiatric comorbidity, and interpersonal deficits, which has, in turn, been found to be a major contributor to the severity of functional impairments (Biederman et al., 2012; Qian et al., 2016; Sasser et al., 2016; Shaw et al., 2014).

The present study found significant associations between reaction time variability (Hit RT (SE)) and ADHD symptoms using longitudinal data collected at baseline and three-year follow-up. Intra-individual variability has been considered to be a core feature of ADHD and refers to moment-to-moment (within-subject) fluctuations in behavior and task performance (Tamm et al., 2012). Our longitudinal study design extends previous cross-sectionally findings, on the validity of reaction time variability and ADHD (Kofler et al., 2013; Pievsky & McGrath, 2018). However, there were no other significant associations between performance-based inattention and parent-rated ADHD symptom severity. The current results are largely consistent with previous studies which have found non-significant correspondence over time between scores on performance-based measures of CPTs and ADHD outcomes (Coghill et al., 2014; Gordon & Hinshaw, 2020; Karalunas et al., 2017; Lin & Gau, 2019; Murray et al., 2017; van Lieshout et al., 2019; Vaughn et al., 2011; Wang et al., 2015).

Differences in the relationship between parent-rated ADHD symptom severity and parent-rated EF and performance-based measures of inattention may reflect a form of method variance, i.e., variability in the observed relationship owing to informant and the method of assessment. Meta-analyses have found higher correspondence between respondents from the same environmental contexts than between respondents from different environmental contexts (e.g., parents at home versus teachers in school) (Achenbach et al., 1987; De Los Reyes et al., 2015). Therefore, our results indicating an association between parent-rated EF and parent-rated ADHD outcomes may not be generalizable to outcomes rated by other informants (e.g., teachers’ rating of academic performance and behaviors). However, the present results can be viewed as providing support for the long-term validity of parent-ratings of behavioral manifestations of EF in the home context in relation to parent-rated ADHD symptoms.

The results of the current study imply that both performance-based inattention tests and behavioral, parent-rated EF show long-term validity identifying impairments in children and young people with ADHD. Behavioral EF appear to be psychological mechanisms that should be taken into consideration when developing new treatment approaches in ADHD. In interview studies, youth with ADHD have expressed that they need scaffolding support of EF shortcomings (Eccleston et al., 2019). A meta-analysis found that adding increased physical exercise to standard ADHD treatment reduced ADHD symptoms and EF-related behavioral impairments in youth with ADHD (Christiansen et al., 2019), as well as emotional symptoms (Cerrillo-Urbina et al., 2015). So far, there is a limited indication that computerized cognitive/EF training programs improve outcomes in respect of core symptoms of ADHD (Bikic et al., 2017; Cortese et al., 2015; Meyer et al., 2020; Rapport et al., 2013). However, one meta-analysis found that organizational skills training in children with ADHD improves these skills moderately according to ratings by teachers and large improvements as rated by parents (Bikic et al., 2017). Taken together, further work is needed to enhance the efficacy of EF interventions in ADHD youth. For example, more needs to be done to understand how medical treatments, as well as parent-focused interventions, interact with child-focused EF training programs (Fay-Stammbach et al., 2014; Mazursky-Horowitz et al., 2018; Valcan et al., 2018).

Limitations

The results of the current study must be viewed within the context of certain limitations. We have examined a treatment-seeking group of youth with ADHD in a naturalistic clinical context. No validated structured diagnostic interviews were included in the diagnostic process at baseline. The diagnoses of ADHD were assessed according to international guidelines (NICE, 2008) by child psychiatrists in conjunction with licensed clinical psychologists, all of whom were senior clinicians with specialist training in the assessment and treatment of ADHD. One hundred and eleven (81%) completed one or more of the assessment methods at the follow-up in the current study. However, there was partial attrition concerning these methods. The parents of the young people who did not participate in the BRIEF and SNAP-IV ratings rated the drop-outs’ inattention symptoms and metacognitive functioning worse than those who participated in the follow-up assessments. It cannot be ruled out that the attrition affected the results despite the mixed level modeling techniques used in this study, employing all available data to analyze the associations between ADHD symptoms and EF, so that the findings represent all participants. Another limitation concerns the BRIEF: for the BRIEF-A, the reported T scores are only age specific, whereas the parent-reported BRIEF has both age- and sex-specific T scores. There was no reliability analyzes for collected data of follow-up measures of the SNAP-IV, or baseline and follow-up data of the BRIEF. Though, it may be argued that the analyzes of change are long-term test–retest analyzes that supports the reliability and validity of the scales in the clinical ADHD population. Furthermore, we performed the attrition analyzes using the student’s t-test in spite of ordinal raw scores of the scales. Finally, no power analysis was conducted before the analyses.

Summary

Standardized measures of parent-rated EF and performance-based inattention were largely stable over a three-year interval, except for improvements in reaction time variability and omission scores. The present study found a slight improvement in ADHD symptom scores over this same interval. Performance-based tests and parent-rated behavioral EF showed long-term validity identifying impairments in children and young people with ADHD. Parent-rated EF of behavior regulation and metacognition index scores and reaction time variability were associated with ADHD symptoms using longitudinal data collected at baseline and three-year follow-up.

Conclusion

While further longitudinal studies of clinically referred youth are needed, there appears to be a long-term relationship between parent-rated executive functioning and ADHD in youth, suggesting that parent-ratings of EF can be an important part of assessment and treatment planning for youth with ADHD.

Data availability statement

Due to the nature of this research, participants of this study did not agree for their data to be shared publicly, so supporting data is not available.