Further validation of a new ADHD screening questionnaire measuring parents’ explanations (time processing, cognition, and motivation) of inattention symptoms in their school-aged children

ABSTRACT

The triple pathway model suggests that different neuropsychological factors underlie symptoms of inattention (i.e., time, cognition and/or motivation problems). However, screening instruments asking individuals to judge the link between these neuropsychological factors and inattention are lacking. The recently developed screening questionnaire, PASSC, aims to examine these factors possibly causing inattention by asking parents to indicate to what extent their child experiences inattention symptoms and to what extent different neuropsychological factors explain this inattention. The present study extends prior validation research of the PASSC by examining associations between PASSC inattention explained by time, cognition, and/or motivation and children’s performance on tests measuring these same three constructs. Results indicated positive correlations between PASSC inattention explained by time and less accurate performance on a time discrimination test, and between PASSC inattention explained by cognition and more working memory errors as well as higher attention switching costs. Furthermore, children whose parents indicated that their inattention was best explained by cognition showed higher switching costs than children whose inattention was best explained by motivation. This support for construct validity of the PASSC is limited to two PASSC explanations (i.e., time, cognition) and a subset of tests (i.e., time discrimination, attention switching, memory span). Future research should focus on integrating PASSC and performance test results to differentiate between children with attention problems but different underlying neuropsychological problems. Concluding, the PASSC can be a promising screening tool to identify inattention in children and the underlying explanation indicated by parents.

KEYWORDS:

• Construct validity
• inattention
• parent report
• neuropsychological assessment
• neuropsychological heterogeneity

Attention-Deficit/Hyperactivity Disorder (ADHD), characterized by inattention and/or hyperactivity-impulsivity symptoms, is a serious neurodevelopmental disorder inducing lower quality of life and impairments in academic and social functioning (e.g., American Psychiatric Association [APA], 2013; Faraone et al., 2015; Sonuga‐Barke et al., 2022). It affects approximately 5% of children and another 5–7% of children experience attention and/or behavioral problems with accompanying limitations in daily functioning, but do not fulfill sufficient DSM−5 ADHD criteria (APA, 2013; Sayal et al., 2018).

ADHD is increasingly recognized as a psychiatric syndrome that encloses heterogeneity at various levels (e.g., Nigg et al., 2020; Sonuga‐Barke et al., 2022). At the neuropsychological level, research has shown overlap between ADHD and control samples as well as differences within ADHD samples (e.g., Coghill et al., 2014; Fair et al., 2012; van Hulst et al., 2015). So-called multiple pathway models have explained this neuropsychological heterogeneity by identifying multiple, distinct neuropsychological profiles that underlie ADHD behavioral symptoms, in which individuals are to varying degrees affected by problems in, for instance, timing, working memory, inhibition, and/or delay aversion (Coghill et al., 2014; Nigg, 2013; Sonuga-Barke et al., 2010). In line with this, the Triple Pathway Model (TPM; see Sonuga-Barke et al., 2010 for a detailed description of the model) distinguishes three dissociable neuropsychological pathways to ADHD associated with distinct neural brain networks (Lecei et al., 2019). Empirical studies validated the TPM in children, showing that ADHD samples performed worse than control samples on temporal processing, inhibitory control and delay-aversion tasks, but that within ADHD samples subgroups of individuals are affected in only one of these three domains (De Zeeuw et al., 2012; Sonuga-Barke et al., 2010). In sum, the central idea of the TPM is that some children have a problem with a given function (such as time processing), while others have a different dysfunction (such as motivation) that causes ADHD symptoms, e.g., inattention, at the behavioral level (e.g., Nigg et al., 2020; Sonuga-Barke et al., 2010).

This neuropsychological heterogeneity implicates the need for tailored treatment that targets individual neuropsychological problems underlying ADHD symptoms (Lambek et al., 2018), since the DSM-classification ADHD on its own is not predictive of treatment response in individuals (Insel et al., 2010). Being able to differentiate between various neuropsychological profiles underlying ADHD symptoms is of crucial importance for personalized interventions (e.g., focus on improving cognitive functions versus immediately rewarding positive behavior), thereby increasing intervention effectivity.

Recently, we have developed an ADHD screening questionnaire, called “Parent ADHD Screening Questionnaire: Signaling the Core explanation underlying the behavioral symptoms (PASSC)”, of which the background, aim and psychometric properties are described elsewhere (Keulers & Hurks, 2021). This screening questionnaire aids in measuring and distinguishing between the above mentioned different neuropsychological problems underlying inattention symptoms. The PASSC assesses parents’ view on (possible) attention problems in their child and the extent to which these inattention symptoms are explained by problems in time processing, cognition and/or motivation (i.e., Triple Pathway Model). In a prior, large general population study, the PASSC has already been proven to have high internal consistency and sufficient construct validity (Keulers & Hurks, 2021). In that psychometric study (Keulers & Hurks, 2021), the convergent and discriminant validity was established by comparing PASSC scores with other parent report questionnaires only, a design that can be subject to biases and subjectivity. Here, we aim to extend the PASSC validation by comparing PASSC explanation sumscores, i.e., inattention symptoms explained by time, cognition, and/or motivation problems indicated by parents, to the children’s score on neuropsychological tests measuring these same three constructs. We expected the three PASSC inattention explanation sumscores to correlate only positively with performance on neuropsychological tests measuring the same construct.

Methods

Since the PASSC was developed as a screening instrument for the general population (Keulers & Hurks, 2021), children and their parent(s) were recruited from regular primary schools. Seven schools in the south of the Netherlands agreed to participate and distributed an information letter to parents of children in grades 3–8, asking them to sign an informed consent and fill in the PASSC and demographic questions via an online link (Qualtrics software), which in total took 10–15 minutes. Children performed a computerized neuropsychological test battery comprising performance tests measuring time processing, diverse cognitive functions and motivation (see Table 1 for detailed descriptions). The battery was administered 1-on-1 by trained psychology students, in a quiet room outside the classroom. Tests were programmed in Presentation (version 18.3; Neurobehavioral Systems, Inc.) or E-prime (version 2.0; Psychology Software Tools). All tests were preceded by instruction screens and practice trials. All children performed the tests in the same order and the total battery took approximately 60 minutes. Parents and children did not receive a reward for participation. The present study was approved by the Ethical Review Committee Psychology and Neuroscience (ERCPN), Maastricht University, The Netherlands (approval number ERCPN−161_02_04_2011).

Table 1. Descriptions of neuropsychological tests and dependent variables.

Test and measured construct	Description	Dependent variable(s)	Psychometric properties
T- Time reproduction task: reproduce various time intervals (adapted from Hurks & Hendriksen, 2010).	A jungle was presented for 500 ms, after which a fixation point (i.e., bananas) appeared in the middle of the screen. After 1000 ms, an ape cartoon picture appeared for a variable time interval. After the ape disappeared, the fixation point reappeared for 1000 ms. Then the ape appeared for the second time and children were instructed to press the spacebar for the same period as the ape had appeared during the previous time. As soon as the spacebar was released, the ape disappeared and the procedure repeated itself. Four different time intervals (i.e., 3, 6, 9, 12s) were presented twice in randomized order.	Reproduction accuracy coefficient across time intervals was calculated by (a) dividing the reproduced duration by the presented duration and (b) subtracting this ratio score from 1 to calculate the absolute difference from the optimal ratio.	Validity is supported by relevant group differences: children with ADHD were less accurate than controls, and in control samples accuracy increased between childhood and adolescence (Block et al., 1999; Hurks & Hendriksen, 2010). Internal consistency and test – retest reliability were fair in typically developing children and children with ADHD (Marx et al., 2021).
T- Time discrimination task: discriminate between 2 time intervals (adapted from Rubia et al., 2007).	Cartoon pictures of a lion and a pig were presented in succession. The lion was always presented first for a fixed duration of 500 ms. After this, the pig was presented for 500 ± 200 ms (i.e., the comparison). Then children had to indicate which picture appeared the longest, by pressing either “1” (lion) or “2” (pig). The test is adaptive in a way that the duration difference between the 2 pictures presented decreased with 5 ms when children answered 2 consecutive trials correctly and increased with 5 ms when children answered 1 trial incorrectly. Children completed 48 trials.	Minimal time discrimination difference between the 2 pictures in ms.	Validity is supported by relevant group differences: children with ADHD have less sensitive duration discrimination thresholds than controls (Noreika et al., 2013). Test – retest reliability was poor in typically developing children and children with ADHD (Marx et al., 2021).
C- Attention Switching Task (AST): switch attention between different task instructions (Cepeda et al., 2000; Keulers & Jonkman, 2019).	AST consisted of 2 non-switch (each 48 trials) and 1 switch (96 trials) blocks. In each trial 1 of 4 stimuli appeared on the screen with equal probability, i.e., 1, 3, 111 or 333. In the first non-switch block, a cue “WHICH NUMBER?” indicated that children had to identify whether the presented stimulus contained the number 1 (press left) or 3 (press right), irrespective of the number of presented digits. In the second non-switch block, a cue “HOW MANY NUMBERS?” indicated that children had to identify how many digits the number was made up of, 1 digit (press left) or 3 digits (press right). During the switch block the cues alternated randomly between “WHICH NUMBER?” and “HOW MANY NUMBERS?,” requiring the children to switch between rules on approximately half of the trials. Each trial consisted of the presentation of the cue followed 200 ms later by 1 of the 4 stimuli, which remained on the screen until a response was given.	Switching costs: difference in percentage correct responses between switch trials and non-switch trials (mean across the 2 non-switch blocks).	Validity is supported by relevant group differences: switching costs decreased between childhood and adulthood and were higher in ADHD than non-ADHD children (Cepeda et al., 2000, 2001). Research on reliability is not available.
C- Number Running Span: update working memory (Keulers & Jonkman, 2019).	Series of numbers (from 1–9) were presented 1 at a time on the screen for 1s, followed by an inter-stimulus-interval of 500 ms. Children were instructed to remember the sequence. After presentation of each sequence, children were presented with a screen showing the beginning N numbers of the sequence followed by question marks at the places where the missing numbers had to be filled in, for example 3 7 2? ? ?. Always the last 3 numbers of a sequence had to be recalled. Four trials of 5, 6 and 7 number sequence lengths (12 in total) were presented randomly.	Mean percentage errors across all trials.	Validity is supported by relevant group differences (on a pictorial variant of this task): mean percentage of error decreased between ages 6–15 in typically developing children (Lee et al., 2013). Research on reliability is not available.
M- Delay aversion task: choose between small immediate and larger delayed rewards (adapted from Kuntsi, Stevenson, et al., 2001).	Children are captain of a spaceship and have to destroy the spacecrafts of the enemy in this task. They can choose between (a) destroying 1 hostile spacecraft after a 2s delay and receiving 1 point (e.g., small immediate reward) and (b) destroying 2 hostile spacecrafts after 30s delay and receiving 2 points (larger delayed reward). The goal of the task is to obtain as many points as possible. Children get to choose 20 times in total.	Percentage immediate reward choices.	Validity is supported by relevant group differences: percentage immediate reward choices was higher in hyperactive compared to control children (Kuntsi, Oosterlaan, et al., 2001). Test-retest reliability is satisfactory in typically developing children aged 7–11 (Kuntsi, Stevenson, et al., 2001).

T = Time; C = Cognition; M = Motivation.

In total, n = 72 children (35 boys) and their parents participated. The children’s age ranged from 5.88 to 12.91 years (M = 9.17, SD = 1.79). The majority of children had the Dutch nationality (90.3%; 5.6% Belgian nationality; 4.2% other). Since we wanted to include a sample representative of the general population, children with a DSM−5 diagnosis as reported by parents, such as ADHD (9.7%), another developmental disorder (6.9%), and/or children using medication (e.g., methylphenidate; 5.6%), were not excluded. The level of parental education (LPE) scale included in our study ranged from primary school (1) to university degree (8) (De Bie, 1987), on a scaling comparable to the International Standard Classification of Education (UNESCO, 2012). If LPE differed between mother and father, the highest level of education was chosen. In total, 22.2% of parent-pairs had a low/moderate level of education (1–4) and 77.8% of parent-pairs had a high level of education (5–8).

The PASSC includes nine symptom items that equal the ADHD DSM−5 symptom descriptions of attention problems (APA, 2013). For those nine items parents were asked to indicate (a) to what extent their child shows the behavioral symptom and (b) to what extent each of the three given neuropsychological factors (i.e., time, cognition, and/or motivation problems) explains this symptom, both on a five points scale ranging from 0 (never) to 4 (very often). Additionally, parents were forced to choose the core explanation out of the three factors which best and/or most often explains the child’s behavioral symptom, or choose “none” in case their child does not show the symptom (see Keulers & Hurks, 2021 for an example item). The sumscore of the nine inattention symptoms and four inattention explanation sumscores (three sumscores indicating to what extent parents explained inattention symptoms by time, cognition, and/or motivation problems and one sumscore that indicates what parents answered when forced to choose the core explanation per symptom) were used as outcomes. All sumscores were shown to have a high internal consistency (i.e., reliability coefficients vary between 0.88–0.91; Keulers & Hurks, 2021). Validation of the PASSC was provided by a principal component analysis that supported the three TPM explanations (i.e., time, cognition, and/or motivation problems) given by parents, by positive associations with other questionnaires measuring the same constructs and by relevant group differences with e.g., children with ADHD scoring higher on the PASSC than children without a diagnosis (Keulers & Hurks, 2021). There were missing data from n = 4 children on the delay aversion task, n = 2 on the running span task and n = 2 for age; these were all replaced by the mean scores of the remaining children in the same grade (Tabachnick & Fidell, 2013). Data inspection indicated that the distributions of all variables were normal (i.e., skewness and kurtosis values between −1.5 and 1.5; Tabachnick & Fidell, 2013). Outliers were defined as a score of more than three SD above or below the mean and subsequently excluded from the relevant analyses (ranging between n = 0–2 dependent on the variable, see Table 2). All dependent variables were transformed such that a higher score indicated worse performance. Additionally, all neuropsychological test variables were adjusted for age such that the validation analyses of the PASSC were not contaminated by age effects, as the age range in the current sample was large (i.e., 5.88–12.91 years) and age and test performance were positively associated (i.e., three out of five test variables showed a significant negative correlation with age). The PASSC scores were not adjusted for age, as none of the sumscores correlated significantly with age and inattention symptoms are thought to remain stable across ages (Keulers & Hurks, 2021). Age adjusted scores on performance tests were calculated by using linear regression for each dependent test variable, with age as independent factor. The standardized residuals were saved and used as test performance measures in the analyses described below. Analyses were conducted using the statistical package IBM SPSS Statistics (version 27.0; SPSS, Inc., Chicago, IL), α was set at .05 one-tailed, and Bonferroni correction for multiple testing was applied when two neuropsychological tests were used to measure one construct (i.e., α=.025 for tests measuring time and cognition).

Table 2. Raw scores on and correlations^{^} between PASSC inattention sumscores and performance tests measuring the same/another construct (i.e., convergent/discriminant validity).

	Mean	SD (range)	PASSC inattention symptoms	PASSC inattention symptoms explained by Time	PASSC inattention symptoms explained by Cognition	PASSC inattention symptoms explained by Motivation
PASSC sumscores Inattention symptoms	n = 71 12.15	7.96 (0–35)
Inattention symptoms explained by Time	5.62	5.20 (0–20)
Inattention symptoms explained by Cognition	11.18	9.10 (0–36)
Inattention symptoms explained by Motivation	9.10	7.05 (0–30)
Core explanation^# 1 core explanation Time Cognition Motivation None 2 core explanations 3 core explanations	87.3% 1.4% 45.1% 32.4% 8.5% 2.8% 5.6%
T- Reproduction Reproduction accuracy coefficient	n=70 0.169	0.109 (0.027–0.460)	−.029	−.118	−.101	−.093
T- Discrimination Minimal time discrimination difference	n=72 168.68	20.60 (120–195)	.073	.244**	.087	.002
C- AST Switching cost accuracy	n = 72 16.32	11.04 (−2.92–43.70)	.166	.267**	.197*	−.123
C- Running Span Percentage errors	n = 72 0.456	0.304 (0–1)	.177	.337***	.225*	−.012
M- Delay Aversion Percentage immediate reward choices	n = 72 64.26	26.37 (0–100)	.129	.142	.083	.015

Note. ^{^}Correlation coefficients are based on age-regressed neuropsychological test scores and raw PASSC sumscores; PASSC= Parental ADHD Screening Questionnaire: Signaling the Core explanation underlying the behavioral symptoms; ^#in the case of one core explanation parents chose the same neuropsychological explanation (i.e., time, cognition or motivation) for the majority of inattention symptoms (e.g., minimal fvie out of nine symptoms) and otherwise parents indicated two or three core explanations equally for the attention problems of their child; T = Time; C = Cognition; AST = Attention Switching Task; M = Motivation. Grey blocks indicate convergent validity, and white blocks in the last three columns discriminant validity. Correlation coefficients up to 0.30 were considered low, from 0.31 to 0.50 were considered moderate and those exceeding 0.50 were considered strong (Cohen et al., 2013). ***p < .010; **p < .025; p < .050*.

Results and discussion

To examine convergent and discriminant validity of the PASSC, bivariate correlation analyses were performed to assess the association between PASSC inattention explanation sumscores on the one hand and neuropsychological tests measuring time processing, cognition, and motivation on the other hand (Table 2). In general, these associations were low and not significant. However, there were indications for convergent validity of two out of the three explanation sumscores, i.e., PASSC inattention explained by time correlated positively with children’s minimal time discrimination difference (r₍₇₁₎ = 0.244; p = .020) and PASSC inattention explained by cognition correlated positively with both working memory errors (r₍₇₁₎ = 0.225; p = .029) and attention switching costs (r₍₇₁₎ = 0.197; p = .050), although significance of the latter two (i.e., cognition) did not hold up after correction for multiple testing. Further evidence for convergent validity of these two explanation sumscores is neither found for the other time and cognition tests, nor is there a significant association between PASSC inattention explained by motivation and performance on the delay aversion task. The overall absence of associations between PASSC explanation sumscores and neuropsychological tests measuring other constructs suggests proof for discriminant validity, although inattention symptoms explained by time did correlate significantly with attention switching costs (r₍₇₁₎ = 0.267; p = .012) and working memory errors (r₍₇₁₎ = 0.337; p = .002). These latter are measures of the cognition construct, but some research suggested that time processing and working memory are not completely independent processes (Baudouin et al., 2006). Thus, there is some evidence of both convergent and/or discriminant validity of PASSC inattention explained by time, cognition, and motivation, although limited.

A second analysis was performed on parents’ responses to the forced choice question asking for the best explaining neuropsychological factor for inattention symptoms in their child. The core explanation parents chose across the nine attention symptoms was determined per child. In the majority of children (87.3%), parents chose one and the same explanation (either time, cognition, or motivation) for the majority of inattention symptoms (e.g., minimal five out of nine symptoms; one core explanation) and the remaining parents indicated two or three core explanations equally for the attention problems of their child (see Table 2). To have substantial subgroup sizes, we only compared children whose parents indicated that inattention was best explained by either cognition (n = 32) or motivation (n = 23). These two subgroups did not differ in age (t₍₅₃₎ = −0.54; p = .59), gender (X₍₅₃₎² = 0.09; p = .77), LPE (X₍₅₃₎² = 0.32; p = .57) or ADHD diagnosis (X₍₅₃₎² = 1.75; p = .19). Independent t-tests only yielded significant differences for attention switching costs (t₍₅₃₎ = 2.92; p = .005; the core cognition explanation subgroup showed higher switching costs than the core motivation explanation subgroup). Summarizing, differences between subgroups of children in whom inattention was differently explained by parents were in the expected direction, although this evidence is limited to cognition and motivational subgroups as well as to only one neuropsychological performance test.

The current findings provide additional evidence for construct validity of the PASSC beyond parental questionnaires (Keulers & Hurks, 2021), but the evidence is limited to a number of neuropsychological tests and the PASSC explanation sumscores time and cognition. This is in line with prior reports of low and non-significant associations between scores on questionnaires and neuropsychological performance tests (e.g., on self-regulation), which is attributed to differences in operationalization (trait versus state measures) and lower between-subject variability and reliability of behavioral test measures compared to questionnaires (Eisenberg et al., 2019; Enkavi et al., 2019; Friedman & Banich, 2019; Toplak et al., 2013). Future research should focus on how to best integrate both measures to optimally differentiate between subgroups with inattention symptoms but different underlying neuropsychological problems.

Limitations of the study include the relatively small sample size and the potential restricted variation in scores in the present non-clinical, general population sample. Nevertheless, the current PASSC scores were comparable to those reported in the prior large population study (n = 1166; Keulers & Hurks, 2021). Additionally, a post hoc power analysis (G*Power 3.1.9.7; Faul et al., 2007) indicated that with a sample size of 71, an α of 0.05 and statistical power of 0.80, correlations could detect small-medium effect sizes (|ρ| = .29) and independent t-tests (comparing n = 32 vs n = 23) could detect medium-large effect sizes (Cohen’s d of .69). When examining convergent validity, we have found significant correlations between PASSC sumscores and time discrimination, attention switching and running span tests. Unfortunately, even though we opted for frequently used tests in developmental research, extensive research on establishing psychometric properties of these tests is only marginally available and reliability information is thus lacking. Future studies should further study psychometric properties of these and potentially new neuropsychological tests. The current findings should be replicated using tests with known, good psychometric properties measuring the same constructs.

Concluding, the current findings provide further evidence for the construct validity of a parent report assessing neuropsychological problems in time processing, cognition and/or motivation underlying children’s inattention symptoms. Although the current and our prior study (Keulers & Hurks, 2021) indicate that the PASSC can be a promising tool to assess different underlying neuropsychological factors of children’s attention problems in primary schools, it should be seen as a first step in an extensive assessment procedure.

Consent to participate

Informed consent was obtained from parent(s) of all children included in the study.

Ethics approval

Approval was obtained from the ethics committee of Maastricht University, Faculty of Psychology and Neurosciences (approval number ERCPN−161_02_04_2011). The procedures used in this study adhere to the tenets of the Declaration of Helsinki.

Acknowledgments

The authors want to thank all participating schools, teachers and parents for their cooperation, and Robin Barenbrug, Caro Paffen, Jeanine Rongen, Denise Stefens, and Anke van Treek for their invaluable contribution to recruitment and testing.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the non-commercial Abbas Foundation [in Dutch: ‘Stichting Abbas Fonds’], which aims to stimulate the psychometric quality of psychological instruments in the Netherlands.