9,832
Views
0
CrossRef citations to date
0
Altmetric
Articles

The Quantified Behavioural Test Plus (QbTest+) in adult ADHD

, , , , &

Abstract

The Quantified Behavioural Test Plus (QbTest+) is widely used in clinical practice to assess patients with attention-deficit hyperactivity disorder (ADHD). This study mapped its behaviour in a group of adults with ADHD. Does it signal problems with impulsivity, attention and/or activity? To what extent are patients’ self-reported problems reflected in QbTest performance? Does Qb testing foretell the future, as reflected in the patients’ and clinicians’ judgements 4 years later? We here recorded the three QbTest+ cardinals–QbActivity, QbImpulsivity and QbInattention – in 67 consecutive ADHD patients diagnosed in adulthood. Among the 54 patients who medicated as usual on the day of testing, 35 (65%) scored above the clinical cut-off (Q-score ≥ 1.25) on at least one of the QbTest+ cardinals. Out of the 13 patients who suspended medication prior to the test, 11 (85%) scored above the clinical cut-off on at least one of the Qb-variables. There were modest associations between QbTest+ cardinals and symptom self-ratings [Brown ADD scale (BADDS); Adult Self-Report Scale (ASRS)]. Forty-one patients completed a second QbTest+ approximately 4 years after the first. Performance was improved on the follow-up test and fewer patients scored in the clinical range (34%). The scores on the QbInattention cardinal at baseline correlated positively with BADDS and ASRS self-ratings at the 4-year follow-up.

Introduction

Attention-deficit hyperactivity disorder (ADHD) is a neuropsychiatric disorder with roots in childhood affecting approximately 3% of the adult population (Fayyad et al., Citation2007, 2017). Although we note that the validity of adult ADHD diagnosis disputed by some (e.g. Moncrieff & Timimi, Citation2010), it is commonly diagnosed in adult psychiatric care: the prevalence of adult ADHD in nonpsychotic adult psychiatric care has been estimated to 17% in a study of eight European countries, Sweden included (Deberdt et al., Citation2015). This makes ADHD one of the largest patient groups in Swedish mental health care. Adult ADHD is associated with poorer functioning in everyday life (Asherson et al., Citation2016), with substantial negative economical and personal consequences (e.g. Chang et al., Citation2014; Chung et al., Citation2019).

The FDA cleared and CE marked Quantified Behavioural Test Plus (QbTest+) is widely used in clinical practice to assess patients with (suspected) ADHD. Research has suggested that this continuous performance test, tapping the three core symptoms of ADHD (inattention, impulsivity and hyperactivity), can help diagnosing ADHD in children and youth (Vogt & Shameli, Citation2011; Hall et al., Citation2016,2017; Hollis et al., 2018), aid in the search for neurobiological mechanisms (Oades et al., Citation2010), optimize treatment (Bijlenga et al., Citation2015; Tallberg et al., Citation2019) and facilitate communication with patients (Hall et al., Citation2017). However, problems have been reported by other studies with regard to its ability to tell ADHD apart from other neurodevelopmental and psychiatric conditions (Hult et al., Citation2018; Johansson et al., Citation2021) or even from healthy controls (Brunkhorst-Kanaan et al., Citation2020). It must be said, though, that the QbTest + is not meant to be a standalone test but be used together with other assessment tools (clinical interviews and rating scales) since many mental disorders are associated with cognitive problems involving attention, impulsivity and/or activity (Millan et al., 2012).

This report is a naturalistic study which aimed to map QbTest+ performance in a group of adult patients already diagnosed with ADHD (the diagnostic process itself did not involve the QbTest+). Its naturalistic design meant that we, as is usual in ordinary clinical settings, relied on the test manufacturer’s cut-off proposals to judge whether a particular patient’s performance deviated from normal or not (see Methods). Based on previous studies, we expected a robust impairment of QbTest+ performance (see Edebol et al., Citation2013b; Sharma & Singh, Citation2009). ADHD-medication has been shown to affect QbTest performance (Bijlenga et al., Citation2015; Ginsberg et al., Citation2012; Edebol et al., Citation2013a). Since some ADHD patients frequently miss taking their prescribed medication, this might influence test results. We therefore investigated if those patients who missed their medication performed worse than those who took their prescribed medication. Finally, we wished to explore whether QbTest+ performance at baseline predicted future attentional performance, subjective symptom ratings and/or clinicians’ reports at a follow-up 4 years later.

Materials and methods

Participants

The study sample derived from the St. Göran Bipolar Project carried out within the Northern Stockholm Mental Health Service, which assesses patients with ADHD (Nylander et al., Citation2021) and bipolar disorder (Sparding et al., Citation2015) over several years. The St. Göran Bipolar Project is an interdisciplinary, prospective and naturalistic study. It aims to study the course and outcome of treatment in standard psychiatric health care, and to investigate the pathophysiology of bipolar disorder and ADHD. The project’s ultimate aim is to reduce time spent ill and suffering, and to improve everyday functioning and quality of life.

Eighty-two patients diagnosed with ADHD were enrolled from a tertiary outpatient clinic specialized in assessment and treatment of ADHD. Experienced board-certified psychiatrists (E.R. or O.F.) conducted structured anamnestic interviews with all ADHD patients. The interview structure relies upon the clinical assessment instrument Affective Disorder Evaluation, originally from a study of bipolar disorder (Sachs et al., Citation2003). The Affective Disorder Evaluation includes social anamnesis, medical history and family history, and was in this study complemented with a section covering the DSM-IV diagnostic criteria for ADHD. In addition, the M.I.N.I. International Neuropsychiatric Interview (Sheehan et al., Citation1998) was used to screen for psychiatric diagnoses other than ADHD and bipolar disorder. The Adult ADHD Self-report Scale [ASRS; Kessler, Adler, Barkley et al. (2005)] and the Brown Attention Deficit Disorder-Scales [BADDS; Rucklidge and Tannock (Citation2002)] were used to assess current ADHD symptoms. Clinicians used the Global Assessment of Functioning (GAF) and Clinical Global Impression-Severity (CGI-S) to rate the patients’ functioning level and symptom severity, respectively. All available sources of information, encompassing patient interview, case records and, if available, interviews with next of kin were utilized in to arrive at a best estimate diagnosis.

The investigations also included a magnetic resonance imaging brain scan at baseline and follow-up. Cerebrospinal fluid was also collected through lumbar puncture to shed light on the pathophysiology of bipolar disorder and ADHD. None of these measures are, however, used in this study. All patients also underwent rigorous testing of cognitive functioning.

The sample consisted of 67 patients (see ) who had ongoing ADHD drug treatment and had completed a QbTest + at baseline (41 of them also completed a QbTest+ 4 years later).

Table 1 Descriptives of the 67 patients presented in N, percentages and, where applicable, in medians and interquartile range (IQR).

Out of the 67 patients, 13 (19%) missed by mistake taking their ADHD medication on the day of testing (see the flow-chart in ). The remaining 54 patients took their medication as prescribed. Forty-one patients completed a second QbTest + approximately 4 years later. At this time point, 12 patients missed their medicine prior to the test, whereas the rest (n = 29) medicated as usual. Hence, between baseline and the 4-year follow-up, 26 dropouts were registered. When comparing these 26 dropouts to the remaining 41 patients who participated in the follow-up, no statistical significant differences were found in their baseline QbTest scores or in their scale scores (BADDS, ASRS, GAF and CGI-S). We do not know the reason why the 26 patients chose to leave the study. According to ethical rules in our country, participants are free to drop out from research studies without being asked to declare the reason why.

Figure 1 Flowchart showing how the patients with ADHD, while on or off medication, performed the QbTest + at baseline (T1) and at a follow-up 4 years later (T2). “Scores above clinical cut-off” refers to the number of patients in a particular group who had Q-scores ≥ 1.25 on at least one of the cardinals. “Subclinical scores” refers to the number of patients in a particular group who had Q-scores < 1.25 on all of the cardinals.

Figure 1 Flowchart showing how the patients with ADHD, while on or off medication, performed the QbTest + at baseline (T1) and at a follow-up 4 years later (T2). “Scores above clinical cut-off” refers to the number of patients in a particular group who had Q-scores ≥ 1.25 on at least one of the cardinals. “Subclinical scores” refers to the number of patients in a particular group who had Q-scores < 1.25 on all of the cardinals.

The S.t Göran project is a naturalistic study and does not affect the management of care. Thus, medication type, doses, discontinuation, stop–start patterns and visits over the course of the 4-year study period were determined based solely on clinical needs and not defined in a study protocol. However, since the mainstay treatment for adult ADHD is central stimulants (Cortese et al., Citation2018), most patients used catecholaminergic reuptake blockers (e.g. methylphenidate and dexamphetamine) at least periodically, in 60% of the cases along with other psychopharmacological treatments to mitigate comorbid disorders. Antidepressants were the commonest [22 of 67 (33%)]. Sixteen (24%) were prescribed sleeping pills, and 6 (9%) patients were also prescribed anxiolytics. Most patients were prescribed more than one add-on drug to the stimulant at both baseline and follow-up. At follow-up, anxiolytics were the commonest [21 of 44 (48%)]. Eighteen (41%) were prescribed antidepressants at follow-up.

Instruments

The commercially available QbTest+, described in detail by others (Bijlenga et al., Citation2015; Johansson et al., Citation2021; Knagenhjelm & Ulberstad, Citation2010), is a 20-min computerized continuous performance test measuring activity level, impulsivity and inattention. Instructions on how to take the test were given verbally and by means of a standardized film (QbTech AB) presenting the procedure of the test. The patient then performed a 1-min practice to make sure the instructions were understood. The test took place in a room with minimal visual and auditory stimuli, the test instructor being present in the room but discretely, so as to not distracting the test-taker.

During testing, the patient sits in front of a computer screen and is presented with 600 stimuli (25% of which are randomly distributed targets), one-by-one at a pace of two presentations per second. There are four different kinds of stimuli, differentiated by colour and form. The test-taker is instructed to press a hand-held clicker when two consecutive stimuli are identical by colour and form. Other stimulus sequences are non-targets (no clicker-press). The test generates a score for each of three ADHD core symptoms. The first cardinal, QbActivity, is recorded by a motion-tracking system and provides a weighted index of movements during the test, in terms of total distance, vividness, and change of position. The second, QbImpulsivity, is a weighted index of various commission errors (responses to non-targets). QbInattention, finally, is a weighted composite of reaction time averages and variability plus omission errors (non-responses to targets). The scores are expressed as z-scores (i.e. standard deviations), which are called Q-scores by the test developers. A Q-score ≥ 1.25 on at least one of the parameters indicates that the participant may have ADHD (Knagenhjelm & Ulberstad, Citation2010). Qb Tests are not meant to be a standalone tool for diagnosing ADHD. However, it is designed to be added to the assessment process along with a rating scales and a clinical interview.

Questionnaires

The Brown ADD scale (BADDS) is a 40-item self-report scale assessing executive functioning. Individual items are rated on a scale from 0 to 3: “never”; “once a week or less”; “twice a week or more” or “almost daily.” The patient does not further specify for how long the rated symptom have been present. The items are clustered into five subscales. BADDS is primarily designed to measure the inattentiveness of the ADHD symptomatology. The total score can range from 0 to 120 and the clinical cutoff score of 50 indicates “probable ADHD” (Brown et al., Citation2011).

The Adult Self-Report Scale (ASRS) has 18 items, which correspond to the 18 DSM criteria of ADHD. Six of the 18 questions were found to be most predictive of ADHD; those questions form the part A (the screener version of ASRS) of the symptom checklist. Part B contains the remaining 12 symptoms of the ADHD diagnosis. The responses are given in a five-point Likert-scale from “never” to “always” (1–5). The patient is instructed to recall the preceding six months. The ASRS has shown good reliability and validity for evaluation of ADHD in adults (Adler et al., Citation2006). The six-item screener of ASRS has shown to outperform the full 18-item checklist in sensitivity, specificity and total classification accuracy (Kessler, Adler, Ames, et al., 2005). In this study, the six-item screener version is used.

GAF ranges from 100 (extremely well-functioning) to 1 (extremely severe impairment). GAF is used to rate overall psychological functioning plus social and occupational functioning, e.g. how well the patient is handling their everyday problems, during the preceding month. The GAF was included as axis V in DSM-IV and has been widely used in routine clinical settings around the world (Monrad-Aas, Citation2010; Piersma & Boes, Citation1997; Söderberg et al., Citation2005).

CGI-S is a 3-item scale measuring symptom severity, global improvement and therapeutic response. In this study, the symptom severity item was included, which summarizes the clinician’s global impression of symptom severity. The CGI-S is rated on a 7-point scale, from 1 to 7 (“not ill at” all “to extremely ill”) and refers to the actual time point for the examination. Psychometric evaluations have reported good internal consistency and concurrent validity (Leon et al., Citation1993) although some problems with validity or test-retest reliability also have been noted (Beneke & Rasmus, Citation1992).

Statistical analyses

The results in the tables are presented as medians and interquartile ranges. Non-parametric statistical tests were chosen because of their robustness to violations of normality and because their use in psychiatric studies is encouraged (see Urbano Blackford, Citation2017). The median test was used to test between-groups differences. It assesses whether two independent groups have been drawn from populations with the same median (Siegel, Citation1956). Within-groups differences were assessed using the Wilcoxon matched-pairs signed-ranks test. Associations between variables were computed as Spearman correlations. SPSS Statistics for Mac version 22.0 (SPSS Inc., Chicago, IL) was used for these calculations. Box plots were made using software developed by Spitzer et al. (Citation2014).

The QbTest+ developer suggests that a Q-score of ≥ 1.25 on at least one of the three parameters should be used as a cut-off in clinical practice (Knagenhjelm & Ulberstad, Citation2010; see also Hult et al., Citation2018), which equals performance in the ≥ 89th percentile (see Ulberstad, Citation2012c for information about normative data). This suggested cut-off was adopted in this study. Patients’ Q-scores, which are automatically provided after the test is completed, are adjusted to the performance of a healthy population separated according to gender and age (age groups in the normative group: 18–19, 20–29 and 30–60 years of age).

Ethics

The Regional Ethics Committee in Stockholm approved this study (2005/554-31/3 and 2011/1700-32), which was conducted in accordance with the latest Helsinki Protocol. All patients and controls consented both orally and in writing to participation in this study.

Results

QbTest+ performance with and without ADHD medication

illustrates the considerable individual variation in QbTest+ performance. Among the 54 patients who medicated as usual on the day of testing at baseline, 35 (65%) scored above the clinical cut-off (Q-score ≥ 1.25) on at least one of the QbTest+ cardinals. Out of the 13 patients who by mistake omitted their ADHD medication on the day of testing, 11 (85%) scored above the clinical cut-off on at least one of the Qb-variables. The difference between the medicated and the temporarily non-medicated patients was not significant by the χ2-test [χ2(1) = 1.91, p = .167]. Neither was the difference in the median Q-scores between patients on and off medication () significant by the Median test: QbInattention [χ2 (1) = 1.23, p = .27; QbActivity: χ2 (1) = 0.06, p = .80; QbImpulsivity: χ2 (1) = 2.58, p = .11].

Figure 2 Box plots representing QbTest + performance in 67 patients diagnosed with ADHD. Some (N = 13) suspended taking their ADHD medicine on the day of testing (white boxes) and the others (N = 54) had medicated as usual (light grey boxes). Centre lines show the medians; box limits indicate the 25th and 75th percentiles; whiskers extend 1.5 times the interquartile range from the 25th and 75th percentiles. The notches represent the 95% confidence interval for each median. Non-overlapping notches give roughly 95% confidence that two medians differ.

Figure 2 Box plots representing QbTest + performance in 67 patients diagnosed with ADHD. Some (N = 13) suspended taking their ADHD medicine on the day of testing (white boxes) and the others (N = 54) had medicated as usual (light grey boxes). Centre lines show the medians; box limits indicate the 25th and 75th percentiles; whiskers extend 1.5 times the interquartile range from the 25th and 75th percentiles. The notches represent the 95% confidence interval for each median. Non-overlapping notches give roughly 95% confidence that two medians differ.

Table 2 Performance on the QbTest + and scores on self-reports (BADDS and ASRS) and clinical measures (GAF and CGI-S), presented as medians and interquartile ranges (IQR) in adult patients with a diagnosis of ADHD.

Associations between QbTest+ performance and rating scale scores

shows the Spearman correlation coefficients between QbTest+ performance on the one hand and rating scales scores in patients medicating at baseline on the other. QbActivity correlated positively with concurrent ASRS ratings and QbInattention with concurrent BADDS ratings. The correlations were in the moderate range. There were also significant correlations between baseline Q-scores (particularly ObInattention) and self-rated BADDS- and ASRS-scores 4 years later ().

Table 3 Spearman correlation coefficients in non-medicated patients with ADHD (N = 13) between QbTest+ cardinals and self-reports (BADDS and ASRS) and clinical measures (GAF and CGI-S) at baseline (N = 13 (12 for CGI-S); upper panel) or at the 4-year follow-up (N = 7–8; lower panel).

Rating scale and Q-scores over the course of 4 years

shows that patients tested twice on the QbTest+ (N = 41) performed significantly better on the follow-up test compared with the baseline test, 4 years earlier (QbActivity: Z = −3.07, p = .002 and η2 = 0.23; QbImpulsivity: Z = −2.50, p = .012 and η2 = 0.15; QbInattention: Z = −3.97, p = .000 and η2 = 0.39). At this time, 34% of the patients (14 out 41) scored in the clinical QbTest+ range. The three cardinals were significantly correlated with oneself between the first and second tests (QbActivity: r = 0.66 and p = .000; QbImpulsivity: r = 0.49 and p = .001; QbInattention: r = 0.69, p = .000).

Conforming to the general pattern reported in a previous study of ours (Nylander et al., Citation2021), the ADHD patients also improved over the course of 4 years () according to the BADDS (Z = −3.44, p = .001 and η2 = 0.29), ASRS (Z = −2.20, p =0.028 and η2 = 0.11) and the CGI-S (Z = −3.86, p = .000 and η2 = 0.37). GAF estimates, by contrast, did not change significantly (Z = −0.91 and p = .362).

Discussion

A striking feature in our data is the considerable individual variation in QbTest+ performance (). Many patients had Q-scores within the normal range on at least one of the cardinals and the medians were below the clinical range. This variability notwithstanding, 65% of the patients tested while on medication scored above the clinical cut-off proposed by the manufacturer (i.e. a Q-score > 1.25 on at least one of the cardinals). A group of patients omitted by mistake to take their stimulant medication on the day of testing. In this unintended “experimental” group, 85% scored above the QbTest+ clinical threshold. This difference failed to reach significance in the statistical sense. Likewise, the median Q-scores of patients tested on or off medication were similar statistically. However, since many studies report that the QbTest+ can detect beneficial effects of central stimulant treatment in adults (Bijlenga et al., Citation2015; Edebol et al., Citation2013a; Ginsberg et al., Citation2012) it is quite possible that our failure to unveil significant effects of no-medication was simply due to a lack of statistical power as there were only 13 patients in our drug-free condition.

At baseline, we found significant but modest correlations between Q-scores on the one hand and scores on the BADDS- and ASRS self-reports on the other. Similar results have been reported by Bijlenga et al. (2015) and see also Lis et al. (Citation2010) in a larger group of patients. More surprisingly, we found that Q-scores at baseline were correlated with symptom self-ratings 4 years later. For instance, the correlations between baseline QbInattention and follow-up BADDS- and ASRS scores were in the +0.5 range, and the correlation between baseline QbActivity and follow-up ASRS scores were only slightly lower. Needless to say, this finding needs to be affirmed by independent studies before being fully accepted. If indeed confirmed, QbTest+ performance might constitute a useful tool (along with other sources) to predict the likely long-term prognosis of the disorder.

Even though we report quite high correlations between the objective test and the subjective self-reports, the associations explain part of the variance only (10–25% in this case). This disparity may be due to several factors. The two test modes may tap qualitatively different constructs (see Bijlenga et al., Citation2015) or reflect metacognitive difficulties in rating one’s own shortcomings (Alexander & Liljequist, Citation2016; Butzbach et al., Citation2021; Du Rietz et al., Citation2016; Eckerström et al., Citation2016; Fuermaier et al., Citation2015). A patient’s responses on a symptom self-report might mirror in part accumulated long-term outcomes (e.g. academic, self-esteem and social functioning) rather than the actual symptoms (see Shaw et al., Citation2012). A final possibility is that self-reports versus objective tests capture different, albeit related, cognitive endophenotypic levels and the assessment goal determining the appropriate measuring method (see Barkley & Fischer, Citation2011).

At follow-up after 4 years, both the QbTest+ and the self-reports indicated that the patients had improved. This observation is in line with the common observation that ADHD symptoms can abate or change in character with time (Kooij et al., Citation2019), possibly as a consequence of stimulant treatment (Edebol et al., Citation2013b; Ginsberg et al., Citation2012). Many studies show that ADHD-medication alleviates core ADHD symptoms in the short term (Kooij et al., Citation2019). However, its long-term efficiency remains more uncertain (e.g. Cortese et al., Citation2018).

Strengths of this study include the careful diagnostic procedure that all patients passed through by experienced psychiatrists at a specialized ADHD unit, as well as the inclusion of various and complementary measures. Another strength is the length of the follow-up: follow-ups over several years are rare. The naturalistic study design also yields a high ecological validity.

The naturalistic study design is also a limitation, with no control over medication type, discontinuations, doses and visits over the course of 4 years. Another limitation is the relatively small sample size. Several patients also had other psychopharmacological treatments to mitigate comorbid disorders, such as depression. The role of these comorbid affective disorders (and/or their medications) in causing QbTest+ performance deficits is insufficiently researched and this situation hampers any firm conclusions. However, according to Mesquita et al. (Citation2016), depressive symptom scores do not correlate with performance on a continuous performance test related to the QbTest+ in adults with ADHD. Nevertheless, an interesting avenue for future research would be to explore the potential contribution of, for instance, depression in causing impairments in QbTest+ performance in patients with ADHD. For example Elliot et al. (1996) reported that clinically depressed people are overly sensitive to perceived failure during neuropsychological testing: having solved one task incorrectly, depressed patients were more prone than controls to fail the subsequent problem. Hypersensitivity to negative feedback may impair performance in a continuous performance task such as the QbTest+, and should perhaps profitably be distinguished from more rash testing approaches that would result in an equally poor result, but for partly different reasons. Unfortunately, our present data lack the resolution needed for doing these kinds of fine-grained analyses.

Conclusions

There was large individual variability in QbTest+ performance among adults with ADHD, even though a fair share scored above the clinical cutoff while under medication. At the 4-year follow-up, fewer patients scored above the QbTest+ cutoff and their symptom self-ratings had improved. The QbInattention score at baseline correlated significantly with follow-up self-ratings.

Acknowledgements

We are deeply grateful for the participation of all patients contributing to this research, and we are indebted to the collection team at the Northern Stockholm Psychiatry Clinic that worked to recruit them. We especially wish to thank study nurse Lena Lundberg and data manager Mathias Kardell.

Disclosure statement

M.L. declares that he has received lecture honoraria from Lundbeck pharmaceutical. The other authors have no competing interests to declare.

Data availability statement

The datasets generated and/or analysed for this study are not publicly available due to the Swedish law for register data. However, data will be available from the corresponding author on reasonable request.

Additional information

Funding

This research was supported by grants from the Swedish Research Council [2018-02653] and the Swedish Federal Government under the LUA/ALF agreement [ALF 20170019].

References

  • Adler, L. A., Spencer, T., Faraone, S. V., Kessler, R. C., Howes, M. J., Biederman, J., & Secnik, K. (2006). Validity of pilot adult ADHD self-report scale (ASRS) to rate adult ADHD symptoms. Annals of Clinical Psychiatry, 18(3), 145–148. https://doi.org/10.1080/10401230600801077
  • Alexander, L., & Liljequist, L. (2016). Determining the accuracy of self-report versus informant-report using the Conners’ adult ADHD rating scale. Journal of Attention Disorders, 20(4), 346–352. https://doi.org/10.1177/1087054713478652
  • Asherson, P., Buitelaar, J., Faraone, S. V., & Rohde, L. A. (2016). Adult attention-deficit hyperactivity disorder: Key conceptual issues. The Lancet Psychiatry, 3(6), 568–578. https://doi.org/10.1016/S2215-0366(16)30032-3
  • Barkley, R. A., & Fischer, M. (2011). Predicting impairment in major life activities and occupational functioning in hyperactive children as adults: Self-reported executive function (EF) deficits versus EF tests. Developmental Neuropsychology, 36(2), 137–161. https://doi.org/10.1080/87565641.2010.549877
  • Beneke, M., & Rasmus, W. (1992). Clinical impressions” (ECDEU): Some critical comments. Pharmacopsychiatry, 25(04), 171–176. https://doi.org/10.1055/s-2007-1014401
  • Bijlenga, D., Jasperse, M., Gehlhaar, S. K., & Kooij, J. J. S. (2015). Objective QbTest + and subjective evaluation of stimulant treatment in adult attention deficit-hyperactivity disorder. European Psychiatry: The Journal of the Association of European Psychiatrists, 30(1), 179–185. https://doi.org/10.1016/j.eurpsy.2014.06.003.
  • Brown, T. E., Brams, M., Gasior, M., Adeyi, B., Babcock, T., Dirks, B., Scheckner, B., & Wigal, T. (2011). Clinical utility of ADHD symptom thresholds to assess normalization of executive function with lisdexamfetamine dimesylate treatment in adults. Current Medical Research and Opinion, 27(2), 23–33. https://doi.org/10.1185/03007995.2011.605441
  • Brunkhorst-Kanaan, N., Verdenhalven, M., Kittel-Schneider, S., Vainieri, I., Reif, A., & Grimm, O. (2020). The quantified behavioral test-A confirmatory test in the diagnostic process of adult ADHD? Frontiers in Psychiatry, 11(216), 216. https://doi.org/10.3389/fpsyt.2020.00216
  • Butzbach, M., Fuermaier, A., Aschenbrenner, S., Weisbrod, M., Tucha, L., & Tucha, O. (2021). Metacognition in adult ADHD: Subjective and objective perspectives on self-awareness of cognitive functioning. Journal of Neural Transmission, 128(7), 939–955. https://doi.org/10.1007/s00702-020-02293-w
  • Chang, Z., Lichtenstein, P., D'Onofrio, B. M., Sjölander, A., & Larsson, H. (2014). Serious transport accidents in adults with attention-deficit/hyperactivity disorder and the effect of medication: A population-based study. JAMA Psychiatry, 71(3), 319–325. https://doi.org/10.1001/jamapsychiatry.2013.4174
  • Chung, W., Jiang, S. F., Paksarian, D., Nikolaidis, A., Castellanos, F. X., Merikangas, K. R., & Milham, M. P. (2019). Trends in the prevalence and incidence of Attention-Deficit/Hyperactivity Disorder among adults and children of different racial and ethnic groups. JAMA Network Open, 2(11), e1914344. https://doi.org/10.1001/jamanetworkopen.2019.14344
  • Cortese, S., Adamo, N., Del Giovane, C., Mohr-Jensen, C., Hayes, A. J., Carucci, S., Atkinson, L. Z., Tessari, L., Banaschewski, T., Coghill, D., Hollis, C., Simonoff, E., Zuddas, A., Barbui, C., Purgato, M., Steinhausen, H.-C., Shokraneh, F., Xia, J., & Cipriani, A. (2018). Comparative efficacy and tolerability of medications for attention-deficit hyperactivity disorder in children, adolescents, and adults: A systematic review and network meta-analysis. The Lancet Psychiatry, 5(9), 727–738. https://doi.org/10.1016/S2215-0366(18)30269-4
  • Deberdt, W., Thome, J., Lebrec, J., Kraemer, S., Fregenal, I., Ramos-Quiroga, J. A., & Arif, M. (2015). Prevalence of ADHD in nonpsychotic adult psychiatric care (ADPSYC): A multinational cross-sectional study in Europe. BMC Psychiatry, 15, 242. https://doi.org/10.1186/s12888-015-0624-5
  • Du Rietz, E., Cheung, C. H., McLoughlin, G., Brandeis, D., Banaschewski, T., Asherson, P., & Kuntsi, J. (2016). Self-report of ADHD shows limited agreement with objective markers of persistence and remittance. Journal of Psychiatric Research, 82, 91–99. https://doi.org/10.1016/j.jpsychires.2016.07.020
  • Eckerström, M., Berg, A. I., Nordlund, A., Rolstad, S., Sacuiu, S., & Wallin, A. (2016). High prevalence of stress and low prevalence of Alzheimer Disease CSF biomarkers in a clinical sample with subjective cognitive impairment. Dementia and Geriatric Cognitive Disorders, 42(1–2), 93–105. https://doi.org/10.1159/000448326
  • Edebol, H., Helldin, L., & Norlander, T. (2012). Objective measures of behavior manifestations in adult ADHD and differentiation from participants with bipolar II disorder, borderline personality disorder, participants with disconfirmed ADHD as well as normative participants. Clinical Practice and Epidemiology in Mental Health, 8, 134–143. https://doi.org/10.2174/1745017901208010134
  • Edebol, H., Helldin, L., & Norlander, T. (2013a). The weighed core symptomscale and prediction of ADHD in adults - objective measures of remission and response to treatment with methylphenidate. Clinical Practice & Epidemiology in Mental Health, 9(1), 171–179. https://doi.org/10.2174/1745017901309010171
  • Edebol, H., Helldin, L., & Norlander, T. (2013b). Measuring adult attention deficit hyperactivity disorder using the quantified behavior test plus. Psychological Journal, 2(1), 48–62. https://doi.org/10.1002/pchj.17
  • Elliott, R., Sahakian, B. J., McKay, A. P., Herrod, J. J., Robbins, T. W., & Paykel, E. S. (1996). Neuropsychological impairments in unipolar depression: The influence of perceived failure on subsequent performance. Psychological Medicine, 26(5), 975–989. https://doi.org/10.1017/s0033291700035303
  • Fayyad, J., De Graaf, R., Kessler, R., Alonso, J., Angermeyer, M., Demyttenaere, K., De Girolamo, G., Haro, J. M., Karam, E. G., Lara, C., Lépine, J.-P., Ormel, J., Posada-Villa, J., Zaslavsky, A. M., & Jin, R. (2007). Cross-national prevalence and correlates of adult attention-deficit hyperactivity disorder. The British Journal of Psychiatry: The Journal of Mental Science, 190(5), 402–409. https://doi.org/10.1192/bjp.bp.106.034389
  • Fayyad, J., Sampson, N. A., Hwang, I., Adamowski, T., Aguilar-Gaxiola, S., Al-Hamzawi, A., Andrade, L. H. S. G., Borges, G., de Girolamo, G., Florescu, S., Gureje, O., Haro, J. M., Hu, C., Karam, E. G., Lee, S., Navarro-Mateu, F., O’Neill, S., Pennell, B.-E., Piazza, M., … Kessler, R. C. (2017). The descriptive epidemiology of DSM-IV adult ADHD in the World Health Organization world mental health surveys. Attention Deficit and Hyperactivity Disorders, 9(1), 47–65. https://doi.org/10.1007/s12402-016-0208-3
  • Fuermaier, A., Tucha, L., Koerts, J., Aschenbrenner, S., Kaunzinger, I., Hauser, J., Weisbrod, M., Lange, K. W., & Tucha, O. (2015). Cognitive impairment in adult ADHD-perspective matters!!. Neuropsychology, 29(1), 45–58. https://doi.org/10.1037/neu0000108
  • Ginsberg, Y., Hirvikoski, T., Grann, M., & Lindefors, N. (2012). Long-term functional outcome in adult prison inmates with ADHD receiving OROS-methylphenidate. European Archives of Psychiatry and Clinical Neuroscience, 262(8), 705–724. https://doi.org/10.1007/s00406- 012-0317-8
  • Hall, C. L., Selby, K., Guo, B., Valentine, A. Z., Walker, G. M., & Hollis, C. (2016). Innovations in practice: An objective measure of attention, impulsivity and activity reduces time to confirm attention deficit/hyperactivity disorder diagnosis in children – a completed audit cycle. Child and Adolescent Mental Health, 21(3), 175–178. https://doi.org/10.1111/camh.12140
  • Hall, C. L., Valentine, A. Z., Walker, G. M., Ball, H. M., Cogger, H., Daley, D., Groom, M. J., Sayal, K., & Hollis, C. (2017). Study of user experience of an objective test (QbTest) to aid ADHD assessment and medication management: A multi-methods approach. BMC Psychiatry, 17(1), 66. https://doi.org/10.1186/s12888-017-1222-5
  • Hollis, C., Hall, C. L., Guo, B., James, M., Boadu, J., Groom, M. J., Brown, N., Kaylor‐Hughes, C., Moldavsky, M., Valentine, A. Z., Walker, G. M., Daley, D., Sayal, K., & Morriss, R, on behalf of the AQUA Trial Group. (2018). The impact of a computerised test of attention and activity (QbTest) on diagnostic decision-making in children and young people with suspected attention deficit hyperactivity disorder: Single-blind randomised controlled trial. Journal of Child Psychology and Psychiatry, 59(12), 1298–1308. https://doi.org/10.1111/jcpp.12921
  • Hult, N., Kadesjö, J., Kadesjö, B., Gillberg, C., & Billstedt, E. (2018). ADHD and the QbTest+: diagnostic validity of QbTest+. Journal of Attention Disorders, 22(11), 1074–1080. https://doi.org/10.1177/1087054715595697
  • Johansson, V., Norén Selinus, E., Kuja-Halkola, R., Lundström, S., Durbeej, N., Anckarsäter, H., Lichtenstein, P., & Hellner, C. (2021). The quantified behavioral test failed to differentiate ADHD in adolescents with neurodevelopmental problems. Journal of Attention Disorders, 25(3), 312–321. https://doi.org/10.1177/1087054718787034
  • Kessler, R. C., Adler, L. A., Barkley, R., Biederman, J., Conners, C. K., Faraone, S. V., Greenhill, L. L., Jaeger, S., Secnik, K., Spencer, T., Ustün, T. B., & Zaslavsky, A. M. (2005a). Patterns and predictors of attention-deficit/hyperactivity disorder persistence into adulthood: Results from the national comorbidity survey replication. Biological Psychiatry, 57(11), 1442–1451. https://doi.org/10.1016/j.biopsych.2005.04.001
  • Kessler, R. C., Adler, L., Ames, M., Demler, O., Faraone, S., Hiripi, E., Howes, M. J., Jin, R., Secnik, K., Spencer, T., Ustun, T. B., & Walters, E. E. (2005b). The world health organization adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35(2), 245–256. https://doi.org/10.1017/s0033291704002892
  • Knagenhjelm, P., & Ulberstad, F. (2010). QbTest + technical manual. QbTech AB.
  • Kooij, J. J. S., Bijlenga, D., Salerno, L., Jaeschke, R., Bitter, I., Balázs, J., Thome, J., Dom, G., Kasper, S., Nunes Filipe, C., Stes, S., Mohr, P., Leppämäki, S., Casas, M., Bobes, J., Mccarthy, J. M., Richarte, V., Kjems Philipsen, A., Pehlivanidis, A., … Asherson, P. (2019). Updated European consensus statement on diagnosis and treatment of adult ADHD. European Psychiatry: The Journal of the Association of European Psychiatrists, 56, 14–34. https://doi.org/10.1016/j.eurpsy.2018.11.001
  • Leon, A. C., Shear, M. K., Klerman, G. L., Portera, L., Rosenbaum, J. F., & Goldenberg, I. A. (1993). Comparison of symptom determinants of patient and clinician global ratings in patients with panic disorder and depression. Journal of Clinical Psychopharmacology, 13(5), 327–331. https://doi.org/10.1097/00004714-199310000-00005
  • Lis, S., Baer, N., Stein-En-Nosse, C., Gallhofer, B., Sammer, G., & Kirsch, P. (2010). Objective measurement of motor activity during cognitive performance in adults with attention-deficit/hyperactivity disorder. Acta Psychiatrica Scandinavica, 122(4), 285–210. https://doi.org/10.1111/j.1600-0447.2010.01549.x
  • Mesquita, C., Nazar, B. P., Pinna, C. M., Rabelo, B., Serra-Pinheiro, M. A., Sergeant, J., & Mattos, P. (2016). How can continuous performance test help to assess inattention when mood and ADHD symptoms coexist? Psychiatry Research, 243, 326–330. https://doi.org/10.1016/j.psychres.2016.06.054
  • Millan, M. J., Agid, Y., Brüne, M., Bullmore, E. T., Carter, C. S., Clayton, N. S., Connor, R., Davis, S., Deakin, B., DeRubeis, R. J., Dubois, B., Geyer, M. A., Goodwin, G. M., Gorwood, P., Jay, T. M., Joëls, M., Mansuy, I. M., Meyer-Lindenberg, A., Murphy, D., … Young, L. J. (2012). Cognitive dysfunction in psychiatric disorders: Characteristics, causes and the quest for improved therapy. Nature Reviews Drug Discovery, 11(2), 141–168. https://doi.org/10.1038/nrd3628
  • Moncrieff, J., & Timimi, S. (2010). Is ADHD a valid diagnosis in adults? No. BMJ, 340, c547. https://doi.org/10.1136/bmj.c547
  • Monrad-Aas, I. H. (2010). Global assessment of functioning (GAF): Properties and frontier of current knowledge. Annals of General Psychiatry, 9, 20. https://doi.org/10.1186/1744-859X-9-20
  • Nylander, E., Floros, O., Sparding, T., Rydén, E., Hansen, S., & Landén, M. (2021). Five-year outcomes of ADHD diagnosed in adulthood. Scandinavian Journal of Psychology, 62(1), 13–24. https://doi.org/10.1111/sjop.12692
  • Oades, R. D., Myint, A. M., Dauvermann, M. R., Schimmelmann, B. G., & Schwarz, M. J. (2010). Attention-deficit hyperactivity disorder (ADHD) and glial integrity: An exploration of associations of cytokines and kynurenine metabolites with symptoms and attention. Behavioral and Brain Functions: BBF, 6, 32. https://doi.org/10.1186/1744-9081-6-32.
  • Piersma, H. L., & Boes, J. L. (1997). The GAF and psychiatric outcomes: A descriptive report. Community Mental Health Journal, 33(1), 35–41. https://doi.org/10.1023/A:1022413110345
  • Rucklidge, J. J., & Tannock, R. (2002). Validity of the Brown ADD scales: An investigation in a predominantly inattentive ADHD adolescent sample with and without reading disabilities. Journal of Attention Disorders, 5(3), 155–164. https://doi.org/10.1177/108705470200500303
  • Sachs, G. S., Thase, M. E., Otto, M. W., Bauer, M., Miklowitz, D., Wisniewski, S. R., Lavori, P., Lebowitz, B., Rudorfer, M., Frank, E., Nierenberg, A. A., Fava, M., Bowden, C., Ketter, T., Marangell, L., Calabrese, J., Kupfer, D., & Rosenbaum, J. F. (2003). Rationale, design, and methods of the systematic treatment enhancement program for bipolar disorder (STEP-BD). Biological Psychiatry, 53(11), 1028–1042. https://doi.org/10.1016/S0006-3223(03)00165-3
  • Sharma, A., & Singh, B. (2009). Evaluation of the role of Qb testing in attention deficit hyperactivity disorder. [Abstract. Archives of Disease in Childhood, 94, A72.
  • Shaw, M., Hodgkins, P., Caci, H., Young, S., Kahle, J., Woods, A. G., & Arnold, L. E. (2012). A systematic review and analysis of long-term outcomes in attention deficit hyperactivity disorder: Effects of treatment and non-treatment. BMC Medicine, 10, 99. https://doi.org/10.1186/1741-7015-10-99
  • Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., Dunbar, G. C. (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59(20), 22–33.
  • Siegel, S. (1956). Nonparametric statistics for the behavioral sciences. McGraw-Hill.
  • Sparding, T., Silander, K., Pålsson, E., Östlind, J., Ekman, C. J., Sellgren, C., Joas, E., Hansen, S., & Landén, M. (2015). Cognitive functioning in clinically stable patients with bipolar disorder I and II. PLoS One, 10(1), e115562. https://doi.org/10.1371/journal.pone.0115562
  • Spitzer, M., Wildenhain, J., Rappsilber, J., & Tyers, M. (2014). BoxPlotR: A web tool for generation of box plots. Nature Methods, 11(2), 121–122. https://doi.org/10.1038/nmeth.2811
  • Söderberg, P., Tungström, S., & Armelius, B. Å. (2005). Reliability of global assessment of functioning ratings made by clinical psychiatric staff. Psychiatric Services (Washington, D.C.), 56(4), 434–438. https://doi.org/10.1176/appi.ps.56.4.434
  • Tallberg, P., Råstam, M., Wenhov, L., Eliasson, G., & Gustafsson, P. (2019). Incremental clinical utility of continuous performance tests in childhood ADHD - an evidence-based assessment approach. Scandinavian Journal of Psychology, 60(1), 26–35. https://doi.org/10.1111/sjop.12499
  • Ulberstad, F. (2012a). Discriminant validity for the Quantified Behavioral Test (QbTest+) in children. Data on file.
  • Ulberstad, F. (2012b). Discriminant validity for the Quantified Behavioral Test (QbTest+) in adolescents and adults. Data on file.
  • Ulberstad, F. (2012c). Characterization of the normative database and methods to generate age and gender specific comparisons for the Quantified Behavioral Test (QbTest+). Data on file.
  • Urbano Blackford, J. (2017). Leveraging statistical methods to improve validity and reproducibility of research findings. JAMA Psychiatry, 74(2), 119–120. https://doi.org/10.1001/jamapsychiatry.2016.3730
  • Vogt, C., & Shameli, A. (2011). Assessments for attention-deficit hyperactivity disorder: Use of objective measurements. The Psychiatrist, 35(10), 380–383. https://doi.org/10.1192/pb.bp.110.032144