Is motor activity during cognitive assessment an indicator for feigned attention-deficit/hyperactivity disorder (ADHD) in adults?

ABSTRACT

Objectives: Several approaches, ranging from self-ratings of symptoms and impairments to objective neuropsychological testing, have been utilized during clinical evaluation in order to assess symptom and performance validity of individuals with attention-deficit/hyperactivity disorder (ADHD) in adulthood. Motor activity has not been considered yet in this context, which is surprising given that hyperactivity is a prominent characteristic of ADHD. Hence, the goal of the present study was to explore the incremental value of motor activity when assessing the credibility of individuals with adult ADHD at clinical evaluation.

Method: Forty-six patients diagnosed with ADHD took part in the study. A simulation design was performed, in which 152 healthy individuals were allocated to either a control condition (n = 36) or one of three simulation conditions (n = 116), the latter requesting participants to feign ADHD. All participants completed a self-rating scale of cognitive functioning and performed a computerized test for vigilance. Body movements were recorded during vigilance testing via a motion tracker attached to the back of the participant’s chair.

Results: Patients with ADHD reported significantly more pronounced cognitive complaints and performed significantly poorer on the vigilance test than control participants. Simulators of ADHD, as compared to genuine patients, showed excessively low performance on the vigilance test. However, neither self-ratings of cognitive functioning nor measures of motor activity were suitable to distinguish genuine from feigned ADHD. A hierarchical logistic regression model showed that motor activity had no incremental value in detecting feigned ADHD when vigilance test performance has already been considered.

Conclusions: Standard neuropsychological tests of vigilance may be useful to measure both performance and credibility of individuals with adult ADHD at clinical evaluation. In contrast, self-reports of symptoms and impairments, as well as measures of body movements, may not support the assessment of credibility in this context.

KEYWORDS:

• ADHD
• adult attention-deficit/hyperactivity disorder
• cognition
• faking
• feigning
• hyperactivity
• motor activity
• noncredible
• performance validity
• symptom validity
• neuropsychological assessment

The assessment of symptom and performance validity is an essential part of neuropsychological assessment, in order to ensure and maximize confidence in the results of such assessment for the purpose of clinical evaluation (Bush et al., 2005; Heilbronner et al., 2009). For example, noncredible symptom reporting and test performance have been observed when clinically evaluating attention-deficit/hyperactivity disorder (ADHD) in adulthood (Hirsch & Christiansen, 2015; Marshall, Hoelzle, Heyerdahl, & Nelson, 2016). In this context, various incentives have been identified that may motivate people to feign or grossly exaggerate symptoms of ADHD. Such incentives include, but are not limited to, access to stimulant medication (either as a cognitive enhancer or for recreational purposes) or advantages in the academic context such as being awarded extra time for assignments and exams (Lensing, Zeiner, Sandvik, & Opjordsmoen, 2013; Pella, Hill, Shelton, Elliott, & Gouvier, 2012; Rabiner, 2013). The relevance of this issue was emphasized by findings of noncredible reporting of both ADHD symptoms and cognitive performance in 15 to 48% of young adults, in particular college students, who presented for clinical evaluation of ADHD (Harrison & Edwards, 2010; Marshall et al., 2016; Sullivan, May, & Galbally, 2007). A large number of studies have used performance validity tests (Edmundson et al., 2017; Fuermaier, Tucha, Koerts, Grabski, et al., 2016; Harrison, Rosenblum, & Currie, 2010; Jasinski et al., 2011; Leppma, Long, Smith, & Lassiter, 2017; Morey, 2017; Sollman, Ranseen, & Berry, 2010; Suhr, Hammers, Dobbins-Buckland, Zimak, & Hughes, 2008) or validity indicators of personality assessment inventories (Aita, Sofko, Hill, Musso, & Boettcher, 2017; Musso, Hill, Barker, Pella, & Gouvier, 2016; Smith, Cox, Mowle, & Edens, 2017) in order to detect possible noncredible performance of adults being clinically evaluated for ADHD. The vast majority of these studies revealed mostly a moderate usefulness with high specificity but often low sensitivity. Because of the frequent noncredible reporting of ADHD symptoms, it has been strongly advised that the clinical assessment of adult ADHD should include a careful exploration of the extent to which credibility of clients may be an issue (Bryant et al., 2017; Fuermaier, Tucha, Koerts, Butzbach, et al., 2017; Marshall et al., 2016).

In the last decade, several other approaches have been examined that may support the detection of feigned adult ADHD (for reviews on this topic, see Musso & Gouvier, 2014; Tucha, Fuermaier, Koerts, Groen, & Thome, 2015). For example, numerous studies demonstrated that self-report instruments, commonly used to quantify symptoms and impairments associated with ADHD in routine clinical practice, may not have substantial clinical value in helping to distinguish honest from dishonest symptom reporting (Booksh, Pella, Singh, & Gouvier, 2010; Edmundson et al., 2017; Fuermaier, Tucha, Koersts, Weisbrod, et al., 2016; Jachimowicz & Geiselman, 2004; Quinn, 2003; Suhr et al., 2008; Tucha, Sontag, Walitza, & Lange, 2009). Furthermore, most of these studies concluded that healthy individuals need only brief instructions and little preparation time in order to feign ADHD—that is, meet the criteria for an ADHD diagnosis. It was therefore suggested that in this context the use of infrequency or exaggeration indices embedded in routine self-report scales are more promising for the detection of noncredible symptom reporting (Cook et al., 2017; Cook, Bolinger, & Suhr, 2016; Harrison & Armstrong, 2016; Suhr, Buelow, & Riddle, 2011; Walls, Wallace, Brothers, & Berry, 2017).

Objective neuropsychological tests used for routine cognitive assessment have been shown to be more beneficial for the identification of noncredible performance in the clinical evaluation of adult ADHD than the commonly used self-report scales. Promising candidates are variants of the Continuous Performance Test (CPT), a test that is frequently used in neuropsychological practice for the assessment of vigilance and sustained attention of adults with ADHD (Avisar & Shalev, 2011; Epstein, Conners, Sitarenios, & Erhardt, 1998; Huang-Pollock, Karalunas, Tam, & Moore, 2012; Marchetta, Hurks, De Sonneville, Krabbendam, & Jolles, 2008; Tucha, Tucha, et al., 2009). In the context of performance validity assessment, the CPT has been found helpful in distinguishing individuals with genuine ADHD from those simulating the disorder (Booksh et al., 2010; Leppma et al., 2017; Quinn, 2003), although it must be noted that the value of CPTs for the assessment of performance validity has not been supported by all studies (Sollman et al., 2010; Suhr, Sullivan, & Rodriguez, 2011). When considering these findings, it can be concluded that routine neuropsychological tests have the potential to support the assessment of credibility of individuals being evaluated for adult ADHD; however, when used as the only effort measure, such tests are likely not sensitive enough to identify genuine ADHD with sufficient accuracy (Marshall et al., 2016; Musso & Gouvier, 2014; Tucha et al., 2015).

While self-reports and neuropsychological test performances have been evaluated in a large number of studies with regard to their utility to detect feigned adult ADHD, motor activity has not been considered in this context yet. This appears surprising given that hyperactivity is part of the diagnostic criteria for ADHD as outlined in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM–5; American Psychiatric Association, 2013) and can be successfully measured in adults with ADHD by using actigraphs (small devices worn by participants to record motor activity level; Boonstra et al., 2007; Tuisku et al., 2003). There are reasons to assume that the measurement of motor activity would add predictive value to the detection of feigned adult ADHD, as it might be very difficult for individuals who simulate ADHD to mimic the level of body movements of genuine patients with ADHD. Furthermore, research showed that motor hyperactivity is a less salient characteristic of ADHD in adulthood than in childhood (Biederman, Mick, & Faraone, 2000; Brocki, Tillman, & Bohlin, 2010; Davidson, 2008), which may not be known by most individuals who are not experts in the field. This assumption is supported by findings of a previous simulation study using a self-report scale for ADHD symptoms (Fuermaier, Tucha, Koerts, Weisbrod, et al., 2016). In this study, instructed simulators exaggerated symptoms of hyperactivity/impulsivity more grossly than symptoms of inattention when compared to genuine patients with ADHD. Individuals attempting to feign ADHD in clinical practice thus may show excessive levels of motor activity, which are presumably not only higher than those of healthy participants, but also higher than the levels shown by genuine patients with ADHD. We therefore hypothesize that the measurement of motor activity during neuropsychological assessment complements objective neuropsychological test performance in detecting feigned adult ADHD.

The present study employed a simulation design in which healthy participants were randomly allocated to either a control group or one of three simulation groups in which participants were instructed to perform the assessment as if they were suffering from ADHD. Data of the simulation groups were compared to data of genuine patients with ADHD. The assessment included three types of measurements; self-ratings of cognitive functioning, neuropsychological performance in a vigilance test, and the measurement of motor activity during neuropsychological testing (accomplished via a motion tracker attached to the back of the participant’s chair). The goal of this study was to explore the usefulness of these approaches for the detection of noncredible symptom reporting and performance by adults being evaluated for ADHD. It is expected that (a) self-ratings of cognitive functioning are not useful for the detection of feigned adult ADHD as instructed simulators are expected to be able to produce scores in a believable range for genuine ADHD, that (b) vigilance test performance reveals significant differences between patients with ADHD and instructed simulators, with moderate predictive accuracy for feigned ADHD, and that (c) instructed simulators overestimate the level of hyperactivity shown by genuine patients with ADHD, resulting in excessively high motor activity that supports neuropsychological test performance in detecting feigned adult ADHD.

Method

Participants

With regard to the required sample size, it must be noted that power issues are usually not a major concern in this type of research, as relatively large effects are required for the detection of feigning. These large effects are revealed with relatively small samples. Rogers (2008), for example, introduced the classification of effects into moderate if Cohen’s d ≥ 0.75, and large if Cohen’s d ≥ 1.25. Based on a group comparison with a two-tailed test and α = .05, revealing a moderate effect (d = 0.75) with a desired power (1 – β) of .85 requires a sample size of 33 participants per group, while a large effect (d = 1.25) requires a sample size of only 13 participants per group. However, a reliable estimation of classification accuracy (i.e., as indicated by area under the curve statistics and which is determined by sensitivity and specificity) requires both a sufficient number of individuals instructed to feign ADHD (to determine sensitivity) and a sufficient number of patients with ADHD (to determine specificity). We therefore aimed to exceed the minimum number of participants as indicated in the power analysis, attempting to reach a group size of 50 patients with ADHD.

Patients with ADHD

Patients with ADHD were referred from local psychiatrists or neurologists to the Department of Psychiatry and Psychotherapy of the SRH Clinic Karlsbad, Germany. Participation in the study was offered to patients with ADHD on a voluntary basis. It was pointed out to patients with ADHD that data collected for this study were analyzed anonymously, were used for research purposes exclusively, and did not affect their clinical evaluation and/or treatment.

Diagnostic assessments were performed independently by experienced clinicians associated with the Department of Psychiatry and Psychotherapy, separate from each individual’s participation in the present study. Each assessment involved a clinical psychiatric interview according to Diagnostic and Statistical Manual of Mental Disorders–Fourth Edition (DSM–IV; American Psychiatric Association, 1994) criteria for ADHD, as devised by Barkley and Murphey (1998), including the retrospective assessment of symptoms in childhood as well as current symptoms. The diagnostic assessment also included the identification and examination of objective impairments supporting the diagnosis of ADHD (e.g., evidence derived from school reports, impairment in academic and/or occupational achievement). Further, the assessment comprised, if available, multiple informants, such as employer, partner or parent(s). Moreover, all participants completed two standardized self-report rating scales designed to quantify current and retrospective ADHD symptoms. Childhood ADHD symptoms were self-rated on the short version of the Wender Utah Rating Scale (WURS-K; Ward, Wender, & Reimherr, 1993), which includes 25 items rated on a 5-point scale. Severity of current ADHD symptoms was self-rated with the ADHD Self-Report Scale (ASRS; Adler et al., 2006; Kessler et al., 2005) consisting of 18 items rated on a 4-point scale corresponding to the diagnostic criteria of the DSM–IV. All diagnoses were made by mutual agreement between at least two clinicians who were part of a diagnostic team and experienced in the assessment and treatment of adults with ADHD.

In total, 50 adults diagnosed with ADHD agreed to take part in the present study. Performance validity of patients was assessed by applying a cutoff of an embedded validity measure—that is, the verbal working memory test N-back (Vienna Test System (VTS); Schuhfried, 2013). The N-back appears to be useful as an embedded validity indicator in studies on adults with ADHD as working memory deficits are prominent in ADHD (Alderson, Kasper, Hudec, & Patros, 2013) and as various working memory tests have been successfully applied in the assessment of credibility of adults with ADHD (Booksh et al., 2010; Fuermaier, Tucha, Koerts, Lange, et al., 2017; Harrison et al., 2010; Suhr et al., 2008). For the present purpose, the embedded validity cutoff of the N-back was derived from a separate sample of 95 patients with ADHD (age = 35 ± 11 years; 39% female) who were recruited in the same context as the patients of the present study. All of these 95 patients were assessed for performance validity with the Test of Memory Malingering (TOMM; Tombaugh, 1997) or the Groningen Effort Test (GET; Fuermaier, Tucha, Koerts, Aschenbrenner, & Tucha, 2017; Fuermaier, Tucha, Koerts, Grabski, et al., 2016). Of the 95 patients with ADHD, 12 patients failed, and 83 patients passed the performance validity testing (PVT). Of the latter 83 patients, the distribution of scores on the N-back was inspected (range = 1–15 correct responses), and a cutoff was derived that identified those individuals that scored equally or lower than one standard deviation below the mean (16th percentile). In this case, 16% of patients with ADHD had a score of 7 points (i.e., number of correct responses) or less. This cutoff (≤7 correct responses) resulted in a reasonable sensitivity of 58% in the sample of patients with ADHD failing the PVT. Applying the cutoff of 7 or fewer correct responses in the N-back task to the present sample resulted in the identification of four patients with ADHD (8%) who could be classified as noncredible. These four patients with ADHD were removed from all further analyses.

Of the remaining 46 patients with ADHD (Table 1), four patients met criteria for ADHD–I (predominantly inattentive type), 40 patients met DSM–IV criteria for ADHD–C (combined type), and two of the patients met criteria for ADHD–HI (hyperactive–impulsive type). Fourteen patients with ADHD exhibited one or more psychiatric comorbidities, including mood disorders (n = 10), personality disorders (n = 2), anxiety disorders (n = 2), obsessive compulsive disorders (n = 2), pathological gambling (n = 1), and substance abuse disorder (with no substance abuse in the previous six months; n = 1). Patients with ADHD suffering from comorbid psychiatric disorders were not excluded because comorbidity is very prevalent among patients with ADHD and is therefore representative for the clinical presentation of this condition (Biederman et al., 1993). Thirteen patients with ADHD were treated with antidepressant medication at the time of the study because of comorbid disorders. However, none of the patients were currently taking medication for the treatment of ADHD symptoms (i.e., stimulant drug treatment). Current treatment with medication for ADHD symptoms (i.e., stimulant medication) was an exclusion criterion as individuals of simulation groups were trying to simulate test performance of patients with ADHD that were not being treated with stimulant medication at the time of assessment. This is important considering that those attempting to feign ADHD at clinical evaluation do so in order to gain access to certain incentives, such as stimulant medication. Thus, the level of performance that individuals feigning ADHD try to mimic at clinical evaluation is the performance of (still untreated) genuine patients with ADHD.

Table 1. Characteristics of participants.

	CG (N = 36) (26f/10m) (M ± SD)	ADHD (N = 46) (21f/25m) (M ± SD)	NSG (N = 38) (29f/9m) (M ± SD)	SSG (N = 38) (28f/10m) (M ± SD)	SSG+M (N = 40) (28f/12m) (M ± SD)
Age (years)	20.0 ± 2.0	33.4 ± 11.7	20.8 ± 2.5	20.6 ± 2.3	19.8 ± 1.6
Years of education	13.7 ± 2.0	15.4 ± 4.2	13.0 ± 2.4	13.3 ± 1.7	13.3 ± 1.9
Childhood ADHD symptoms^a	13.1 ± 7.8	40.2 ± 12.3	13.1 ± 10.1	14.0 ± 9.2	11.6 ± 7.2
Current ADHD symptoms^b	13.0 ± 8.2	37.1 ± 9.3	10.3 ± 8.6	10.7 ± 9.4	8.8 ± 5.5

Note. ADHD = attention-deficit/hyperactivity disorder; f = female; m = male; CG = control group; NSG = naïve simulation group;

SSG = symptom-coached simulation group; SSG+M = symptom-coached simulation group + motor activity.

^aWender Utah Rating Scale–Short Version (WURS-K). ^bADHD Self Rating Scale (ASRS).

Healthy individuals

Furthermore, 152 first-year psychology students (111 female, 41 male) of the University of Groningen, a university with a large international study program in the Netherlands, took part in the study. Although participation of healthy individuals was voluntary and not paid, the students did receive course credits in exchange for participation: a requirement for their undergraduate study program. Explorative analyses showed that demographic background had no significant effect on any of the outcome measures of the present study (i.e., self-ratings of cognitive functioning, vigilance test performance, and motor activity). Participants had a mean age of 20.3 years (SD = 2.1 years) and a mean education of 13.3 years (SD = 2.0). None of the healthy individuals reported to have a history of neurological or psychiatric diseases, and none were taking any medication known to affect the central nervous system. Furthermore, none of the healthy participants endorsed clinically relevant scores of ADHD symptom severity as measured by two standardized self-report rating scales designed to quantify current and retrospective ADHD symptoms (Adler et al., 2006; Kessler et al., 2005; Ward et al., 1993).

Prior to the assessment, healthy participants were randomly assigned to one of four conditions; the control group (CG; n = 36), the naïve simulation group (NSG; n = 38), the symptom-coached simulation group (SSG; n = 38), and the symptom-coached simulation group + motor activity (SSG+M; n = 40). The groups differed with regard to instruction and information that they were provided with prior to the assessment. The information provided to the various groups is thoroughly described below in the Design and Procedure section. Descriptive variables (Table 1) did not differ significantly between groups of healthy individuals with regard to age, F(3) = 1.783, p = .153, education, F(3) = 0.922, p = .432, sex, χ²(3) = 0.415, p = .937, or ADHD symptom severity, including both childhood, F(3) = 0.513, p = .674, and current symptoms, F(3) = 1.729, p = .163. However, patients with ADHD differed significantly from healthy individuals in all descriptive variables, including age, F(1) = 174.420, p < .001, education, F(1) = 20.859, p < .001, sex, χ²(1) = 11.908, p = .001, and ADHD symptom severity, including both childhood, F(1) = 283.898, p < .001, and current symptoms, F(1) = 344.003, p < .001.

Materials

Self-rating of cognitive functioning

The Questionnaire for Complaints of Cognitive Disturbances (FLei; Beblo et al., 2010) was administered to assess subjectively experienced levels of cognitive functioning. The FLei is a self-report instrument that includes items of commonly reported cognitive complaints in everyday-life situations. The questionnaire is composed of three subscales measuring difficulties in attention (10 items; e.g., to stay focused for a continuous period of time), memory (10 items; e.g., memory for names), and executive functioning (10 items, e.g., to plan a party). The questionnaire additionally includes five control items that were not included in the analysis. Participants were asked to indicate on a scale from 0 (never) to 4 (very frequently) how often the described problems occur. The sum score on each of the three scales was calculated.

Assessment of vigilance

Vigilance was measured with the WAFV (Perception and Attention Functions–Vigilance) of the Vienna Test System (VTS; Schuhfried, 2013; Sturm, 2006). The VTS is a computerized test system for the measurement of various neuropsychological functions. The WAFV is a subtest of the VTS and is widely used in clinical practice to assess cognitive impairments of patients with psychiatric or neurological conditions. The WAFV shares many characteristics with a CPT and was included in a test battery that was specifically designed to detect cognitive impairments of adults with ADHD (Cognitive Functions ADHD, CFADHD; Tucha, Fuermaier, Aschenbrenner, & Tucha, 2013).

The vigilance test was administered in a quiet laboratory equipped with an office chair and desk. The VTS was placed on the office desk and was installed on a laptop computer, which was connected to a VTS specific response device. In the WAFV, visual stimuli (squares) are presented in the center of the computer screen in consecutive order at regular intervals (500 ms interstimulus interval). Each stimulus is presented for 1,500 ms. After 500 ms presentation time the stimulus may change its intensity (i.e., it gets darker). The test consists of 900 stimuli in total, of which 50 change their intensity. Participants were instructed to sit on the office chair, focus on the stimulus presentation, and press a specific button on the response device as quickly as possible whenever a stimulus turned darker. The test measures vigilance by requiring the participants to remain alert and ready to react to infrequently occurring target stimuli over a relatively long and continuous period of time (30 min administration time). The mean reaction time, the standard deviation of reaction times, the number of omission errors, and the number of commission errors were registered as measures of vigilance.

Assessment of motor activity

Motor activity was assessed during cognitive testing by an ActiTrac motion tracker (IM Systems, 2006), which is an ambulatory activity monitor shown to be useful for the measurement of body movements of patients with neurological or psychiatric conditions in clinical studies (Garcia Ruiz & Sanchez Bernardos, 2008; Kodaka et al., 2010). Usually, the ActiTrac is attached to the participants’ wrist; however, in the present study the motion tracker was attached to the back of the office chair on which participants were sitting during the performance of the vigilance test in order to measure movements of the entire body instead of arm movements only, and also to avoid the measurement of the confound arm movements that are required for the execution of neuropsychological tests. The motion tracker was attached in such a way that it was invisible to the participant being assessed, but that it was sensitive to register even subtle movements of participants. For example, the office chair was equipped with wheels standing on a smooth base, which allowed for movements of the chair by minimal effort of the participant. Further, the office chair was flexible on the front–back axis (back of the chair), as well as on the horizontal plain (circular movements of the seat).

ctric sensor measuring acceleration signals. The acceleration signals were sampled at a rate of 40 times per second, digitally integrated, and averaged over a 2-s time epoch (one data point). The motion tracker reports the quantity of acceleration signals in milliG (mG) and is calibrated according to the manufacturer’s settings to a maximum sensitivity level of 0.312 mG. G is the Earth’s gravitational acceleration, which equals 9.8 m/s² at the Earth’s surface. For the duration of the vigilance test (30 min), the total amount of movement was registered and was recalculated as activity per hour (activity in mG hour^–1). Furthermore, an active period was defined if at least 9 consecutive data points indicated activity. An inactive period was defined if at least 9 consecutive data points were below a software defined activity threshold. The numbers of active periods and inactive periods indicate change in activity status. The active-to-inactive ratio was calculated by dividing the total active time periods by the sum of the inactive and active time periods. The active-to-inactive ratio thus represents the percentage of time during vigilance testing in which motor activity was shown above threshold.

Design and procedure

Assessment of patients with ADHD

Patients with ADHD were assessed individually and received no reward for their participation. Written informed consent was obtained from all participants prior to the assessment. Patients with ADHD were requested to complete the self-rating of cognitive functioning (FLei) to the best of their knowledge and to perform the vigilance test (WAFV on the VTS) to the best of their abilities. Patients with ADHD were not informed that motor activity was recorded during cognitive testing. Patients were debriefed at the end of the assessment and informed about the motion tracker registering their motor activity. The total duration of the assessment of patients with ADHD was about 60 min. The study was conducted in compliance with the latest version of the Declaration of Helsinki (DoH) with regard to ethical standards of research involving human subjects. The study was approved by the local institutional ethical review board of the Medical Faculty of the University of Heidelberg, Germany.

Assessment of healthy participants

All healthy participants were tested individually in a quiet laboratory. At the beginning of the experiment, descriptive and anamnestic information was obtained including age, sex, educational level, and self-reported ADHD symptom severity. Furthermore, participants were asked for any history of psychiatric or neurological diseases as well as pharmacological treatment. Descriptive information was obtained from all participants before simulation groups were instructed to feign ADHD. The subsequent assessment procedure differed between participants of the various groups (CG, NSG, SSG, and SSG+M). Table 2 presents an overview of instructions given per group, including the type of information that participants received. The researcher performing the assessment was aware of the respective experimental condition and provided the information in written form to participants. Given that the test administration and scoring were automated (computerized neuropsychological test and assessment of body movements) and not under control of the researcher (self-report scale), it is highly unlikely that the researcher’s knowledge about the experimental condition has influenced results. Furthermore, data analysis was performed by researchers not involved in the assessments of participants. The assessment of healthy individuals was approved by the Ethical Committee Psychology (ECP) affiliated to the University of Groningen, the Netherlands. All participants gave written informed consent prior to participation.

Table 2. Type of information and instruction given per group.

Experimental condition	Scenario to feign ADHD (vignette)	Information about ADHD symptoms (symptom-coaching)	Information about assessment of motor behavior	Instruction
Patients with ADHD	No	No	No	Normal behavior
Control group (CG)	No	No	No	Normal behavior
Naïve simulation group (NSG)	Yes	No	No	Feign ADHD
Symptom-coached simulation group (SSG)	Yes	Yes	No	Feign ADHD
Symptom-coached simulation group + motor activity (SSG+M)	Yes	Yes	Yes	Feign ADHD

Control group (CG)

Participants of the CG were asked to complete the self-rating of cognitive functioning (Flei) and to perform the vigilance test (WAFV on the VTS) to the best of their knowledge and abilities. Participants received a notification prior to the assessment in which the clinical significance of the study was outlined, but which did not contain information with regard to the aim of the study (detection of feigned ADHD). Participants of the CG were also not informed about the registration of their body movements, but were informed about the motion tracker during the debriefing. The duration of the assessment of the CG was about 50 min.

Simulation groups (NSG, SSG, SSG+M)

Participants of the simulation groups were asked to perform the assessment (including the FLei and WAFV) while pretending to have ADHD (feigning ADHD). They were instructed to feign all aspects of ADHD in all of its facets in a realistic manner. For this purpose, participants of the simulation groups were presented with a vignette, describing a scenario of a person motivated to feign ADHD. Several benefits were introduced in the vignette that may come with a diagnosis of ADHD. Examples include financial benefits and accommodations, more flexibility and freedom in working hours and deadlines, and the prescription of stimulant medication (which may then instead be used for improvement of work/academic performance or for recreational use). Information provided in the vignette was restricted to support participants to assume their role but did not contain information about the symptoms or nature of ADHD. Moreover, participants were explicitly instructed to feign ADHD realistically by not exaggerating matters too much. In order to encourage participants to feign ADHD in a believable manner, participants were informed that the participant who feigns the condition best would be awarded with a top-of-the-range tablet PC. This incentive was implemented because of methodological considerations in malingering research suggesting that an external incentive is an important element when using simulation designs (Dunn, Shear, Howe, & Ris, 2003; Rogers, Harrell, & Liff, 1993). However, due to ethical reasons, the tablet PC was in fact assigned randomly to one of the participants across all conditions, independently of the participants’ scores and test performance.

After the vignette has been presented (to NSG, SSG, and SSG+M), the NSG did not receive any further information or suggestion of how to fake ADHD. The SSG and SSG+M received a description of the diagnostic criteria for adult ADHD as outlined in the DSM. This approach has been shown in previous studies to provide instructed malingerers with sufficient information to become familiar with the characteristics of ADHD (Harrison, Edwards, & Parker, 2007; Tucha, Sontag, et al., 2009). In addition, the SSG+M was explicitly informed about the measurement of motor activity via the motion tracker, and that these data would also be used to determine the diagnostic veracity of ADHD (see supplementary material for the scenario (vignette) as well as simulation instructions). Participants of the SSG and SSG+M were requested to respond to a number of questions on the content of the information with which they were provided in order to ascertain that they indeed read but also understood the information. All participants were able to answer these questions, so that no participant had to be excluded. Finally, participants were requested to start feigning ADHD and to perform the assessment (including the FLei and WAFV) as if they were suffering from ADHD. At the end of the assessment, participants were instructed to stop feigning ADHD and were debriefed. Participants of the NSG and SSG were informed about the registration of body movements during the debriefing. All participants of simulation groups indicated that they followed instructions sincerely, including instructions to feign ADHD; thus, no participant had to be excluded by the end of the simulation instructions. The assessment of participants of the simulation groups took about 80 min.

Statistical analysis

Self-ratings of cognitive functioning, vigilance test performance, and motor activity were statistically compared between groups. First, multivariate (per type of assessment—that is, self-rating, cognitive testing, and motor activity) and univariate (per assessment measure) analyses of variance (ANOVAs) were carried out to compare the control group to patients with ADHD. This analysis gives information about the level of cognitive functioning of patients with ADHD when compared to a healthy control group. Age and sex were considered as covariates in these calculations in order to increase comparability of both groups.

Second, the control group was compared in univariate analyses (ANOVAs with Tukey’s honestly significant difference post hoc tests) with each group of instructed simulators. This analysis provides information on the success of experimental manipulation as one would expect that healthy individuals instructed to feign ADHD change their normal behavior and score differently on the assessment compared to healthy individuals who were instructed to show normal behavior.

Third, patients with ADHD were compared with each group of instructed simulators, using univariate analyses (ANOVAs with Tukey’s honestly significant difference post hoc tests). This analysis is most important for the present context as it indicates whether the assessment measures applied were able to distinguish genuine from feigned ADHD or whether individuals instructed to feign ADHD can produce comparable scores to individuals with genuine ADHD. Age and sex were not considered as covariates in these analyses as there is a systematic difference in the instructions that these groups received (instruction to show normal behavior vs. instruction to feign ADHD), thus a direct comparison of cognitive functioning or motor behavior between these groups does not apply. It is not uncommon in simulation designs that individuals instructed to feign are not matched to patients with regard to descriptive variables, as performance abilities are not directly compared between these groups under normal conditions. Generally, effort tests have not been found to be impacted by variables such as age and education, with the exception of distorted test results that may occur at extremes of these variables (Green & Flaro, 2003; Heilbronner et al., 2009). Alpha-error inflation in multiple testing (comparison of 11 variables) was controlled for by a Bonferroni corrected significance level of .0045 in all univariate analyses.

Moreover, effect sizes (Cohen’s d) were calculated to indicate the magnitude of group differences between patients with ADHD and instructed simulators. Effect sizes were interpreted based on Cohen’s classification into negligible effects (d < 0.20), small effects (0.20 ≤ d < 0.50), medium effects (0.50 ≤ d < 0.80), and large effects (d ≥ 0.80; Cohen, 1988). Furthermore, effect sizes were also interpreted based on more rigorous standards as it was stressed by Rogers (2008) to be required for the assessment of malingering. Rogers (2008) introduced a categorization of effects sizes (Cohen’s d) and distinguished between moderate effects (0.75 ≤ d < 1.25), large effects (1.25 ≤ d < 1.50), and very large effects (d ≥ 1.50). For those comparisons that showed at least moderate effects (d ≥ 0.75), receiver operating characteristics (ROCs) were calculated. A ROC curve plots the sensitivity against “1 – specificity” at each variable cutoff to predict the criterion—that is, feigned ADHD relative to genuine ADHD. ROC analysis allows for determination of diagnostic accuracy as measured by the area under the curve (AUC), indicating the probability that a score drawn at random from the first sample (e.g., people feigning ADHD) is higher than a score drawn at random from the second sample (e.g., people with genuine ADHD; Rice & Harris, 2005).

Next, three binary logistic regression analyses were carried out in order to determine the validity of the three types of assessments (self-ratings of cognitive functioning, vigilance test performance, and motor activity) in predicting feigned ADHD (n = 76) relative to genuine ADHD (n = 46). For the purpose of this analysis, the simulation groups NSG and SSG were collapsed into one feigning group as it likely remains unknown in any given assessment context whether individuals had prepared themselves prior to the diagnostic evaluation. The SSG+M was not considered in this analysis because of differences in demand characteristics in this group (explicit instruction that motor activity is recorded), as compared to patients with ADHD, which otherwise may confound results of the logistic regression. A bootstrap resampling procedure was applied in order to examine the internal validity of the prediction models. In the bootstrap analyses, 3,000 random samples were drawn with replacement from the original dataset. Bootstrap 95% confidence intervals of regression coefficients were calculated in order to estimate the internal validity of the findings of the regression analyses as an approximation to external validity (i.e., generalizability; Steyerberg et al., 2001). Finally, in order to determine the incremental validity of the motor activity given the cognitive performance in detecting feigned ADHD, a hierarchical logistic regression model was performed in which cognitive test scores were entered first to the model (Step 1), followed by variables of motor activity (Step 2), and self-ratings of cognitive functioning (Step 3).

Results

Group comparisons

Table 3 presents self-ratings of cognitive performance, vigilance test performance, and motor activity per group. Significance levels (Bonferroni corrected) and effect sizes of group differences are indicated for each comparison of interest.

Table 3. Self-ratings of cognitive performance, cognitive test performance, and motor activity per group.

	CG (n = 36)	ADHD (n = 46)		NSG (n = 38)		SSG (n = 38)		SSG+M (n = 40)
	M ± SD	M ± SD		M ± SD	ES	M ± SD	ES	M ± SD	ES
Self-rating of cognitive functioning^a
Attention (score)	11.6 ± 4.8		30.1 ± 5.5*	27.9 ± 7.5*	0.34	29.2 ± 6.6*	0.15	29.3 ± 7.0*	0.13
Memory (score)	13.6 ± 4.9		28.6 ± 5.7*	25.1 ± 6.1*	0.60	26.1 ± 6.9*	0.40	25.5 ± 6.4*	0.51
Executive functions (score)	9.8 ± 5.8		25.0 ± 5.8*	21.9 ± 6.9*	0.49	23.8 ± 7.4*	0.18	24.3 ± 7.7*	0.10
Vigilance test performance^b
Omission errors	5.3 ± 8.7		9.6 ± 8.8*	22.5 ± 13.9*^†	1.13	25.2 ± 14.6*^†	1.33	23.1 ± 11.9*^†	1.30
Commission errors	5.6 ± 9.6		8.8 ± 11.4	14.7 ± 13.2	0.48	31.6 ± 50.5*^†	0.65	27.6 ± 36.6*^†	0.72
Reaction time (ms)	502 ± 123		589 ± 116*	739 ± 144*^†	1.16	754 ± 175*^†	1.13	758 ± 126*^†	1.40
Variability of reaction time (ms)	144 ± 58		189 ± 60*	226 ± 121	0.40	248 ± 175*	0.47	255 ± 175*	0.52
Motor activity during vigilance testing^c
Activity (mG hour^–1)	2,418 ± 2372		6,099 ± 9104	3,868 ± 4124	0.30	6,074 ± 8386	0.003	8,948 ± 8056*	0.33
Active periods	3.8 ± 7.2		2.3 ± 1.7	2.2 ± 2.1	0.05	1.9 ± 1.4	0.26	1.6 ± 1.4	0.45
Inactive periods	2.2 ± 1.5		2.1 ± 1.8	1.7 ± 2.2	0.20	1.4 ± 1.5	0.42	0.9 ± 1.6	0.70
Active-to-inactive ratio (%)	60.7 ± 31.4		64.8 ± 34.9	65.7 ± 35.3	0.03	67.5 ± 37.6	0.08	92.8 ± 16.8*^†	1.02

Note. CG = control group; ADHD = attention-deficit/hyperactivity disorder; NSG = naïve simulation group; SSG = symptom-coached simulation group; SSG+M = symptom-coached simulation group + motor activity; ES = effect sizes (Cohen’s d) of group differences between patients with ADHD and simulation groups.

^aMeasured with the Questionnaire for Complaints of Cognitive Disturbances (FLei). ^bMeasured with the Perception and Attention Functions: Vigilance (WAFV). ^cMeasured with an ActiTrac motion tracker. Note that motor activity was not measured in five patients with ADHD due to technical errors.

*Significant when compared to CG. Multivariate (per type of assessment) and univariate (per assessment measure) analyses were carried out to compare ADHD with CG, while considering age and sex as covariates. Furthermore, each simulation group was individually compared to patients with ADHD in univariate analyses. Bonferroni corrected significance level (α = .0045) was applied to control for multiple testing of 11 variables. ^†Significant when compared to ADHD. Each simulation group was individually compared to patients with ADHD in univariate analyses. Bonferroni corrected significance level (α = .0045) was applied to control for multiple testing of 11 variables.

When considering age and sex as covariates, control participants and patients with ADHD demonstrated significant differences in their self-ratings of cognitive functioning, Wilks’s lambda = .331, F(3, 74) = 49.822, p < .001, their vigilance, Wilks’s lambda = .779, F(4, 74) = 5.233, p = .001, and motor activity, Wilks’s lambda = .856, F(4, 69) = 2.909, p = .028. Compared to control participants, patients with ADHD reported significantly more severe problems of attention, F(1, 76) = 148.993, p < .001, memory, F(1, 76) = 79.672, p < .001, and executive functioning, F(1, 76) = 67.793, p < .001. In the vigilance test, patients with ADHD showed significantly more omission errors, F(1, 77) = 10.890, p = .001, longer reaction times, F(1, 77) = 16.887, p < .001, and larger fluctuation of reaction times, F(1, 77) = 16.268, p < .001, than controls. However, using the Bonferroni corrected significance level of .0045, no differences were obtained between patients with ADHD and control participants with regard to the number of commission errors, F(1, 77) = 6.637, p = .012, and the different measures of motor activity, including activity, F(1, 72) = 6.086, p = .016, number of active periods, F(1, 72) = 2.294, p = .134, number of inactive periods, F(1, 72) = 0.238, p = .627, and the active-to-inactive ratio, F(1, 72) = 0.016, p = .901.

Moreover, control participants were compared to each simulation group (NSG, SSG, and SSG+M) on each variable of the assessment. ANOVAs indicated significant differences between groups in all measures of self-reported cognitive functioning, including attention, F(3, 145) = 63.010, p < .001, memory, F(3, 145) = 34.767, p < .001, and executive functioning, F(3, 145) = 35.458, p < .001. Significant effects were also shown in all measures of the vigilance test [i.e., omissions, F(3, 148) = 19.946, p < .001, commissions, F(3, 147) = 5.050, p = .002, reaction times, F(3, 148) = 27.404, p < .001, and variability of reaction times, F(3, 148) = 4.770, p = .003]. With regard to motor activity, significant effects were observed in activity, F(3, 147) = 7.629, p < .001, and the active-to-inactive ratio, F(3, 147) = 8.077, p < .001; however, no significant effects were revealed for the number of active periods, F(3, 147) = 2.374, p = .073, and the number of inactive periods, F(3, 147) = 3.819, p = .011. Pairwise comparisons between control participants and each simulation group are presented in Table 3.

Finally, patients with ADHD were compared to each simulation group on each variable of the assessment. Significant differences between groups were found for omissions, F(3, 158) = 14.547, p < .001, commissions, F(3, 158) = 4.718, p = .004, and reaction times, F(3, 158) = 14.637, p < .001, of the vigilance test. With regard to motor activity, a significant effect was demonstrated for the active-to-inactive ratio, F(3, 152) = 6.795, p < .001. No significant differences were revealed in the remaining measures—that is, the standard deviation of reaction times in the vigilance test, F(3, 158) = 2.006, p = .115, the remaining measures of motor activity [i.e., activity, F(3, 152) = 2.820, p = .041, number of active periods, F(3, 152) = 1.444, p = .232, and number of inactive periods, F(3, 152) = 3.354, p = .021], and the self-reports of cognitive functioning [i.e., attention, F(3, 154) = 0.802, p = .495, memory, F(3, 154) = 2.683, p = .049, and executive functions, F(3, 154) = 1.492, p = .219]. Significances and effect sizes of pairwise comparisons between patients with ADHD and each simulation group are presented in Table 3. Largest effects were found on the vigilance test, with effect sizes larger than one standard deviation for omission errors and reaction times. Mostly negligible to small effects were found on self-ratings of cognitive functioning and motor activity (Cohen’s classification). When inspecting motor activity of the various groups of instructed simulators, it is observed that individuals of the SSG+M increased their activity level compared to the remaining simulation groups (Table 3). As a consequence, the largest difference when compared to patients with ADHD was found for the SSG+M, reaching effects of medium to large size (classification according to Cohen).

Detection of feigned adult ADHD

ROC analyses were carried out for omission errors and reaction times of the vigilance test, as these variables showed moderate group differences (according to the categorization of Rogers) between patients with ADHD and each of the instructed simulator groups. Omission errors were significantly predictive for feigned ADHD, with AUCs close to or at 80%—that is, for NSG: AUC = .776, SE = .051, 95% confidence interval, CI [0.675, 0.877], p < .001; the SSG: AUC = .807, SE = .049, 95% CI [0.711, 0.903], p < .001; and the SSG+M: AUC = .857, SE = .040, 95% CI [0.779, 0.936], p < .001. Similar predictive values were found for reaction times, with AUCs around 80%, including the NSG: AUC = .779, SE = .051, 95% CI [0.680, 0.879], p < .001; the SSG: AUC = .777, SE = .055, 95% CI [0.669, 0.886], p < .001; and SSG+M: AUC = .856, SE = .040, 95% CI [0.777, 0.935], p < .001. Furthermore, a moderate effect to predict feigned ADHD of the SSG+M relative to patients with ADHD was observed on the basis of the active-to-inactive ratio, AUC = .740, SE = .056, 95% CI [.630, .851], p < .001.

In addition, binary logistic regression models were calculated in order to determine the utility of the different types of assessments (self-ratings of cognitive functioning, vigilance test performance, motor activity during cognitive testing) for the prediction of feigned ADHD (collapsed group of NSG and SSG, n = 76) relative to genuine ADHD (n = 46). As presented in Table 4, significant models were derived from all types of assessments, with the largest proportion of explained variance by the vigilance test, χ²(4) = 34.490, p < .001, R² = 24.8%, while only small proportions of variance could be explained by the self-ratings of cognitive functioning, χ²(3) = 10.333, p = .016, R² = 8.3%, and motor activity, χ²(4) = 10.131, p = .038, R² = 8.3%.

Table 4. Binary logistic regression models for the prediction of feigned ADHD relative to genuine ADHD.

Predictors	B	SE B	Wald	p	OR	Bootstrap 95% CI
Model 1: Self-rating of cognitive functioning^a
Attention	0.122	0.068	3.200	.074	1.130	[−0.035, 0.297]
Memory	−0.179	0.070	6.521	.011	0.836	[−0.343, −0.058]
Executive functions	−0.018	0.051	0.124	.724	0.982	[−0.130, 0.099]
Model 2: Vigilance test performance^b
Omissions	0.071	0.052	1.828	.176	1.073	[−0.046, 0.186]
Commissions	0.017	0.016	1.069	.301	1.017	[−0.015, 0.104]
Reaction time	0.002	0.004	0.189	.664	1.002	[−0.006, 0.012]
Variability of reaction time	−0.001	0.003	0.249	.618	0.999	[−0.009, 0.001]
Model 3: Motor activity during vigilance testing^c
Activity	<0.001	<0.001	2.255	.133	1.000	[0.000, 0.000]
Active periods	0.954	0.425	5.040	.025	2.595	[0.196, 1.944]
Inactive periods	−1.129	0.428	6.952	.008	0.323	[−1.929, −0.618]
Active-to-inactive ratio	−0.013	0.101	1.711	.191	0.987	[−0.034, 0.004]

Note. ADHD = attention-deficit/hyperactivity disorder; OR = odds ratio; CI = confidence interval. Feigned ADHD (n = 76); genuine ADHD (n = 46).

^aMeasured with the Questionnaire for Complaints of Cognitive Disturbances (FLei). ^bMeasured with the Perception and Attention Functions: Vigilance (WAFV). ^cMeasured with an ActiTrac motion tracker. Note that motor activity was not measured in five patients with ADHD due to technical errors.

The incremental validity of the assessment of motor activity for the detection of feigned ADHD relative to genuine ADHD was explored by a hierarchical logistic regression model, in which vigilance test scores were entered to the model in Step 1, followed by motor activity during cognitive testing (Step 2), and self-ratings of cognitive functioning (Step 3). After vigilance test performance has been entered to the model, χ²(4) = 36.729, p < .001, R² = 27.7%, no significant additional value was given by measures of motor activity in Step 2, χ²(4) = 8.454, p = .076, R² = 33.0%, with only 5.3% additionally explained variance. A significant proportion of additional variance (8.6%) is explained in Step 3 by adding self-ratings of cognitive functioning, χ²(3) = 15.502, p = .001, R² = 41.6%.

Discussion

The goal of this study was to investigate the incremental value of motor activity for the assessment of credibility of adults being clinically evaluated for ADHD. The present study employed a simulation design in which patients with ADHD were compared to various groups of healthy individuals that differed with regard to the instructions they received prior to the assessment. Groups were compared on measures of self-reported cognitive functioning, vigilance test performance, and motor activity.

As expected, patients with ADHD reported significantly more pronounced problems of cognitive functioning and showed poorer vigilance test performance than healthy controls, also after adjusting for group differences in age and sex. These findings were in accordance with previous neuropsychological research on adult ADHD that demonstrated cognitive complaints (Fuermaier et al., 2014) as well as impaired performance in variants of the CPT of adults with ADHD (Huang-Pollock et al., 2012; Tucha et al., 2017; Tucha, Tucha, et al., 2009). Furthermore, on the basis of a Bonferroni adjusted significance level, the present study failed to reveal differences in motor activity between patients with ADHD and control participants, which is in line with the assumption that motor hyperactivity is a characteristic predominantly seen in children rather than adults with ADHD (Biederman et al., 2000; Davidson, 2008).

A crucial aspect of the validity of simulation designs is to what extent participants of simulation groups followed instructions and put effort into feigning ADHD. In the present study, all participants of simulation groups were able to respond correctly to questions regarding simulation instructions and after completion of the study also indicated that they followed instructions sincerely (no participants had to be excluded). Indeed, there is good indication that group manipulation was successful as instructed simulators, compared to control participants, showed significantly elevated scores in cognitive complaints and poorer vigilance test performance. The sensitivity of the measurement of motor activity to the different levels of group manipulation was supported by an inspection of body movements of the SSG+M compared to the CG and other simulation groups (NSG and SSG). Instructed simulators who have been informed about the measurement of motor activity prior to the assessment (SSG+M) showed higher levels of body movements than the CG and the remaining simulation groups. An explanation for these findings could be that individuals may indeed associate adult ADHD with motor hyperactivity and show such behavior in excess if they are aware that body movements are recorded. As an alternative explanation, it must be considered that the comparison of body movements between the CG and the SSG+M is confounded by differences in demand characteristics (Orne, 1962), as the SSG+M was explicitly informed that body movement was registered and analyzed. Thus, the participants of the SSG+M might have changed their behavior as a response to the hypothesis they may have assumed the researcher was testing (expectation effects).

Most importantly for the present context is the comparison between patients with ADHD and simulation groups. As expected, self-ratings of cognitive functioning had only little utility in detecting feigned ADHD, as demonstrated by nonsignificant group differences between the ADHD group and any of the simulation groups, and only two of these differences reaching medium size (classification according to Cohen). These medium-sized differences, however, did not reach clinical relevance according to the classification devised by Rogers for the assessment of malingering. This conforms with numerous previous studies that failed to show clinical utility of standard measures of self-report rating scales for the detection of noncredible ADHD symptom reporting (Booksh et al., 2010; Edmundson et al., 2017; Fuermaier, Tucha, Koerts, Weisbrod, et al., 2016; Jachimowicz & Geiselman, 2004; Quinn, 2003; Suhr et al., 2008; Tucha, Sontag, et al., 2009). Also conforming with our expectations, moderate utility for the detection of noncredible cognitive performance was obtained by the vigilance test. Instructed simulators showed overly poor performance on this test compared to genuine patients with ADHD, as indicated by significant differences in omission errors, commission errors, and reaction times, with effects reaching up to moderate and even large size (according to the classification devised by Rogers). ROC analyses revealed adequate predictive accuracy (AUC about 80%) on the basis of omission errors and reaction times, which is in line with previous research that highlighted the promising use of CPTs for the detection of noncredible cognitive performance in the clinical evaluation of adult ADHD (Booksh et al., 2010; Leppma et al., 2017; Quinn, 2003). However, contrary to our expectations, the measurement of motor activity failed to show clinical utility in distinguishing genuine from feigned ADHD, as only one of the group differences reached relevance according to the classification devised by Rogers. Some evidence on the utility of considering motor behavior in the attempt to detect feigned ADHD was given by the inspection of the simulation group, which has been informed about the measurement of motor activity (SSG+M) prior to the assessment. This group exhibited more body movement than other simulation groups, resulting in a clearly exaggerated activity (up to one standard deviation difference). For example, whereas patients with ADHD, the NSG, and SSG showed activity in about two thirds of the time during neuropsychological testing (as indicated by the active-to-inactive ratio), the SSG+M showed activity during almost the entire time of the vigilance assessment (i.e., 93% of the time). The active-to-inactive ratio also showed moderate accuracy (AUC = 74%) in classifying individuals of the SSG+M relative to genuine patients with ADHD. However, as stressed before, the comparison of body movements between the SSG+M and the ADHD group is confounded by differences in demand characteristics and must therefore be interpreted with caution.

The conclusions drawn above on the basis of effect sizes of group differences are confirmed by results of logistic regression analyses. The vigilance test revealed the largest accuracy in detecting feigned ADHD (collapsed group of NSG and SSG) relative to genuine ADHD, with 25% of the variance explained. Only small amounts of variance, though significant, were explained by models including self-ratings of cognitive functioning (8%) and motor behavior (8%). The SSG+M was not considered in logistic regression analyses in order to avoid a confounding factor of demand characteristics. Hierarchical analyses demonstrated that the assessment of motor behavior does not complement cognitive testing in detecting feigned ADHD, as indicated by a nonsignificant and small amount of additionally explained variance (5%) if measures of motor activity were added to the model after vigilance test performance has been entered first. A significant and small amount of additional variance (9%) can be explained if self-ratings of cognitive functioning are added to the model in a third and final step.

The present data do not support the utility of motor behavior for the detection of feigned adult ADHD. One may conclude that, based on the findings of the present study, motor behavior in general is not helpful for the assessment of credibility of clients in the clinical evaluation of adult ADHD. Alternatively, it can be argued that motor activity could indeed be helpful in this context if a different type of assessment measure were to be chosen. However, this suggestion only holds true if the current measure was not sensitive for the detection of feigned ADHD. Although it remains speculation, the latter explanation appears unlikely given the clearly elevated levels of motor activity of simulators who have been informed about the use of motion trackers prior to cognitive assessment, indicating that the assessment of motor behavior was in general sensitive to detect differences between groups who follow different strategies. Future studies that aim to include body movements in the clinical evaluation of credibility of clients could consider assessing body movements via different tools and measurement principles. For example, the QbTest-Plus (QbTech AB, 2010) integrates cognitive performance and head movements of participants in the diagnostic evaluation of adults with ADHD. In this measure, body movement is recorded by an infrared camera that follows a reflective marker attached to a headband and was shown to represent a reasonably sensitive and specific mean to distinguish patients with ADHD from healthy controls (Edebol, Helldin, & Norlander, 2013; Lis et al., 2010). However, a recent study of Hirsch and Christiansen (2015) showed that motor activity as measured with the QbTest did not differ significantly between adults with ADHD passing and failing an effort test (i.e., the Amsterdam Short Term Memory Test). Furthermore, when using ambulatory activity monitors during neuropsychological testing, such devices could be attached closer to the body of participants than it was done in the present study, such as around the chest or ankles of participants, in order to yield a more direct and sensitive indication of body movement that is still not confounded with movements required for the execution of neuropsychological tests (Tuisku et al., 2003). Moreover, one could consider performing measurements outside the laboratory setting, for example by requesting participants to carry ambulatory activity monitors attached to their wrist for several days during their daily routines in order to get a representative and ecologically valid indication of motor activity levels of individuals. Such measurements have been shown to be useful to detect differences in motor activity between patients with ADHD and healthy individuals (Boonstra et al., 2007), although their utility for the assessment of credibility of individuals still needs to be determined. Such measures appear promising to distinguish genuine from feigned symptoms, as it can be assumed that it is much more difficult to keep up with the attempt to simulate a condition over a longer and continuous period of time (i.e., several days), instead of convincing the clinician about having a condition within a comparable short laboratory clinical assessment.

Limitations

As a limitation to the present study, it must be noted that performance validity of patients with ADHD was assessed with an embedded validity indicator (N-back working memory test) which may have a lower sensitivity than established stand-alone PVTs. Diagnostic veracity was therefore corroborated by a careful diagnostic evaluation, including the mutual agreement between two clinicians, the identification of objective evidence of impairment, and/or the consult of collateral information. Furthermore, participation in the study was separated from clinical assessment and treatment in order to exclude possible incentives for individuals feigning ADHD to take part in the study. It was pointed out to all patients with ADHD that participation in the study was for research purposes only. Thus, it was stressed that all data were analyzed anonymously and had no consequence for their diagnostic evaluation and/or treatment.

Furthermore, the remaining sample of patients with ADHD included in the present study contained a large proportion of individuals diagnosed with the combined subtype, which is in contrast to previous research demonstrating that the inattentive subtype is most common among individuals with ADHD (Wilens et al., 2009; Willcutt, 2012). Patients with the combined subtype have been shown to be referred to clinical services more frequently (Willcutt, 2012), which may explain the large proportion of patients with this subtype in the present study. Nevertheless, it must be considered that the characteristics of the present patient sample may limit the generalization of the results.

Finally, as noted above, the interpretation of differences in body movements between the ADHD and the SSG+M is complicated by differences in demand characteristics between these groups (Orne, 1962). In order to control for this methodological issue, we considered simulation groups in individual analyses and did not include the SSG+M in logistic regression analyses to predict feigned ADHD. However, an inspection of body movements of the SSG+M is still interesting for the present study as it provides information about the sensitivity of the present approach to measure motor activity. Furthermore, this information allows the derivation of hypotheses about how the measurement of body movement could complement future assessment of credibility.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary materials

Supplemental data for this article can be accessed here.