Utility of an attention-based performance validity test for the detection of feigned cognitive dysfunction after acquired brain injury

Figures & data

Table 1. Characteristics of patients with acquired brain injury.

	TBI		Stroke		Total
Descriptives
N	23		27		50
Age (in years)	36.5 ± 15.5		47.6 ± 15.0		42.5 ± 16.1
Gender (female/male)	6/17		14/13		20/30
Education (in years)	14.0 ± 3.9		13.9 ± 3.6		13.9 ± 3.6
Employment status (unemployed/in training/working/on sick leave/retired)	2/2/2/15/2		0/0/4/21/2		2/2/6/36/4
IQ (vocabulary skills)^a	96.4 ± 9.0		108.5 ± 14.8		103.3 ± 14.0
Neuroimaging^b	20		27		47
Medication^c (yes/no)	16/7		23/4		39/11
TBI severity^d (minor/moderate/severe)	2/1/14		-		-
Stroke type (hemorrhagic/ischemic)	-		6/21		-
Stroke^e Hemisphere (right/left/both)	-		11/12/1		-
Duration of illness in months	12.5 ± 15.7		2.6 ± 2.6		7.5 ± 12.1
Rehabilitation phase (C/D/E)^f	1/17/2		0/25/1		1/42/3
Cognitive functioning		% percentile ≤ 10		% percentile ≤ 10
Visuoconstructive abilities^g	34.4 ± 18.6	28.6	26.0 ± 16.0	18.5	29.7 ± 17.6
Alertness RT^h	367.8 ± 373.7	34.8	259.1 ± 61.4	15.4	310.1 ± 262.7
Selective Attention RT^i	624 ± 269	42.9	683 ± 259	63.6	655 ± 263
Selective Attention Om^j	8.3 ± 6.8	19.0	6.4 ± 7.8	17.4	7.3 ± 7.3
Divided Attention Om^k	3.8 ± 5.0	40.9	2.1 ± 2.1	25.0	2.9 ± 3.9
Word Fluency^l	16.2 ± 6.1	47.4	17.6 ± 7.4	33.3	17.0 ± 6.8

^aMultiple Choice Vocabulary Test (MWT-B);

^bBrain damage was supported by neuroimaging;

^cNumber of patients known to be taking medication at the time of the assessment: Antidepressants, stimulants, analgesics, beta blockers;

^dGlasgow Coma Scale (GCS) severity classification for patients with TBI;

^eSide of the lesion;

^fGerman classification of rehabilitation phases after ABI;

^gBlock design of the WAIS-IV;

^hTAP phasic Alertness reaction time in milliseconds;

ⁱTAP Visual Scanning reaction time in seconds;

^jTAP Visual Scanning Omissions;

^kTAP Divided Attention Omission;

^lRegensburg Word Fluency Test (S-words). Stroke = All patients with Stroke; TBI = All patients with TBI; Total = All Patients with ABI.

Table 2. Characteristics of participants: control group, patients with ABI, and simulation groups (Mean±SD).

	CG	PABI	Sim	NSTBI	CSTBI	NSStroke	CSStroke
N	40	50	192	52	48	46	46
Age (in years)	19.8 ± 1.5	42.5 ± 16.1	20.2 ± 2.2	20.8 ± 2.7	20.3 ± 2.1	20.1 ± 2.0	19.7 ± 1.7
Gender (female/male)	26/14	20/30	113/79	32/20	30/18	27/19	24/22
Education (in years)	13.2 ± 1.3	13.9 ± 3.6	13.5 ± 1.6	13.6 ± 2.0	13.5 ± 1.3	13.3 ± 1.6	13.3 ± 1.4
IQ (vocabulary skills)^a	^b	103.3 ± 14.0	97.2 ± 7.9	96.5 ± 6.7	96.9 ± 7.7	98.0 ± 8.2	97.6 ± 9.4

^aMultiple Choice Vocabulary Test (MWT-B); CG = Control group; P_ABI = Patients with ABI; Sim = All instructed simulation groups collapsed; NS_TBI = Naïve simulation TBI group; CS_TBI = Symptom-coached simulation TBI group; NS_Stroke = Naïve simulation stroke group; CS_Stroke = Symptom-coached simulation stroke group;

^b= Not assessed in the control group.

Table 3. Groningen Effort Test (GET) performance of control participants, patients with ABI, and simulation groups (Mean±SD).

Measures	CG (n = 40)	PABI (n = 50)	Sim (n = 192)
Block 1, RT (sec)	3.8 ± 2.5	6.7 ± 5.1	5.0 ± 2.3
Block 1, Errors	1.1 ± 1.4	2.2 ± 2.6	6.6 ± 3.8
Block 2, RT (sec)	3.7 ± 3.4	5.3 ± 3.5	4.4 ± 2.2
Block 2, Errors	1.2 ± 1.3	2.2 ± 2.5	7.4 ± 4.1
Block 3, RT (sec)	2.0 ± 1.9	3.7 ± 2.7	3.5 ± 2.0
Block 3, Errors	.6 ± .7	1.5 ± 2.3	6.7 ± 4.1
Block 4, RT (sec)	1.7 ± 1.3	3.6 ± 3.0	3.1 ± 1.8
Block 4, Errors	.3 ± .5	1.3 ± 2.0	6.7 ± 4.3
Mean RT^a	2.8 ± 2.2	4.8 ± 3.2^c	4.0 ± 1.9^c,d
Total Errors^b	3.1 ± 2.3	7.2 ± 7.8^c	27.3 ± 14.8^c,d

^aMean reaction time in seconds for all four blocks combined;

^bNumber of total Errors over all four blocks. CG = Control group; P_ABI = Patients with ABI; Sim = All instructed simulation groups collapsed;

^cSignificantly different from CG at p < .05;

^dSignificantly different from P_ABI at p < .05.

Figure 1. STARD diagram reporting flow of participants and diagnostic accuracy of GET-ABI index (cutoff .791).

Figure 2. ROC curve depicting diagnostic accuracy of the GET-ABI index and GET-ADHD index in identifying feigned cognitive dysfunction (n = 192) relative to genuine cognitive dysfunction (n = 50).

Table 4. Performance validity tests of control participants, patients with ABI, and simulation groups (Mean ± SD).

	CG (n = 40)	PABI (n = 50)	Sim (n = 192)
GET-ABI index	−.7 ± .5	−1.3 ± 1.9	5.0 ± 4.0
GET-ADHD index	−1.7 ± 1.7	−1.1 ± 3.1	6.7 ± 5.3
b Test E-score	40.2 ± 11.1	^a	655.0 ± 1075.0
DCT E-score	8.6 ± 2.0	10.5 ± 3.9	20.4 ± 8.9
TOMM T2	50.0 ± .0	49.9 ± .5	29.5 ± 8.2
TOMM R	50.0 ± .2	49.8 ± .7	28.6 ± 8.3

CG = Control group; P_ABI = Patients with ABI; Sim = All instructed simulation groups collapsed; GET = Groningen Effort Test; DCT = Dot Counting Test; TOMM T2 = Test Of Memory Malingering Trial 2; TOMM R = Test Of Memory Malingering Retention Trial;

^a= Not assessed in patients with ABI.

Table 5. Diagnostic accuracy of various validity tests in distinguishing feigned (n = 192) from genuine (n = 50) cognitive dysfunction, as indicated by levels of sensitivity, specificity, and area under the curve.

Cutoff	Sensitivity	Specificity	AUC
GET-ADHD index			.914
−.25^a	92.2	64
2.21^b	82.3	90
GET-ABI index			.949
.791^b	88.5	90
TOMM
≤45^c	98.4	^h
DCT (E-Score)
TBI ≥ 20, Stroke ≥ 22^d	39.1	^h
≥13.80^e	80.7	^h
b Test (E-Score)
TBI ≥ 90, Stroke ≥ 170^f	70.3	^i
≥82^g	86.5	^i

AUC = Area under the curve; GET = Groningen Effort Test; DCT = Dot Counting Test; TOMM = Test Of Memory Malingering;

^a= cutoff adopted from ADHD research (Fuermaier et al., 2016);

^b= cutoff derived from this study in order to maximize specificity to at least 90%;

^c= cutoff adopted from test manual suggesting noncredible performance: trial 2 or retention trial ≤45 (Tombaugh, 1996);

^d= cutoff adopted from test manual suggesting noncredible performance: TBI ≥ 20, Stroke ≥ 22 (Boone et al., 2002a);

^ecutoff adopted from McCaul et al. (2018);

^f= cutoff adopted from test manual suggesting noncredible performance: TBI ≥ 90, Stroke ≥ 170 (Boone et al., 2002b);

^gcutoff adopted from Roberson et al. (2013);

^h= not possible to derive as passing was an inclusion criteria for entering this study;

ⁱ= Not assessed in patients with ABI.

Figure 3. ROC curve depicting diagnostic accuracy of the GET-ABI index in identifying feigned cognitive dysfunction (n = 192) relative to genuine cognitive dysfunction (n = 55) after partialing out DCT or TOMM.

Table 6. Diagnostic accuracy of various validity tests to detect feigning (N(%)) per simulation strategy used: speed, inattention, memory, executive functions.

	Speed	Inattention	Memory	Executive functions
	(n = 40)	(n = 102)	(n = 34)	(n = 135)
GET-ABI index	34 (85.0)	91 (89.2)	28 (82.4)	117 (86.7)
TOMM	39 (97.5)	100 (98)	31 (91.2)	133 (98.5)
DCT	12 (30)	29 (28.4)	9 (26.5)	45 (33.3)
b Test	25 (62.5)	68 (66.7)	19 (55.9)	86 (63.7)

GET = Groningen Effort Test; DCT = Dot Counting Test; TOMM = Test Of Memory Malingering.

Fuermaier, A. B., Tucha, O., Koerts, J., Grabski, M., Lange, K. W., Weisbrod, M., … Tucha, L. (2016). The development of an embedded figures test for the detection of feigned attention deficit hyperactivity disorder in adulthood. PloS One, 11(10), e0164297.

 PubMed Web of Science ®Google Scholar

Tombaugh, T. N. (1996). Test of memory malingering. Toronto, Canada: Multi-Health Systems.

 Google Scholar

Boone, K., Lu, P., & Herzberg, D. (2002a). The dot counting test. Los Angeles, CA: Western Psychological Service.

 Google Scholar

McCaul, C., Boone, K. B., Ermshar, A., Cottingham, M., Victor, T. L., Ziegler, E., … Wright, M. (2018). Cross-validation of the Dot Counting Test in a large sample of credible and non-credible patients referred for neuropsychological testing. The Clinical Neuropsychologist, 32(6), 1054–1067.

 PubMed Web of Science ®Google Scholar

Boone, K., Lu, P., & Herzberg, D. (2002b). The b Test. Los Angeles, CA: Western Psychological Service.

 Google Scholar

Roberson, C. J., Boone, K. B., Goldberg, H., Miora, D., Cottingham, M., Victor, T., … Wright, M. (2013). Cross validation of the b test in a large known groups sample. The Clinical Neuropsychologist, 27(3), 495–508.

 PubMed Web of Science ®Google Scholar

Data availability statement

The data that support the findings of this study are available from the corresponding author, ABMF, upon reasonable request.