Not so fast! Limitations of processing speed and working memory indices as embedded performance validity tests in a mixed neuropsychiatric sample

ABSTRACT

Introduction

Validity indicators embedded within standard neuropsychological tests have received increasing attention as more efficient measures for sampling performance validity throughout an evaluation. This cross-sectional study examined multiple performance validity tests (PVTs) embedded in the Wechsler Adult Intelligence Scale-IV (WAIS-IV) Working Memory (WMI) and Processing Speed (PSI) Indices for detecting invalid test performance.

Method

This cross-sectional study examined data from a mixed clinical neuropsychiatric sample of 110 patients referred for outpatient evaluation. The sample was composed of 85 patients with valid neuropsychological performance and 25 with invalid performance based on multiple independent criterion PVTs. Among the patients with valid performance, 54% were cognitively impaired, whereas 46% were cognitively unimpaired.

Results

Among the overall sample, performance on WMI, PSI, and 3/4 constituent subtests (i.e., Digit Span, Symbol Search, Coding) was significantly worse among the invalid group compared to the valid group (η_p² =.06–.16) with areas under the curve (AUCs) of.67–.76 and 24–32% sensitivity (≥88% specificity) for identifying invalid performance at cut-scores that maximized accuracy. When the sample was subdivided by cognitive impairment status, AUCs of.68–.87 and 36–56% sensitivity (≥87% specificity) for detecting invalidity at cut-scores that maximized accuracy were found among those without cognitive impairment. In contrast, for patients with cognitive impairment, Digit Span, Arithmetic, WMI, and Coding were nonsignificant, and AUCs of.66–.67. Further, notably reduced sensitivities of 16–28% (≥91% specificity) were found for the remaining significant indices.

Conclusion

Overall, results indicated that embedded WAIS-IV WMI and PSI are useful embedded PVTs in conditions in which cognitive impairment is not expected; however, these embedded PVTs demonstrated questionable utility among patients with cognitive impairment due to poor sensitivity, if adequate specificity is maintained, suggesting limited efficacy among patients with cognitive impairment due to risk of false-positive classification.

KEYWORDS:

• Performance validity
• processing speed
• working memory
• psychometrics
• assessment

Disclosure statement

No potential conflict of interest was reported by the author(s).