Cross-validation of the Trail Making Test as a non-memory-based embedded performance validity test among veterans with and without cognitive impairment

ABSTRACT

Objective

This study cross-validated multiple Trail Making Test (TMT) Parts A and B scores as non-memory-based embedded performance validity tests (PVTs) for detecting invalid neuropsychological performance among veterans with and without cognitive impairment.

Method

Data were collected from a demographically and diagnostically diverse mixed clinical sample of 100 veterans undergoing outpatient neuropsychological evaluation at a Southwestern VA Medical Center. As part of a larger battery of neuropsychological tests, all veterans completed TMT A and B and four independent criterion PVTs, which were used to classify veterans into valid (n = 75) and invalid (n = 25) groups. Among the valid group 47% (n = 35) were cognitively impaired.

Results

Among the overall sample, all embedded PVTs derived from TMT A and B raw and demographically corrected T-scores significantly differed between validity groups (ηp² = .21–.31) with significant areas under the curve (AUCs) of .72–.78 and 32–48% sensitivity (≥91% specificity) at optimal cut-scores. When subdivided by cognitive impairment status (i.e., valid-unimpaired vs. invalid; valid-impaired vs. invalid), all TMT scores yielded significant AUCs of .80–.88 and 56%–72% sensitivity (≥90% specificity) at optimal cut-scores. Among veterans with cognitive impairment, neither TMT A or B raw scores were able to significantly differentiate the invalid from the valid-cognitively impaired group; however, demographically corrected T-scores were able to significantly differentiate groups but had poor classification accuracy (AUCs = .66–.68) and reduced sensitivity of 28%–44% (≥91% specificity).

Conclusions

Embedded PVTs derived from TMT Parts A and B raw and T-scores were able to accurately differentiate valid from invalid neuropsychological performance among veterans without cognitive impairment; however, the demographically corrected T-scores generally were more robust and consistent with prior studies compared to raw scores. By contrast, TMT embedded PVTs had poor accuracy and low sensitivity among veterans with cognitive impairment, suggesting limited utility as PVTs among populations with cognitive dysfunction.

KEYWORDS:

• Performance validity
• noncredible performance
• assessment
• psychometrics
• neuropsychology

Acknowledgments

We acknowledge the contribution of the staff, trainees, and technicians who have contributed and the Veterans who agreed to have their information included in the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Financial statement

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Additional information

Funding

This material results from work supported by resources and the use of facilities at the South Texas Veterans Health Care System (STVHCS) in San Antonio, Texas. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.