False positive diagnosis of malingering due to the use of multiple effort tests

Abstract

Objective: Effort indicators are used to determine if neuropsychological test results are valid measures of a patient’s cognitive abilities. The use of multiple effort measures is often advocated, but the false positive rate for multiple indicators depends on the number of measures used and the correlation among indicators. This study presents a meta-analysis of correlations among effort measures. False positive rates for multiple correlated indicators are then estimated using Monte Carlo simulations.

Methods: a literature search of published studies identified 22 independent samples in which 407 correlations among 31 effort measures were available in 3564 participants with normal effort. Participants were patients with neurological or psychiatric disorders and healthy volunteers.

Results: Meta-analysis showed a mean correlation among effort indicators of 0.31. Monte Carlo simulation based on a 15% false positive rate for individual indicators showed that, when 10 effort indicators are used together, 38% of patients with valid performance will be incorrectly identified as malingering if two failures is the diagnostic standard. Failure on five of 10 measures is required for a false positive rate of 10% or less. If five effort indicators are interpreted, a false positive rate of 19% results when two test failures are assumed to characterize poor effort and failure on three measures is required to maintain 90% specificity.

Conclusions: False positive rates for effort tests increase significantly as the number of indicators that are administered is increased.

• Base rate
• effort tests
• malingering
• medicolegal
• symptom exaggeration
• symptom validity

Acknowledgements

The authors gratefully thank Drs Patrick Armistead-Jehle, Nina Blaskweitz, Wm. Drew Gouvier, James Holdnack, Ashlee Loughan, William MacAllister, Regina McGlinchey, Thomas Merten, William Milberg, Justin Miller, Scott Millis, Nathaniel Nelson, NCS Pearson Inc., Russell Pella, Robert Perna, Barry Rosenfeld, Adriana Strutt and Antoinette Welsh for providing the correlational data presented in this paper and Dr John Crawford for providing the Monte Carlo simulation program and his assistance in the interpretation of the simulation results.