Invalid before impaired: an emerging paradox of embedded validity indicators

References

• An, K. Y., Kaploun, K., Erdodi, L. A., & Abeare, C. A. (2017). Performance validity in undergraduate research participants: A comparison of failure rates across tests and cutoffs. The Clinical Neuropsychologist, 31, 193–206.10.1080/13854046.2016.1217046
   PubMed Web of Science ®Google Scholar
• An, K. Y., Zakzanis, K. K., & Joordens, S. (2012). Conducting research with non-clinical healthy undergraduates: Does effort play a role in neuropsychological test performance? Archives of Clinical Neuropsychology, 27, 849–857.10.1093/arclin/acs085
   PubMed Web of Science ®Google Scholar
• Arnold, G., Boone, K. B., Lu, P., Dean, A., Wen, J., Nitch, S., & McPhearson, S. (2005). Sensitivity and specificity of finger tapping test scores for the detection of suspect effort. The Clinical Neuropsychologist, 19, 105–120.
   PubMed Web of Science ®Google Scholar
• Axelrod, B. N., Fichteberg, N. L., Millis, S. R., & Wertheimer, J. C. (2006). Detecting incomplete effort with Digit Span from the Wechsler Adult Intelligence Scale – third edition. The Clinical Neuropsychologist, 20, 513–523.10.1080/13854040590967117
   PubMed Web of Science ®Google Scholar
• Babikian, T., Boone, K. B., Lu, P., & Arnold, G. (2006). Sensitivity and specificity of various Digit Span scores in the detection of suspect effort. The Clinical Neuropsychologist, 20, 145–159.10.1080/13854040590947362
   PubMed Web of Science ®Google Scholar
• Baker, D. A., Connery, A. K., Kirk, J. W., & Kirkwood, M. W. (2014). Embedded performance validity indicators within the California Verbal Learning Test, Children’s Version. The Clinical Neuropsychologist, 28, 116–127.10.1080/13854046.2013.858184
   PubMed Web of Science ®Google Scholar
• Baldessarini, R. J., Finklestein, S., & Arana, G. W. (1983). The predictive power of diagnostic tests and the effect of prevalence of illness. Archives of General Psychiatry, 40, 569–573.
   PubMed Web of Science ®Google Scholar
• Bauer, L., Yantz, C. L., Ryan, L. M., Warden, D. L., & McCaffrey, R. J. (2005). An examination of the California Verbal Learning Test II to detect incomplete effort in a traumatic brain injury sample. Applied Neuropsychology, 12, 202–207.10.1207/s15324826an1204_3
   PubMed Web of Science ®Google Scholar
• Bigler, E. D. (2012). Symptom validity testing, effort, and neuropsychological assessment. Journal of the International Neuropsychological Society, 18, 632–642.10.1017/S1355617712000252
   PubMed Web of Science ®Google Scholar
• Bigler, E. D. (2014). Effort, symptom validity testing, performance validity testing and traumatic brain injury. Brain Injury, 28(13–14), 1623–1638.10.3109/02699052.2014.947627
   PubMed Web of Science ®Google Scholar
• Bigler, E. D. (2015). Neuroimaging as a biomarker in symptom validity and performance validity testing. Brain Imaging and Behavior, 9, 421–444. doi:10.1007/s11682-015-9409-1
   PubMed Web of Science ®Google Scholar
• Boone, K. B. (2009). The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examination. The Clinical Neuropsychologist, 23, 729–741.10.1080/13854040802427803
   PubMed Web of Science ®Google Scholar
• Boone, K. B. (2013). Clinical practice of forensic neuropsychology. New York, NY: Guilford.
   Google Scholar
• Bortnik, K. E., Boone, K. B., Marion, S. D., Amano, S., Ziegler, E., Victor, T. L., & Zeller, M. A. (2010). Examination of various WMS-III logical memory scores in the assessment of response bias. The Clinical Neuropsychologist, 24, 344–357.
   PubMed Web of Science ®Google Scholar
• Brooks, B., Holdnack, J., & Iverson, G. (2011). Advanced clinical interpretation of the WAIS-IV and WMS-IV: Prevalence of low scores varies by level of intelligence and years of education. Assessment, 18, 156–167.10.1177/1073191110385316
   PubMed Web of Science ®Google Scholar
• Brooks, B. L., & Ploetz, D. M. (2015). Embedded performance validity on the CVLT-C for youth with neurological disorders. Archives of Clinical Neuropsychology, 30, 200–206.10.1093/arclin/acv017
   PubMed Web of Science ®Google Scholar
• Bush, S. S., Heilbronner, R. L., & Ruff, R. M. (2014). Psychological assessment of symptom and performance validity, response bias, and malingering: Official position of the Association for Scientific Advancement in Psychological Injury and Law. Psychological Injury and Law, 7, 197–205.10.1007/s12207-014-9198-7
   Google Scholar
• Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., … Silver, C. (2005). Symptom validity assessment: Practice issues and medical necessity (NAN Policy and Planning Committees). Archives of Clinical Neuropsychology, 20, 419–426.10.1016/j.acn.2005.02.002
   PubMed Web of Science ®Google Scholar
• Busse, M., & Whiteside, D. (2012). Detecting suboptimal cognitive effort: Classification accuracy of the Conners’ Continuous Performance Test-II, Brief Test of Attention, and Trail Making Test. The Clinical Neuropsychologist, 26, 675–687.10.1080/13854046.2012.679623
   PubMed Web of Science ®Google Scholar
• Chafetz, M. D., Williams, M. A., Ben-Porath, Y. S., Bianchini, K. J., Boone, K. B., Kirkwood, M. W., … Ord, J. S. (2015). Official position of the American Academy of Clinical Neuropsychology Social Security Administration policy on validity testing: Guidance and recommendations for change. The Clinical Neuropsychologist, 29, 723–740.10.1080/13854046.2015.1099738
   PubMed Web of Science ®Google Scholar
• Curtis, K. L., Thompson, L. K., Greve, K. W., & Bianchini, K. J. (2008). Verbal fluency indicators of malingering in traumatic brain injury: Classification accuracy in known groups. The Clinical Neuropsychologist, 22, 930–945.10.1080/13854040701563591
   PubMed Web of Science ®Google Scholar
• Davis, J. J., Axelrod, B. N., McHugh, T. S., Hanks, R. A., & Millis, S. R. (2013). Number of impaired scores as a performance validity indicator. Journal of Clinical and Experimental Neuropsychology, 35, 413–420.10.1080/13803395.2013.781134
   PubMed Web of Science ®Google Scholar
• Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. (2000). The California Verbal Learning Test-second edition. San Antonio, TX: The Psychological Corporation.
   Google Scholar
• Denning, J. H. (2012). The efficiency and accuracy of the Test of Memory Malingering Trial 1, errors on the first 10 items of the Test of Memory Malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27, 417–432.10.1093/arclin/acs044
   PubMed Web of Science ®Google Scholar
• DeRight, J., & Jorgensen, R. S. (2015). I just want my research credit: Frequency of suboptimal effort in a non-clinical healthy undergraduate sample. The Clinical Neuropsychologist, 29, 101–117.10.1080/13854046.2014.989267
   PubMed Web of Science ®Google Scholar
• Erdodi, L. A., Abeare, C. A., Lichtenstein, J. D., Tyson, B. T., Kucharski, B., … Roth, R. M. (2017). WAIS-IV processing speed scores as measures of non-credible responding – The third generation of embedded performance validity indicators. Psychological Assessment, 29, 148–157. doi:10.1037/pas0000319
   PubMed Web of Science ®Google Scholar
• Erdodi, L. A., Kirsch, N. L., Lajiness-O’Neill, R., Vingilis, E., & Medoff, B. (2014). Comparing the Recognition Memory Test and the Word Choice Test in a mixed clinical sample: Are they equivalent? Psychological Injury and Law, 7(3), 255–263.10.1007/s12207-014-9197-8
   Google Scholar
• Erdodi, L. A., Lichtenstein, J. D., Rai, J. K., & Flaro, L. (2016). Embedded validity indicators in Conners’ CPT-II: Do adult cutoffs work the same way in children? Applied Neuropsychology: Child. Advance online publication. doi:10.1080/21622965.2016.1198908
   PubMed Web of Science ®Google Scholar
• Erdodi, L. A., Pelletier, C. L., & Roth, R. M. (2016). Elevations on select Conners’ CPT-II scales indicate noncredible responding in adults with traumatic brain injury. Applied Neuropsychology: Adult. Advance online publication. doi:10.1080/23279095.2016.1232262
   Web of Science ®Google Scholar
• Erdodi, L. A., Richard, D. C. S., & Hopwood, C. (2009). Scoring error variance in the WISC-IV vocabulary subtest. Journal of Psychoeducational Assessment, 27, 374–385.10.1177/0734282909332913
   Web of Science ®Google Scholar
• Erdodi, L. A., & Roth, R. M. (2017). Low scores on BDAE complex ideational material are associated with invalid performance in adults without aphasia. Advance online publication. Applied Neuropsychology: Adult, 24, 264–274.
   PubMed Web of Science ®Google Scholar
• Erdodi, L. A., Roth, R. M., Kirsch, N. L., Lajiness-O’neill, R., & Medoff, B. (2014). Aggregating validity indicators embedded in Conners’ CPT-II outperforms individual cutoffs at separating valid from invalid performance in adults with traumatic brain injury. Archives of Clinical Neuropsychology, 29, 456–466.10.1093/arclin/acu026
   PubMed Web of Science ®Google Scholar
• Erdodi, L. A., Tyson, B. T., Abeare, C. A., Lichtenstein, J. D., Pelletier, C. L., Rai, J. K., & Roth, R. M. (2016). The BDAE Complex Ideational Material – A measure of receptive language or performance validity? Psychological Injury and Law, 9, 112–120.10.1007/s12207-016-9254-6
   Web of Science ®Google Scholar
• Erdodi, L. A., Tyson, B. T., Shahein, A., Lichtenstein, J. D., Abeare, C. A., Pelletiere, C. L., … Roth, R. M. (2017). The power of timing: Adding a time-to-completion cutoff to the Word Choice Test and Recognition Memory Test improves classification accuracy. Journal of Clinical and Experimental Neuropsychology, 39, 369–383. doi:10.1080/13803395.2016.1230181
   PubMed Web of Science ®Google Scholar
• Etherton, J. L., Bianchini, K. J., Heinly, M. T., & Greve, K. W. (2006). Pain, malingering, and performance on the WAIS-III Processing Speed Index. Journal of Clinical and Experimental Neuropsychology, 28, 1218–1237.
   PubMed Web of Science ®Google Scholar
• Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6, 218–224.
   Google Scholar
• Greve, K. W., Curtis, K. L., Bianchini, K. J., & Ord, J. S. (2009). Are the original and second edition of the California Verbal Learning Test equally accurate in detecting malingering? Assessment, 16, 237–248.
   PubMed Web of Science ®Google Scholar
• Hand, D. J. (2009). Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning, 77, 103–123.
   Web of Science ®Google Scholar
• Hayward, L., Hall, W., Hunt, M., & Zubrick, S. R. (1987). Can localized brain impairment be simulated on neuropsychological test profiles? Australian and New Zealand Journal of Psychiatry, 21, 87–93.
   PubMed Web of Science ®Google Scholar
• Heaton, R. K., Miller, S. W., Taylor, M. J., & Grant, I. (2004). Revised comprehensive norms for an expanded Halstead-Reitan battery: Demographically adjusted neuropsychological norms for African American and Caucasian adults. Lutz, FL: Psychological Assessment Resources.
   Google Scholar
• Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Conference Participants (2009). American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23, 1093–1129.10.1080/13854040903155063
   PubMed Web of Science ®Google Scholar
• Heinly, M. T., Greve, K. W., Bianchini, K., Love, J. M., & Brennan, A. (2005). WAIS Digit-Span-based indicators of malingered neurocognitive dysfunction: Classification accuracy in traumatic brain injury. Assessment, 12, 429–444.
   PubMed Web of Science ®Google Scholar
• Hilsabeck, R. C., Gordon, S. N., Hietpas-Wilson, T., & Zartman, A. L. (2011). Use of Trial 1 of the Test of Memory Malingering (TOMM) as a screening measure of effort: Suggested discontinuation rules. The Clinical Neuropsychologist, 25, 1228–1238.10.1080/13854046.2011.589409
   PubMed Web of Science ®Google Scholar
• Iverson, G. L., Lange, R. T., Green, P., & Franzen, M. D. (2002). Detecting exaggeration and malingering with the trail making test. The Clinical Neuropsychologist, 16, 398–406.10.1076/clin.16.3.398.13861
   PubMed Web of Science ®Google Scholar
• Jones, A. (2013). Test of memory malingering: Cutoff scores for psychomeytrically defined malingering groups in a military sample. The Clinical Neuropsychologist, 27, 1043–1059.10.1080/13854046.2013.804949
   PubMed Web of Science ®Google Scholar
• Lange, R. T., Iverson, G. L., Brickell, T. A., Staver, T., Pancholi, S., Bhagwat, A., & French, L. M. (2013). Clinical utility of the Conners’ Continuous Performance Test-II to detect poor effort in U.S. military personnel following traumatic brain injury. Psychological Assessment, 25, 339–352.10.1037/a0030915
   PubMed Web of Science ®Google Scholar
• Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17, 410–425.
   PubMed Web of Science ®Google Scholar
• Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22, 410–425.
   Web of Science ®Google Scholar
• Larrabee, G. J. (2012). Assessment of malingering. In G. J. Larrabee (Ed.), Forensic neuropsychology: A scientific approach (pp. 117–159). New York, NY: Oxford University Press.
   Google Scholar
• Larrabee, G. J., Millis, S. R., & Myers, J. E. (2008). Sensitivity to brain dysfunction of the Halstead-Reitan vs an ability-focused neuropsychological battery. The Clinical Neuropsychologist, 22, 813–825.10.1080/13854040701625846
   PubMed Web of Science ®Google Scholar
• Lichtenstein, J. D., Erdodi, L. A., & Linnea, K. S. (2017). Introducing a forced-choice recognition task to the California Verbal Learning Test – Children’s Version. Child Neuropsychology, 23, 284–299. doi:10.1080/09297049.2015.1135422
   PubMed Web of Science ®Google Scholar
• Lichtenstein, J. D., Erdodi, L. A., Rai, J. K., Mazur-Mosiewicz, A., & Flaro, L. (2016). Wisconsin Card Sorting Test embedded validity indicators developed for adults can be extended to children. Child Neuropsychology. Advance online publication. doi:10.1080/09297049.2016.1259402
   Web of Science ®Google Scholar
• Lobo, J. M., Jimenez-Valverde, A., & Real, R. (2008). AUC: A misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography, 17, 145–151.
   Web of Science ®Google Scholar
• Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth complex figure test and the meyers and meyers recognition trial in the detection of suspect effort. The Clinical Neuropsychologist, 17, 426–440.
   PubMed Web of Science ®Google Scholar
• Michelson, H., & Lundberg, I. (1996). Neuropsychological verbal tests may lack “hold” properties in occupational studies of neurotoxic effects. Occupational and Environmental Medicine, 53, 478–483.10.1136/oem.53.7.478
   PubMed Web of Science ®Google Scholar
• Miele, A. S., Gunner, J. H., Lynch, J. K., & McCaffrey, R. J. (2012). Are embedded validity indices equivalent to free-standing symptom validity tests? Archives of Clinical Neuropsychology, 27, 10–22.10.1093/arclin/acr084
   PubMed Web of Science ®Google Scholar
• Nelson, N. W., Boone, K., Dueck, A., Wagener, L., Lu, P., & Grills, C. (2003). The relationship between eight measures of suspect effort. The Clinical Neuropsychologist, 17, 263–272.10.1076/clin.17.2.263.16511
   PubMed Web of Science ®Google Scholar
• Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
   Google Scholar
• Odland, A. P., Lammy, A. B., Martin, P. K., Grote, C. L., & Mittenberg, W. (2015). Advanced administration and interpretation of multiple validity tests. Psychological Injury and Law, 8, 46–63.10.1007/s12207-015-9216-4
   Web of Science ®Google Scholar
• Ord, J. S., Boettcher, A. C., Greve, K. J., & Bianchini, K. J. (2010). Detection of malingering in mild traumatic brain injury with the Conners’ Continuous Performance Test-II. Journal of Clinical and Experimental Neuropsychology, 32, 380–387.10.1080/13803390903066881
   PubMed Web of Science ®Google Scholar
• Pearson, N. C. S. (2009). Advanced clinical solutions for WAIS-IV and WMS-IV: Administration and scoring manual. San Antonio, TX: The Psychological Corporation.
   Google Scholar
• Reedy, S. D., Boone, K. B., Cottingham, M. E., Glaser, D. F., Lu, P. H., Victor, T. L., Ziegler, E. A., Zeller, M. A., & Wright, M. J. (2013). Cross validation of the Lu and colleagues (2003) Rey-Osterrieth Complex Figure Test effort equation in a large known-group sample. Archives of Clinical Neuropsychology, 28, 30–37.
   PubMed Web of Science ®Google Scholar
• Reese, C. S., Suhr, J. A., & Riddle, T. L. (2012). Exploration of malingering indices in the Wechsler Adult Intelligence Scale – Fourth Edition Digit Span subtest. Archives of Clinical Neuropsychology, 27, 176–181.10.1093/arclin/acr117
   PubMed Web of Science ®Google Scholar
• Root, J. C., Robbins, R. N., Chang, L., & Van Gorp, W. G. (2006). Detection of inadequate effort on the California Verbal Learning Test – Second Edition: Forced choice recognition and critical item analysis. Journal of the International Neuropsychological Society, 12, 688–696.
   PubMed Web of Science ®Google Scholar
• Ross, T. P., Poston, A. M., Rein, P. A., Salvatore, A. N., Wills, N. L., & York, T. M. (2016). Performance invalidity base rates among healthy undergraduate research participants. Archives of Clinical Neuropsychology, 31, 97–104.10.1093/arclin/acv062
   PubMed Web of Science ®Google Scholar
• Ryan, J. J., Prifitera, A., & Powers, L. (1983). Scoring reliability on the WAIS-R. Journal of Consulting and Clinical Psychology, 51, 149–150.10.1037/0022-006X.51.1.149
   Web of Science ®Google Scholar
• Santos, O. A., Kazakov, D., Reamer, M. K., Park, S. E., & Osmon, D. C. (2014). Effort in college undergraduate is sufficient on the Word Memory Test. Archives of Clinical Neuropsychology, 29, 609–613.10.1093/arclin/acu039
   PubMed Web of Science ®Google Scholar
• Schwartz, E. S., Erdodi, L., Rodriguez, N., Jyotsna, J. G., Curtain, J. R., Flashman, L. A., & Roth, R. M. (2016). CVLT-II forced choice recognition trial as an embedded validity indicator: A systematic review of the evidence. Journal of the International Neuropsychological Society, 22, 851–858.10.1017/S1355617716000746
   PubMed Web of Science ®Google Scholar
• Seke, K., Sagar, S., Charron, C., Zougari, A., Baker, J., Roth, R., & Erdodi, L. A. (2017). The Stroop test as a measure of performance validity in adults clinically referred for neuropsychological assessment. Poster presented at the 45th Annual Convention of the International Neuropsychological Society, New Orleans, LA.
   Google Scholar
• Shura, R. D., Miskey, H. M., Rowland, J. A., Yoash-Gatz, R. E., & Denning, J. H. (2016). Embedded performance validity measures with postdeployment veterans: Cross-validation and efficiency with multiple measures. Applied Neuropsychology: Adult, 23, 94–104.10.1080/23279095.2015.1014556
   PubMed Web of Science ®Google Scholar
• Silk-Eglit, G. M., Stenclik, J. H., Gavett, B. E., Adams, J. W., Lynch, J. K., & Mccaffrey, R. J. (2014). Base rate of performance invalidity among non-clinical undergraduate research participants. Archives of Clinical Neuropsychology, 29, 415–421.10.1093/arclin/acu028
   PubMed Web of Science ®Google Scholar
• Slate, J. R., & Jones, C. H. (1990). Examiner errors on the WAIS-R: A source of concern. The Journal of Psychology, 124, 343–345.10.1080/00223980.1990.10543229
   Web of Science ®Google Scholar
• Spencer, R. J., Axelrod, B. N., Drag, L. L., Waldron-Perrine, B., Pangilinan, P. H., & Bieliauskas, L. A. (2013). WAIS-IV Reliable Digit Span is no more accurate than Age Corrected Scaled Score as an indicator of invalid performance in a veteran sample undergoing evaluation for mTBI. The Clinical Neuropsychologist, 27, 1362–1372.10.1080/13854046.2013.845248
   PubMed Web of Science ®Google Scholar
• Sugarman, M. A., & Axelrod, B. N. (2015). Embedded measures of performance validity using verbal fluency tests in a clinical sample. Applied Neuropsychology: Adult, 22, 141–146.10.1080/23279095.2013.873439
   PubMed Web of Science ®Google Scholar
• Suhr, J. A., & Boyer, D. (1999). Use of the Wisconsin Card Sorting Test in the detection of malingering in student simulator and patient samples. Journal of Clinical and Experimental Psychology, 21, 701–708.
   PubMed Web of Science ®Google Scholar
• Trueblood, W. (1994). Qualitative and quantitative characteristics of malingered and other invalid WAIS-R and clinical memory data. Journal of Clinical and Experimental Neuropsychology, 14, 697–607.
   Google Scholar
• Viera, A. J. (2008). Odds ratios and risk ratios: What’s the difference and why does it matter? Southern Medical Journal, 101, 730–735.
   PubMed Web of Science ®Google Scholar
• Wald, N. J. & Bestwick, J. P. (2014). Is the area under an ROC curve a valid measure of the performance of a screening or diagnostic test? Journal of Medical Screening, 21, 51–56.
   PubMed Web of Science ®Google Scholar
• Welsh, A. J., Bender, H. A., Whitman, L. A., Vasserman, M., & MacAllister, W. (2012). Clinical utility of reliable digit span in assessing effort in children and adolescents with epilepsy. Archives of Clinical Neuropsychology, 27, 735–741.10.1093/arclin/acs063
   PubMed Web of Science ®Google Scholar
• Wolfe, P. L., Millis, S. R., Hanks, R., Fichtenberg, N., Larrabee, G. J., & Sweet, J. J. (2010). Effort indicators within the California Verbal Learning Test-II (CVLT-II). The Clinical Neuropsychologist, 24, 153–168.
   PubMed Web of Science ®Google Scholar
• Young, G. (2015). Malingering in forensic disability-related assessments. Psychological Injury and Law, 8, 188–199.10.1007/s12207-015-9232-4
   Web of Science ®Google Scholar