Convergent and discriminant validity of the ImPACT with traditional neuropsychological measures

Abstract

Neuropsychological assessment of cognitive sequelae secondary to sports concussion is limited by lengthy administration times and lack of readily available neuropsychologists. Brief computerized test batteries are now under development to address this, but the validity of these measures is not yet established. The validity of one such computerized test battery, the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT), was administered to 93 healthy NCAA Division I athletes, aged 18–24, along with a battery of traditional, well-described neuropsychological tests. Convergent and discriminant validity between the ImPACT and traditional measures was investigated using multitrait-multimethod matrix (MTMM) analysis. As an example, the ImPACT Visual Motor Speed composite demonstrated reasonably good convergent validity secondary to moderate correlations with traditional measures of processing speed, but it demonstrated relatively poor discriminant validity as it significantly correlated with the Reaction Time composite score. MTMM results were variable across ImPACT composites and data for each are presented. The ImPACT composite’s validity was further investigated using exploratory factor analysis (EFA). Six principal components were termed processing speed, visual memory, verbal memory, attention & working memory, and verbal fluency, based upon traditional test loadings, and a sixth loaded only on the ImPACT Reaction Time composite. EFA indicated content validity of moderate strength for the Visual Motor Speed and Visual Memory composites, but revealed problems with specificity for the other composites. Based upon the present findings, validity problems render the interpretability of the ImPACT composites somewhat questionable, and more research is necessary prior to using the ImPACT for assessment of clinical populations.

Keywords:

• head injury
• traumatic brain injury
• concussion
• test construction

Public Interest Statement

Theoretical constructs are intangible and hypothetical variables used to help understand human behavior. When a new psychological test is developed to measure a theoretical construct, it is critical that the test have adequate construct validity, which means that the test indeed measures the construct of interest. Convergent validity refers to the extent to which a test measures the same thing as other tests purported to measure that construct (i.e. scores on all tests of verbal memory should be highly correlated) and discriminant validity describes the extent to which a test does not measure other, unrelated constructs (i.e. verbal memory test scores should not necessarily correlate with reaction time scores). This study investigated how healthy athlete’s performance on the ImPACT test battery compared to their performance on traditional neuropsychological tests whose construct validity is well understood. It was found that the construct validity of ImPACT subtests was variable leading to the conclusion that more research is needed prior to ImPACT being used as a clinical assessment.

Competing interests

The authors declare no competing interest.

Additional information

Notes on contributors

Robert J. Thoma

Thoma and Campbell are senior faculty members in the University of New Mexico Department of Psychiatry and Behavioral Sciences, and are neuropsychologists who serve clinical patients at the University of New Mexico (UNM) Hospital. McGrew, PhD, is a Sports Medicine physician who works with UNM student athletes. Julia Cook was a research assistant and a UNM student athlete at the time this manuscript was written and is now a Plastic Surgery Resident at the Indiana University School of Medicine. Our co-authors additionally include neuropsychologists and students at all levels who have an ongoing interest and expertise in sports concussion and neuropsychological assessment. This project was conceived as an investigation of alternative approaches to assessment of athletes who had undergone a concussion during play, to better help direct their care. The research team consisted of experts in medicine and neuropsychology, and of student athletes who were determined to help achieve the project goals. Everyone involved was concerned for the health and safety of student athletes.