Neuropsychological tests of the future: How do we get there from here?

Abstract

Objective: This article reviews current approaches to neuropsychological assessment, identifies opportunities for development of new methods using modern psychometric theory and advances in technology, and suggests a transition path that promotes application of novel methods without sacrificing validity. Methods: Theoretical/state-of-the-art review.Conclusions: Clinical neuropsychological assessment today does not reflect advances in neuroscience, modern psychometrics, or technology. Major opportunities for improving practice include both psychometric and technological strategies. Modern psychometric approaches including item response theory (IRT) enable linking procedures that can place different measures on common scales; adaptive testing algorithms that can dramatically increase efficiency of assessment; examination of differential item functioning (DIF) to detect measures that behave differently in different groups; and person fit statistics to detect aberrant patterns of responding of high value for performance validity testing. Opportunities to introduce novel technologies include computerized adaptive testing, Web-based assessment, healthcare- and bio-informatics strategies, mobile platforms, wearables, and the ‘internet-of-things’. To overcome inertia in current practices, new methods must satisfy requirements for back-compatibility with legacy instrumentation, enabling us to leverage the wealth of validity data already accrued for classic procedures. A path to achieve these goals involves creation of a global network to aggregate item-level data into a shared repository that will enable modern psychometric analyses to refine existing methods, and serve as a platform to evolve novel assessment strategies, which over time can revolutionize neuropsychological assessment practices world-wide.

Keywords:

• Neuropsychology
• psychometrics
• psychological tests
• clinical decision-making
• diagnostic techniques and procedures
• information science

Acknowledgements

The authors are grateful for the input from our collaborators, including Russell Bauer, Daniel Drane, James Holdnack, David Loring, and David Sabsevitz.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 IRT models also can consider free-response items where there are in theory an infinite number of responses (i.e. in responses to the Vocabulary subtest, many different kinds of responses are observed in practice); the challenge is for test designers and examiners to capture and evaluate the different responses rapidly enough to be useful.

2 There are notable exceptions (e.g. partial credit is given for certain responses on Vocabulary, Similarities, or the Rey-Osterrieth Complex Figure Test), and different scores are given for other WAIS subtests if completed within certain pre-specified time limits.

3 It is noteworthy that one of the original leaders in the development of IRT was Fred Lord, a leader of development at ETS (Carlson & von Davier, 2013).

4 This example uses some plausible percentile levels but the actual CAT algorithm will select items based on item-information curves, not by picking percentiles.

5 Nielsen Scarborough (n.d.). Number of cell phone users in the United States from spring 2008 to spring 2017 (in millions). In Statista - The Statistics Portal. Retrieved 4 April 2018, from https://www.statista.com/statistics/231612/number-of-cell-phone-users-usa/.

6 CTIA, the International Association for the Wireless Telecommunications Industry. Retrieved 4 April 2018 from https://web.archive.org/web/20120820013725/ http://www.ctia.org/consumer_info/index.cfm/AID/10323.

7 Deloitte Technology Survey, retrieved 4 April 2018 from https://www2.deloitte.com/us/en/pages/technology-media-and-telecommunications/articles/global-mobile-consumer-survey-us-edition.html.

8 Estimating 500,000 NP assessments each year in the United States, to obtain 1000 healthy comparison cases per year would only demand a healthy person be included for every 500 clinical NP assessments. Even assuming that only a fraction of all clinics participate, the distributed burden remains relatively low.

9 Gordon, J. The Future of RDoC By Joshua Gordon on 5 June 2017. Accessed 4/10/2018 at https://www.nimh.nih.gov/about/director/messages/2017/the-future-of-rdoc.shtml.

Additional information

Funding

Preparation of this manuscript was supported by grants from the National Institute of Mental Health (R01MH101478, R03MH106922, and U01MH105578).