What is Test Accuracy? Comparing Unitary Accuracy Metrics for Cognitive Screening Instruments

Abstract

Aim: To examine four different accuracy metrics for assessment of commonly used cognitive screening instruments: correct classification accuracy, area under the receiver operating characteristic curve, F measure (F) or F1 score and Matthews correlation coefficient (MCC). Methods: Raw data were extracted from test accuracy studies of Mini-Mental State Examination. Montreal Cognitive Assessment, Mini-Addenbrooke’s Cognitive Examination, Six-item Cognitive Impairment Test, informant AD8 and Free-Cog, and used to calculate the accuracy measures. Results: Each metric resulted in similar ordering of the screening instruments for diagnosis of both dementia and mild cognitive impairment. Area under the receiver operating characteristic curve gave the highest (most optimistic) and MCC the lowest (most pessimistic) accuracy value for each test examined, with correct classification accuracy and F falling between. Conclusion: All the accuracy measures examined have potential shortcomings. None can be recommended as the definitive unitary outcome measure for test accuracy studies. However, MCC has theoretical advantages and might be more widely adopted.

Keywords:

• accuracy
• cognitive screening instruments
• dementia
• diagnosis
• mild cognitive impairment

Financial & competing interests disclosure

The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.