Accuracy of Cognitive Screening Instruments Reconsidered: overall, Balanced Or Unbiased accuracy?

Abstract

Aim: To examine three different accuracy metrics for evaluation of cognitive screening instruments: overall correct classification accuracy (Acc), the sum of true positives and negatives divided by the total number tested; balanced accuracy (balanced Acc), half of the sum of sensitivity and specificity; and unbiased accuracy (unbiased Acc), removing biasing effects of random associations between test results and disease prevalence. Materials & methods: Data from a prospective test accuracy study of Mini-Addenbrooke’s Cognitive Examination were used to calculate and plot the Acc measures. Results: Each Acc metric resulted in a similar pattern of results across the range of Mini-Addenbrooke’s Cognitive Examination cut-offs for diagnosis of both dementia and mild cognitive impairment. Acc and balanced Acc gave more optimistic outcomes (closer to possible maximum value of 1) than unbiased Acc. Conclusion: Unbiased Acc may have advantages over Acc and balanced Acc by removing biasing effects of random associations between test result and disease prevalence.

Plain language summary

Tests used to assist medical diagnosis hopefully give a correct answer but sometimes may prove incorrect. Various measures are used to assess how good or effective diagnostic tests are. One of these, ‘accuracy’, measures the fraction of correct diagnoses. However, accuracy (Acc) has shortcomings which might be improved upon using other measures such as ‘balanced Acc’ and ‘unbiased Acc’. This study compared these various measures using information from a study of a screening test for dementia. The study showed that balanced Acc and unbiased Acc may be preferable to accuracy in the evaluation of diagnostic tests, especially in circumstances where the numbers of patients in the population studied differ markedly in the numbers with or without the diagnosis being sought.

Keywords:

• data accuracy
• dementia
• diagnosis
• mild cognitive impairment
• neurocognitive test
• psychometrics

Financial & competing interests disclosure

The author has no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants, or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.