A study of indices useful for the assessment of diagnostic markers in non-parametric ROC curve analysis

Abstract

Diagnostic markers that discriminate between two classes, such as healthy and diseased subjects in medical diagnostics, are useful in virtually all fields of applied research. ROC curve analysis is widely used in order to establish the utility of novel diagnostic markers (the single marker case) or for the comparison of competing diagnostic markers. A routine approach in the single marker case is to assess the area under the ROC curve (AUC) by testing the null hypothesis that AUC is equal to 0.5. The empirical estimate of the AUC is equivalent to the Wilcoxon-Mann-Whitney statistic and as such it may have low power to detect departures from the null hypothesis when the distributions of healthy and diseased subjects differ in location and scale and/or in shape. Competing indices include the Youden index, which is equivalent to the Kolmogorov-Smirnov statistic and focuses on a specific point on the ROC curve. In this article, we evaluate various approaches for the assessment of diagnostic markers based on goodness-of-fit testing and compare with existing standard approaches. We conclude that useful diagnostic markers may erroneously be discarded using standard practice.

Keywords:

• Area under the ROC curve (AUC)
• goodness-of-fit testing
• Kolmogorov-Smirnov
• receiver operating characteristic (ROC) curves

Acknowledgments

The authors would like to thank assc. prof. K. Sarafidis for providing the application data. We are grateful to referee for his/her helpful comments that improved this article.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was partially supported by Spanish Ministry of Economy and Competitiveness grant MTM2016-75351-R.