A nonparametric approach to confidence intervals for concordance index and difference between correlated indices

ABSTRACT

Concordance refers to the probability that subjects with high values on one variable also have high values on another variable. This index has wide application in practice, as a measure of effect size in group-comparison studies, an index of accuracy in diagnostic studies, and a discrimination index for prediction models. Herein, we provide a unified framework for statistical inference involving concordance indices for standard variables of binary, ordinal, and continuous types. In particular, we develop confidence interval procedures for a single concordance index and differences between two correlated indices. Simulation results show that procedures based on logit-transformation for a single index and Fisher’s $z$-transformation for a difference between indices perform very well in terms of coverage and tail errors even when the sample size is as small as 30, unless the concordance is high and the standard is a binary variable for which at least 50 subjects are needed. We illustrate the procedures for a variety of standard variables with previously published data. Illustrative SAS code is provided.

KEYWORDS:

• Diagnostic accuracy
• discrimination
• effect size
• Wilcoxon–Mann-Whitney statistic
• receiver operating characteristic curve
• responsiveness
• $U$-statistics

Acknowledgments

Many thanks are due to two referees for constructive comments and suggestions. Dr Zou’s work is partially supported by an Individual Discovery Grant from Natural Sciences and Engineering Research Council (NSERC) of Canada.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Natural Sciences and Engineering Research Council of Canada