A nonparametric approach to confidence intervals for concordance index and difference between correlated indices

References

• Bamber, D. 1975. The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. Journal of Mathematical Psychology 12:387–415. doi:10.1016/0022-2496(75)90001-2.
   Web of Science ®Google Scholar
• Bordley, R. F. 2009. The Hippocratic Oath, effect size, and utility theory. Medical Decision Making 29:377–379. doi:10.1177/0272989X09333128.
   PubMed Web of Science ®Google Scholar
• Bossuyt, P. M., J. B. Reitsma, D. E. Bruns, C. A. Gatsonis, P. P. Glasziou, L. Irwig, J. G. Lijmer, D. Moher, D. Rennie, and H. C. de Vet, et al. 2015. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. British Medical Journal 351:h5527. doi:10.1136/bmj.h5527.
   Google Scholar
• Brunner, E., M. Vandemeulebroecke, and T. Mutze. 2021. Win odds: An adaptation of the win ratio to include ties. Statistics in Medicine 40:3367–3384. doi:10.1002/sim.8967.
   PubMed Web of Science ®Google Scholar
• Casella, G., and R. L. Berger. 2002. Statistical Inference. 2nd. Duxbury, New York.
   Google Scholar
• Cliff, N. 1996. Ordinal methods for behaviorial data analysis. Mahwah, New Jersey: Erlbaum.
   Google Scholar
• Cliff, N., and V. Charlin. 1991. Variance and covariance of Kendall tau and their estimation. Multivariate Behavioral Research 26 (4):693–707. doi:10.1207/s15327906mbr2604_6.
   PubMed Web of Science ®Google Scholar
• Daniels, H. E., and M. G. Kendall. 1947. The significance of rank correlations where parental correlation exists. Biometrika 34:197–208. doi:10.1093/biomet/34.3-4.197.
   PubMed Web of Science ®Google Scholar
• DeLong, E. R., D. M. DeLong, and D. L. Clarke-Pearson. 1988. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44:837–845. doi:10.2307/2531595.
   PubMed Web of Science ®Google Scholar
• Deyo, R. A., and R. M. Centor. 1986. Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostic test performance. Journal of Chronic Diseases 39:897–906. doi:10.1016/0021-9681(86)90038-X.
   PubMedGoogle Scholar
• Dong, G., D. C. Hoaglin, J. Qiu, R. A. Matsouaka, Y. W. Chang, J. Wang, and M. Vandemeulebroecke. 2020. The win ratio: On interpretation and handling of ties. Statistics in Biopharmaceutical Research 12:99–106. doi:10.1080/19466315.2019.1575279.
   Google Scholar
• Esscher, F. 1924. On a method of determining correlation from the ranks of the variates. Scandinavian Actuarial Journal 1:201–219. doi:10.1080/03461238.1924.10405384.
   Google Scholar
• Fay, M. P., E. H. Brittain, J. H. Shih, D. A. Follmann, and E. E. Gabriel. 2018. Causal estimands and confidence intervals associated with Wilcoxon‐Mann‐Whitney tests in randomized experiments. Statistics in Medicine 37:2923–2937. doi:10.1002/sim.7799.
   PubMed Web of Science ®Google Scholar
• Flandre, P., and J. O’Quigley. 2007. Predictive strength of Jonckheere’s test for trend: An application to genotypic scores in HIV infection. Statistics in Medicine 26:4441–4454. doi:10.1002/sim.2871.
   PubMed Web of Science ®Google Scholar
• Halperin, M., M. I. Hamdy, and P. F. Thall. 1989. Distribution-free confidence intervals for a parameter of Wilcoxon-Mann-Whitney type for ordered categories and progressive censoring. Biometrics 45:509–521. doi:10.2307/2531493.
   PubMed Web of Science ®Google Scholar
• Han, X., Y. Zhang, and Y. Shao. 2017. On comparing 2 correlated C indices with censored survival data. Statistics in Medicine 36:4041–4049. doi:10.1002/sim.7414.
   PubMed Web of Science ®Google Scholar
• Hand, D. J., and R. J. Till. 2001. A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning 45 (2):171–186. doi:10.1023/A:1010920819831.
   Web of Science ®Google Scholar
• Hanley, J. A., and B. J. McNeil. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143 (1):29–36. doi:10.1148/radiology.143.1.7063747.
   PubMed Web of Science ®Google Scholar
• Hanley, J. A., and B. J. McNeil. 1983. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148 (3):839–843. doi:10.1148/radiology.148.3.6878708.
   PubMed Web of Science ®Google Scholar
• Harrell, F. E. 2015. Regression modeling strategies: With applications to linear models, logistic regression, and survival analysis. 2nd ed. New York: Springers.
   Google Scholar
• Harrell, F. E., R. M. Califf, D. B. Pryor, K. L. Lee, and R. A. Rosati. 1982. Evaluating the yield of medical tests. Journal of the American Medical Association 247:2543–2546. doi:10.1001/jama.1982.03320430047030.
   PubMed Web of Science ®Google Scholar
• Harrell, F. E., K. L. Lee, and D. B. Mark. 1996. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in Medicine 15:361–387. doi:10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4.
   PubMed Web of Science ®Google Scholar
• Hoeffding, W. 1947. On the distribution of the rank correlation coefficients τ, when the variates are not independent. Biometrika 34:183–196.
   PubMed Web of Science ®Google Scholar
• Hoeffding, W. 1948. A class of statistics with asymptotically normal distribution. Annals of Mathematical Statistics 19:293–325. doi:10.1214/aoms/1177730196.
   Google Scholar
• Husted, J. A., R. J. Cook, V. T. Farewell, and D. D. Gladman. 2000. Methods for assessing responsiveness: A critical review and recommendations. Journal of Clinical Epidemiology 53 (5):459–468. doi:10.1016/S0895-4356(99)00206-1.
   PubMed Web of Science ®Google Scholar
• Jairath, V., J. Jeyarajah, G. Y. Zou, C. E. Parker, A. Olson, R. Khanna, G. R. D’Haens, W. J. Sandborn, and B. G. Feagan. 2019. A composite disease activity index for early drug development in ulcerative colitis: Development and validation of the UC-100 score. Lancet Gastroenterology & Hepatology 4:63–70. doi:10.1016/S2468-1253(18)30306-6.
   PubMed Web of Science ®Google Scholar
• Jonckheere, A. 1954. A distribution-free k-sample test against ordered alternatives. Biometrika 41 (1/2):133–145. doi:10.1093/biomet/41.1-2.133.
   Web of Science ®Google Scholar
• Justice, A. C., K. E. Covinsky, and J. A. Berlin. 1999. Assessing the generalizability of prognostic information. Annals of Internal Medicine 130:515–524. doi:10.7326/0003-4819-130-6-199903160-00016.
   PubMed Web of Science ®Google Scholar
• Kang, L., W. Chen, N. A. Petrick, and B. D. Gallas. 2015. Comparing two correlated c indices with right-censored survival outcome: A one-shot nonparametric approach. Statistics in Medicine 34 (4):685–703. doi:10.1002/sim.6370.
   PubMed Web of Science ®Google Scholar
• Kendall, M. G. 1938. A new measure of rank correlation. Biometrika 30 (1–2):81–93. doi:10.1093/biomet/30.1-2.81.
   Google Scholar
• Kendall, M. G. 1945. The treatment of ties in ranking problems. Biometrika 33 (3):239–251. doi:10.1093/biomet/33.3.239.
   PubMed Web of Science ®Google Scholar
• Kieser, M., T. Friede, and M. Gondan. 2013. Assessment of statistical significance and clinical relevance. Statistics in Medicine 32:1707–1719. doi:10.1002/sim.5634.
   PubMed Web of Science ®Google Scholar
• Kruskal, W. H. 1957. Historical notes on the Wilcoxon unpaired two-sample test. Journal of the American Statistical Association 52:356–360. doi:10.1080/01621459.1957.10501395.
   Web of Science ®Google Scholar
• Long, J., and N. Cliff. 1997. Confidence intervals for Kendall’s tau. British Journal of Mathematical & Statistical Psychology 50 (1):31–41. doi:10.1111/j.2044-8317.1997.tb01100.x.
   Google Scholar
• Lorenz, D. J., S. Datta, and S. J. Harkema. 2011. Marginal association measures for clustered data. Statistics in Medicine 30:3181–3191. doi:10.1002/sim.4368.
   PubMed Web of Science ®Google Scholar
• Luo, J., and C. Xiong. 2012. DiagTest3Grp: An R package for analyzing diagnostic tests with three ordinal groups. Journal of Statistical Software 51 (3):1–24. doi:10.18637/jss.v051.i03.
   PubMed Web of Science ®Google Scholar
• Mann, H. B., and D. R. Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics 18 (1):50–60. doi:10.1214/aoms/1177730491.
   Google Scholar
• Moons, K. G., D. G. Altman, J. B. Reitsma, J. Ioannidis, P. Macaskill, E. W. Steyerberg, A. J. Vickers, D. F. Ransohoff, and G. S. Collins. 2015. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): Explanation and elaboration. Annals Internal Medicine 162:W1–73. doi:10.7326/M14-0698.
   PubMed Web of Science ®Google Scholar
• Nakas, C. T. 2014. Developments in roc surface analysis and assessment of diagnostic markers in three class classification problems. Revstat Statistical Journal 12:43–65.
   Web of Science ®Google Scholar
• Nakas, C. T., and C. T. Yiannoutsos. 2004. Ordinal multiple-class ROC analysis with continuous measurements. Statistics in Medicine 23:3437–3449. doi:10.1002/sim.1917.
   PubMed Web of Science ®Google Scholar
• Obuchowski, N. 1997. Nonparametric analysis of clustered ROC curve data. Biometrics 53 (2):567–578. doi:10.2307/2533958.
   PubMed Web of Science ®Google Scholar
• Obuchowski, N. 2005. Estimating and comparing diagnostic tests’ accuracy when the gold standard is not binary. Academic Radiology 12 (9):1198–1204. doi:10.1016/j.acra.2005.05.013.
   PubMed Web of Science ®Google Scholar
• Obuchowski, N. A. 2006. An ROC-type measure of diagnostic accuracy when the gold standard is continuous-scale. Statistics in Medicine 25 (3):481–493. doi:10.1002/sim.2228.
   PubMed Web of Science ®Google Scholar
• Obuchowski, N. A., M. J. Goske, and K. E. Applegate. 2001. Assessing physicians’ accuracy in diagnosing paediatric patients with acute abdominal pain: Measuring accuracy for multiple diseases. Statistics in Medicine 20 (3):3261–3278. doi:10.1002/sim.944.
   PubMedGoogle Scholar
• Pencina, M. J., R. B. D’Agostino Sr, and L. Song. 2012. Quantifying discrimination of framingham risk functions with different survival c statistics. Statistics in Medicine 31 (15):1543–1553. doi:10.1002/sim.4508.
   PubMed Web of Science ®Google Scholar
• Rahlfs, V., and H. Zimmermann. 2019. Effect size measures and their benchmark values for quantifying benefit or risk of medicinal products. Biometrical Journal 61:973–982. doi:10.1002/bimj.201800107.
   PubMed Web of Science ®Google Scholar
• Rosner, B., and R. J. Glynn. 2017. Estimation of rank correlation for clustered data. Statistics in Medicine 36:2163–2186. doi:10.1002/sim.7257.
   PubMed Web of Science ®Google Scholar
• Sandborn, W. J., B. G. Feagan, D. C. Wolf, G. D’Haens, S. Vermeire, S. B. Hanauer, S. Ghosh, H. Smith, M. Cravets, P. A. Frohna, et al.; TOUCHSTONE Study Group. 2016. Ozanimod induction and maintenance treatment for ulcerative colitis. New England Journal of Medicine 374 (18):1754–1762. doi:10.1056/NEJMoa1513248.
   PubMed Web of Science ®Google Scholar
• Schulz, K. F., D. G. Altman, and D. Moher. 2010. CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomized trials. Annals of Internal Medicine 152 (11):1–7. for the CONSORT Group. doi:10.7326/0003-4819-152-11-201006010-00232.
   PubMed Web of Science ®Google Scholar
• Shih, J. H., and M. P. Fay. 2017. Pearson’s chi-square test and rank correlation inferences for clustered data. Biometrics 73:822–834. doi:10.1111/biom.12653.
   PubMed Web of Science ®Google Scholar
• Steyerberg, E. W. 2019. Clinical prediction models: A practical approach to development, validation, and updating. 2nd ed. New York: Springers.
   Google Scholar
• Uno, H., T. Cai, M. J. Pencina, R. B. D’Agostino, and L. Wei. 2011. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine 30 (10):1105–1117. doi:10.1002/sim.4154.
   PubMed Web of Science ®Google Scholar
• Van Calster, B., V. Van Belle, Y. Vergouwe, and E. W. Steyerberg. 2012. Discrimination ability of prediction models for ordinal outcomes: Relationships between existing measures and a new measure. Biometrical Journal 54 (5):674–685. doi:10.1002/bimj.201200026.
   PubMed Web of Science ®Google Scholar
• Wang, A., and S. Pocock. 2016. A win ratio approach to comparing continuous non-normal outcomes in clinical trials. Pharmaceutical Statistics 15:238–245. doi:10.1002/pst.1743.
   PubMed Web of Science ®Google Scholar
• Wasserstein, R. L., and N. A. Lazar. 2016. The ASA statement on p-values: Context, process, and purpose. The American Statistician 70 (2):129–133. doi:10.1080/00031305.2016.1154108.
   Web of Science ®Google Scholar
• Zou, G. 2021. Confidence interval estimation for treatment effects in cluster randomization trials based on ranks. Statistics in Medicine 40 (14):3227–3250. doi:10.1002/sim.8918.
   PubMed Web of Science ®Google Scholar