Applications of the Bootstrap in ROC Analysis

Abstract

The problem of estimating standard errors for diagnostic accuracy measures might be challenging for many complicated models. We can address such a problem by using the Bootstrap methods to blunt its technical edge with resampled empirical distributions. We consider two cases where bootstrap methods can successfully improve our knowledge of the sampling variability of the diagnostic accuracy estimators. The first application is to make inference for the area under the ROC curve resulted from a functional logistic regression model which is a sophisticated modelling device to describe the relationship between a dichotomous response and multiple covariates. We consider using this regression method to model the predictive effects of multiple independent variables on the occurrence of a disease. The accuracy measures, such as the area under the ROC curve (AUC) are developed from the functional regression. Asymptotical results for the empirical estimators are provided to facilitate inferences. The second application is to test the difference of two weighted areas under the ROC curve (WAUC) from a paired two sample study. The correlation between the two WAUC complicates the asymptotic distribution of the test statistic. We then employ the bootstrap methods to gain satisfactory inference results. Simulations and examples are supplied in this article to confirm the merits of the bootstrap methods.

Keywords:

• Additive model
• Empirical process
• Logistic regression
• Two-sample U-statistic
• von Mises expansion

Mathematics Subject Classification:

• 62G09
• 62G20
• 62P10
• 62P15

Acknowledgments

This work was partially supported by grants ARF R-155000082, R-155000109112 and NMRC/NIG/0054/2009. The author is grateful to Professor John Rush for his helpful discussion about the analysis of STAR*D data set.