Tutorial on Biostatistics: Receiver-Operating Characteristic (ROC) Analysis for Correlated Eye Data

ABSTRACT

Purpose

To demonstrate methods for receiver-operating characteristic (ROC) analysis of correlated eye data.

Methods

We applied the Obuchowski’s nonparametric approach and cluster bootstrap for estimating and comparing the area under ROC curve (AUC) between different sets of predictors to three datasets with varying inter-eye correlation.

Results

In an optic neuritis (ON) study of 152 eyes (80 patients), the AUC of optical coherence tomography retinal nerve fiber layer thickness for diagnosing ON (inter-eye kappa = 0.13) was 0.71 [95% confidence interval (95% CI): 0.622, 0.792] from the naïve approach without accounting for inter-eye correlation was narrower than from nonparametric (95% CI: 0.613, 0.801) or cluster bootstrap (95% CI: 0.614, 0.797) approaches. In an analysis of 198 eyes (135 patients), the baseline Age-related Eye disease Study scale predicted 5-year incidence of advanced age-related macular degeneration (inter-eye kappa = 0.23) with AUC of 0.72. The 95% CI from the naïve approach was slightly narrower (0.645, 0.794) than from the nonparametric (0.641, 0.797) or cluster bootstrap (0.641, 0.793) approaches. In an analysis of 1542 eyes (771 infants), birthweight and gestational age predicted treatment-requiring retinopathy of prematurity (inter-eye kappa = 0.98) with AUC of 0.80. Furthermore, the 95% CI from the naïve approach was narrower (0.769, 0.835) than from the nonparametric (0.755, 0.848) or cluster bootstrap (0.755, 0.845) approaches. 95% CIs for AUC differences between different models were narrower in the naïve approach than the nonparametric or cluster bootstrap approaches.

Conclusion

In ROC analysis of correlated eye data, ignoring inter-eye correlation leads to narrower 95% CI with underestimation dependent on magnitude of inter-eye correlation. Nonparametric and cluster bootstrap approaches properly account for inter-eye correlation.

KEYWORDS:

• Ocular test
• area under ROC curve
• ROC analysis
• cluster bootstrap
• correlated eye data

Disclosure statement

All authors have no conflict of interest disclosure to disclose.

Supplementary material

Supplemental data for this article can be accessed on the publisher’s website.

Additional information

Funding

Supported by grants [R01EY022445 and P30 EY01583-26] from the National Eye Institute, National Institutes of Health, Department of Health and Human Services.