Location Bias in ROC Studies

ASTRACT

Location bias occurs when a reader detects a false lesion in a subject with disease and the falsely detected lesion is considered a true positive. In this study, we examine the effect of location bias in two large MRMC ROC studies, comparing three ROC scoring methods. We compare one method that only uses the maximum confidence score and does not take location bias into account (maxROC), and two methods that take location bias into account: the region of interest ROC (ROI–ROC) and the free-response ROC (FROC). In both studies, when comparing two modalities' ROC areas without adjusting for location bias, the effect size depends on the difference in the frequency of location bias between the two modalities. When the difference in frequency is small, the effect size is similar whether the location bias is corrected for or not. However, when the frequency of location bias is dissimilar, failure to correct for the location bias favors the modality with higher false positive rate. Location bias should be corrected when the next step in the clinical management of the patient depends on the specific location of the detected lesion and/or when the frequency of the bias is dissimilar between the two modalities.

KEYWORDS:

• Bias
• Free-response ROC
• Multi-reader multi-case
• Receiver operating characteristic
• Region of interest ROC