Concordance measures and time-dependent ROC methods

Abstract

The receiver operating characteristic (ROC) curve displays sensitivity versus 1-specificity over a set of thresholds. The area under the ROC curve (AUC) is a global scalar summary of this curve. In the context of time-dependent ROC methods, we are interested in global scalar measures that summarize sequences of time-dependent AUCs over time. The concordance probability is a candidate for such purposes. The concordance probability can provide a global assessment of the discrimination ability of a test for an event that occurs at random times and may be right censored. If the test adequately differentiates between subjects who survive longer times and those who survive shorter times, this will assist clinical decisions. In this context, the concordance probability may support the assessment of precision medicine tools based on prognostic biomarkers models for overall survival. Definitions of time-dependent sensitivity and specificity are reviewed. Some connections between such definitions and concordance measures are also reviewed and we establish new connections via new measures of global concordance. We explore the relationship between such measures and their corresponding time-dependent AUC. To illustrate these concepts, an application in the context of Alzheimer's disease is presented.

Keywords:

• Time-dependent sensitivity and specificity
• diagnostic test
• censored survival times
• inverse probability weighting
• Alzheimer's disease

Acknowledgments

This work was initiated when N. Pantoja-Galicia was a postdoctoral fellow at Harvard University and the manuscript was completed and submitted when he was at the Center for Devices and Radiological Health (CDRH), US Food and Drug Administration.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Alzheimer's Association [grant number NIRG-09-133383] and Harvard NeuroDiscovery Center and National Institute on Aging [grant number P01AG004953], [grant number P50-AG005134], [grant number P30-AG062421] and National Institute of Neurological Disorders and Stroke [grant number R01NS094610].

Notes on contributors

Norberto Pantoja-Galicia

Norberto Pantoja-Galicia has a PhD in Statistics from the University of Waterloo (Ontario, Canada). His career includes a postdoctoral research fellowship at the Harvard T.H. Chan School of Public Health and a position as Visiting Scientist (Mathematical Statistician) at the Center for Devices and Radiological Health (CDRH), US Food and Drug Administration (FDA). He is currently Associate Director, Biostatistics at Foundation Medicine Inc. His research interests include statistical methods in diagnostic medicine and benefit-risk assessment.

Olivia I. Okereke

Olivia Okereke received her Doctor of Medicine degree from Yale University and Master of Science in Epidemiology degree from the Harvard T. H. Chan School of Public Health (HSPH). She is Associate Professor of Psychiatry at Harvard Medical School, Associate Professor in Epidemiology at HSPH, and Director of Geriatric Psychiatry at Massachusetts General Hospital in Boston, MA, USA.

Deborah Blacker

Deborah Blacker holds an MD from Harvard Medical School (HMS) and an ScD from the Harvard T.H. Chan School of Public Health (HSPH) (both Boston, Massachusetts, USA). A geriatric psychiatrist and epidemiologist based at Mass General Hospital (Boston, Massachusetts), she directs the Gerontology Research Unit and serves as Associate Chief for Research in the Department of Psychiatry. She is Professor of Psychiatry at HMS and Deputy Chair and Professor in Epidemiology at HSPH. Her work focuses on the epidemiology, genetics, assessment, and early recognition of Alzheimer's disease.

Rebecca A. Betensky

Rebecca A. Betensky has a PhD in Statistics from Stanford University. She is Professor and Chair of Biostatistics at the New York University School of Global Public Health. Her research focuses on issues of censoring, truncation, biased sampling, biomarker discovery and clinical trials. She collaborates on studies of Alzheimer's disease.