On summary ROC curve for dichotomous diagnostic studies: an application to meta-analysis of COVID-19

Abstract

In a systematic review of a diagnostic performance, summarizing performance metrics is crucial. There are various summary models in the literature, and hence model selection becomes inevitable. However, most existing large-sample-based model selection approaches may not fit in a meta-analysis of diagnostic studies, typically having a rather small sample size. Researchers need to effectively determine the final model for further inference, which motivates this article to investigate existing methods and to suggest a more robust method for this need. We considered models covering several widely-used methods for bivariate summary of sensitivity and specificity. Simulation studies were conducted based on different number of studies and different population sensitivity and specificity. Then final models were selected using several existing criteria, and we compared the summary receiver operating characteristic (sROC) curves to the theoretical ROC curve given the generating model. Even though parametric likelihood-based criteria are often applied in practice for their asymptotic property, they fail to consistently choose appropriate models under the limited number of studies. When the number of studies is as small as 10 or 5, our suggestion is best in different scenarios. An example for summary ROC curves for chemiluminescence immunoassay (CLIA) used in COVID-19 diagnosis is also illustrated.

Keywords:

• Summary ROC curve
• sensitivity
• specificity
• meta analysis
• systematic review
• random effects

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

Financial support for this study was provided in part by a grant from National Science Council, Taiwan [grant number NSC102-2118-M038-002], and Ministry of Science and Technology, Taiwan [grant numbers MOST104-2314-B039-037, MOST107-2118-M-110-004-MY3 and MOST108-2628-M-008-005-MY3]. The funding agreement ensured the authors' independence in designing the study, interpreting the results, writing, and publishing the report.