![Sample our Bioscience journals, sign in here to start your access, Latest two full volumes FREE to you for 14 days]({"type":"image" , "src" : "/sda/5925/TOC-Subject-Sample-2021-Bioscience.jpg"})
Abstract
When evaluating the clinical performance of a medical imaging device, a multi-reader multi-case (MRMC) analysis is usually applied to account for both case and reader variability. For a clinical task that equates to a quantitative measurement, an agreement analysis such as a limits of agreement (LOA) method can be used to compare different measurement methods. In this work, we introduce four types of comparisons; these types differ depending on whether the measurements are within or between readers and within or between modalities. A three-way mixed effect ANOVA model is applied to estimate the variances of individual differences, which is an essential step for estimating LOA. To verify the estimates of LOA, we propose a hierarchical model to simulate quantitative MRMC data. Two simulation studies were conducted to validate both the simulation and the LOA variance estimates. From the simulation results, we can conclude that our estimate of variance is unbiased, and the uncertainty of the estimation drops as the number of readers and cases increases and rises as the value of true variance increases.
Supplementary Materials
There are two files in supplementary materials. One is an appendix in illustrating the derivations of means and variances of differences from the simulation model. In the other supplementary material, we presented the simulation results by the Roe and Metz model and compare them with those of the proposed Normal-IG hierarchical model.
Funding
The author(s) reported there is no funding associated with the work featured in this article.