Estimating Precision Using Duplicate Measurements

Abstract

Precision is a concept for which there is no universally accepted metric. Reports of precision vary depending on the formula and inclusion criteria used to calculate them. To properly interpret and utilize reported precisions, the user must understand exactly what the precision represents. This paper uses duplicate Interagency Monitoring of Protected Visual Environments (IMPROVE) measurements to illustrate distinctions among different approaches to reporting precision. Three different metrics are used to estimate the precision from the relative differences between the duplicate measurements: the root mean square (RMS), the mean absolute value, and a percentile spread. Precisions calculated using the RMS relative difference yield wide distributions that tend to overestimate most of the observed differences. Precisions calculated using percentiles of the relative differences yield narrower distributions that tend to fit the bulk of the observed differences very well. Precisions calculated using the mean absolute relative difference lie between the other two precision estimates. All three approaches underestimate the observed differences for a small percentage of outliers.