A Comparison of Methods for the Evaluation of Binary Measurement Systems

Many quality programs prescribe a measurement system analysis (MSA) to be performed on the key quality characteristics. This guarantees the reliability of the acquired data, which serve as the basis for drawing conclusions with respect to the behavior of the key quality characteristics. When dealing with continuous characteristics, the Gauge R&R is regarded as the statistical technique in MSA. For binary characteristics, no such universally accepted equivalent is available. We discuss methods that could serve as an MSA for binary data. We argue that a latent class model is the most promising candidate.

Keywords:

• Kappa
• Intraclass correlation coefficient
• Latent class model
• Log-linear model
• Measurement system analysis.

Notes

¹We realize that the automotive industry (Automotive Industry Action Group, 2002) has prescribed a way to conduct an attribute gauge study. It assumes the qualitative evaluation of a part can be compared with an underlying continuous measurement. However, as knowledge of this continuous measurement is not always at hand, we disregard it here.

²The proof of these restrictions is beyond the scope of this article.

³This violates the identification restrictions but can be coped with by in addition requiring that π _j (0) = 1 − π _j (1) for j = 1,…,m.

⁴We realize that all previous comparisons of techniques are illustrated by the two rater case, and that the example involves three raters. This inconsistency is due to the following reasons: (1) only for three (or more) raters is the latent class model identifiable, (2) all methods are also applicable to the multiple rater case, and (3) the differences between the latent class model and the alternative techniques also hold for the multiple rater case.

^†Current affiliation: Department of Statistics, Free University of Amsterdam.

^§Current affiliation: Organon NV.