Failures of common measures of agreement in medicine and the need for a better tool: Feinstein's paradoxes and the dual vision method

Abstract

The main objective of the present report is to show failures in binary cases of the kappa, Yule and phi approaches when used for measuring agreement beyond chance. These methods were designed for studying the statistical concept of agreement, and for this reason they may fail when used in clinical situations. To solve these problems, or paradoxes, a new method for studying agreement is used—the dual vision procedure. The capacity of the new method to solve the paradoxes that other approaches cannot solve is demonstrated. The study of agreement is used to decide whether one clinical method can replace another one. An incorrect replacement could be dangerous for the patient. These failures caused by kappa, Yule or phi assessment are illustrated by using the clinical concept of agreement.

• Agreement table
• disagreements odds
• kappa
• level of agreement
• phi
• sensitivity
• specificity
• truth table
• Youden's index
• Yule