Agreement or Association: Choosing a Measure of Reliability for Nominal Data in the 2 × 2 Case—A Comparison of Phi, Kappa, and G

Abstract

This research note compares the ϕ, kappa, and G measures of association and agreement used to estimate the reliability of drug and alcohol self-report data in the 2 × 2 case. Attention is focused upon the problems encountered in using these measures when skewed marginal distributions are found in a test–retest situation. The G index of agreement was found to be a stable estimator, unaffected by skewed marginals, and equilivalent to kappa and ϕ when marginal distributions are equal.