Note on the Cochran Q Test

Abstract

Cochran's Q test for differences between related-sample percentages or proportions has generally been incorrectly presented in secondary sources. The most common mistake results from failure to recognize that rows containing only 1's or only 0's, i.e., only successes or only failures, do not affect the value of Q. The F test, however, is affected by such rows. The probabilities from the χ² and F approximations are compared with the exact probabilities in three sets of data. A rule of thumb, based on extensive study of the distribution of 0 in small samples, is given as an aid in judging when the χ² approximation is satisfactory for practical purposes.