Statistical Significance of Ranking Paradoxes

Abstract

When nonparametric statistical tests are used to rank-order a list of alternatives, Simpson-like paradoxes arise, in which the individual parts give rise to a common decision, but the aggregate of those parts gives rise to a different decision. Haunsperger (2003) and Bargagliotti (2009) showed that the Kruskal-Wallis (Kruskal and Wallis, 1952), Mann-Whitney (Mann and Whitney, 1947), and Bhapkar's V (Bhapkar, 1961) nonparametric statistical tests are subject to these types of paradoxes. We further investigate these ranking paradoxes by showing that when they occur, the differences in rankings are not statistically significant.

Keywords:

• Kruskal-Wallis
• Mann-Whitney
• Nonparametric
• Paradox
• Ranking
• Statistical test

Mathematics Subject Classification:

• 62G10