The Lack of Consistency for Statistical Decision Procedures

Abstract

Simpson's paradox exhibits seemingly deviant behavior where the data generated in independent experiments support a common decision, but the aggregated data support a different outcome. It is shown that this kind of inconsistent behavior occurs with many, if not most, statistical decision processes. Examples are given for the Kruskal-Wallis test and a Bayesian decision problem. A simple theory is given that permits one to determine whether a given decision process admits such inconsistencies, to construct examples, and to find data restrictions that avoid such outcomes.

Key Words:

• Bayesian decision theory
• Kruskal-Wallis test
• Simpson's paradox