Resurrecting the Third Variable: A Critique of Pearl's Causal Analysis of Simpson's Paradox

Abstract

Pearl argued that Simpson's Paradox would not be considered paradoxical but for statisticians’ unwillingness to acknowledge the role of causality in resolving an instance of it. He proposed using a causal calculus to determine which set of contradictory findings in an instance of the paradox should be accepted—the aggregated data or the data disaggregated by conditioning on the third variable. Pearl used the example of a hypothetical quasi-experiment to argue that when third variables are not causal, one should not condition on them, and—assuming no other sources of confounding—the aggregated data should be accepted. Pearl was precipitate in his argument that it would be inappropriate to condition on the noncausal third variables in the example. Whether causal or not, third variables can convey critical information about a first-order relationship, study design, and previously unobserved variables. Any conditioning on a nontrivial third variable that produces Simpson's Paradox should be carefully examined before either the aggregated or the disaggregated findings are accepted, regardless of whether the third variable is thought to be causal. In some cases, neither set of data is trustworthy; in others, both convey information of value. Pearl's hypothetical example is used to illustrate this argument.

• Causality
• Confounding
• Inference
• Intermediate variable
• Reversal effect
• Spurious correlation.

View correction statement:

Correction

Notes

Pearl's point that we should not condition on noncausal variables has been made by many others, (Breslow and Day 1980, Greenland and Neutra 1980; Weinberg 1993; Greenland et al. 1999; Rothman, Greenland, and Lash 2008, pp. 131–134, 194–196, and 260; Schisterman, Cole, and Platt 2009). Philosopher N. Cartwright is of the same mind, arguing this point two years after Lindley and Novick, but in less detail than Pearl and others. See Cartwright (1983, pp. 37–38); also Cartwright (1979, 2001), and Dupre and Cartwright (1988).