Causal Quartets: Different Ways to Attain the Same Average Treatment Effect

Abstract

The average causal effect can often be best understood in the context of its variation. We demonstrate with two sets of four graphs, all of which represent the same average effect but with much different patterns of heterogeneity. As with the famous correlation quartet of Anscombe, these graphs dramatize the way in which real-world variation can be more complex than simple numerical summaries. The graphs also give insight into why the average effect is often much smaller than anticipated.

KEYWORDS:

• Causal inference
• Statistical graphics
• Variation

Acknowledgments

We thank Dan Goldstein, Stephen Stigler, Howard Wainer, and two anonymous reviewers for helpful comments and the U.S. Office of Naval Research and Institute of Education Sciences for partial support of this work.

Disclosure Statement

No potential conflict of interest was reported by the authors.

Notes

1 https://github.com/jhullman/causalQuartet