Sensitivity Analysis and Extensions of Testing the Causal Direction of Dependence: A Rejoinder to Thoemmes (2019)

Abstract

A commentary by Thoemmes on Wiedermann and Sebastian’s introductory article on Direction Dependence Analysis (DDA) is responded to in the interest of elaborating and extending direction dependence principles to evaluate causal effect directionality. Considering Thoemmes’ observation that some DDA principles are already well-established in machine learning, we argue that several other connections between DDA and research lines in theoretical statistics, econometrics, and quantitative psychology exist, suggesting that DDA is best conceptualized as a framework that summarizes and extends existing knowledge on properties of linear models under non-normality. Further, Thoemmes articulates concerns about assumptions of error distributions used in our main article and presents an artificial data example in which some DDA tests have suboptimal statistical power. We present extensions of DDA to entirely relax distributional assumptions about errors and describe two sensitivity analysis approaches to critically evaluate DDA results. Both sensitivity approaches are illustrated using Thoemmes’ artificial data example. Incorporating DDA sensitivity results yields correct causal conclusions. Thus, overall, we stay with our initial conclusion that the use of higher moments in causal inference constitutes an exciting open research area.

Keywords:

• Direction of dependence
• causal inference
• sensitivity analysis

Article information

Conflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This work was supported in part by Grant R305A120706 from the Institute of Education Sciences (IES).

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The authors would like to thank Phillip K. Wood and the Associate Editor Stephen G. West for their comments on prior versions of this manuscript. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institution is not intended and should not be inferred.

Notes

1 We do not embark on a discussion of the conceptual differences between a confirmatory test of an a priori selected causal theory (which is the focus of DDA) and the task of learning causal structures from data alone without a priori expert knowledge - the latter, of course, carries a massive exploratory element and may be prone to incorrect causal conclusions. For a detailed discussion see, e.g., Wiedermann and von Eye (2015b) and Hernán, Hsu, and Healy (2019).

2 In particular Sungur’s (2005) approach using copulas led to another branch of DDA known as copula-based Directional Dependence Analysis (Kim & Kim, 2014; for an overview see Wiedermann, Kim, Sungur & von Eye, 2019).

3 In essence, a link between IV estimation and so-called higher order covariances of x and y, $\mathit{cov}{\left(x,y\right)}_{\mathit{ij}}=E\left[{\left(x-E\left[x\right]\right)}^i{\left(y-E\left[y\right]\right)}^j\right]$ (for details see the section “Relaxing Distributional Assumptions of the Error” in this rejoinder) exist when defining an instrument (w) as higher powers of the predictor x. In the confounded model, $y={\mathrm{\beta }}_{\mathit{yx}}x+{\mathrm{\beta }}_{\mathit{yu}}u+e_{\mathit{yx}}$ with $x={\mathrm{\beta }}_{\mathit{xu}}u+e_{\mathit{xu}}$ (u being the confounder and $e_{\mathit{yx}}$ and $e_{\mathit{xu}}$ denoting error terms), the causal effect ${\mathrm{\beta }}_{\mathit{yx}}$ can be estimated through ${\mathrm{\beta }}_{\mathit{yx}}=\mathit{cov}(w,\mathrm{}y)/\mathit{cov}(w,\mathrm{}x)$ if $\mathit{cov}\left(x,w\right)\ne$ 0 and $\mathit{cov}\left(w,\mathrm{}{e}_{\mathit{yx}}\right)$ = 0 (Durbin, 1954). Using $w=x^2$, for example, one obtains ${\mathrm{\beta }}_{\mathit{yx}}=\mathit{cov}(x^2,\mathrm{}y)/\mathit{cov}\left({\mathrm{x}}^2,x\right)=\mathit{cov}{\left(x,y\right)}_{21}/\mathit{cov}{\left(x,y\right)}_{30}$ which provides an unbiased estimate of the causal effect when u is symmetrically distributed. Evaluating properties of this higher moment estimator (e.g., quantifying biases due to violated distributional assumptions) is material for future work.

4 In the unconfounded model, $y={\mathrm{\beta }}_{\mathit{yx}}x+e_{\mathit{yx}}$, x was gamma-distributed with zero mean, unit variance, and a skewness of ${\mathrm{\gamma }}_x$ = 1.5, the error $e_{\mathit{yx}}$ followed a standard normal distribution and the causal effect was ${\mathrm{\beta }}_{\mathit{yx}}$= 0.39 representing a medium effect size (Cohen, 1988). In the confounded model, $y={\mathrm{\beta }}_{\mathit{yx}}x+{\mathrm{\beta }}_{\mathit{yu}}u+e_{\mathit{yx}}$ with $x={\mathrm{\beta }}_{\mathit{xu}}u+e_{\mathit{xu}}$, the confounder u was gamma-distributed with zero mean, unit variance, and a skewness of ${\mathrm{\gamma }}_u$ = 1.5, the two error terms ($e_{\mathit{xu}}$ and $e_{\mathit{yx}}$) were standard normally distributed, and all coefficients were set to 0.39. The sample size was n = 500 for both models.