Multiple Perspectives on Inference for Two Simple Statistical Scenarios

Figures & data

Table 1 Variable names and their description.

Variable	Description
Cetirizine exposure	Whether exposed to cetirizine
Birth defect	Whether birth defects were detected
Counts	Count data for each cell

Table 2 Cetirizine exposure and birth defects.

	Birth defects
Cetirizine exposure	No	Yes	Total
No	1588	97	1685
Yes	167	14	181
Total	1755	111	1866

Fig. 1 Estimates of the odds of a birth defect when no cetirizine (control) was taken during pregnancy and when cetirizine was taken. Horizontal dashed lines and shaded regions show point estimates and standard errors. The solid line labeled “Cetirizine worst case” shows the upper bound of the one-sided CI as a function of the confidence coefficient (x-axis). The right axis shows the estimated increase in odds of a birth defect for the cetirizine group compared to the control group.

Fig. 2 Frequency representations of the number of birth defects expected under various scenarios. Top: Expected frequency of birth defects when cetirizine was not taken (control). Bottom-left: Point estimate of the expected frequency of birth defects when cetirizine is taken. Bottom-middle (bottom-right): Upper bound of a one-sided 95% (99%) CI for the expected frequency of birth defects when cetirizine was taken. Because the analysis is intended to be comparative, in the bottom panels the no-cetirizine estimate was assumed to be the truth when calculating the increase in frequency.

Fig. 3 Gronau, van Doorn, and Wagenmakers’ Bayesian analysis of the cetirizine dataset. The left panel shows the results of estimating the log odds ratio under ${\mathcal{H}}_1$ with a two-sided standard normal prior. For ease of interpretation, results are displayed in the odds ratio scale. The right panel shows the results of testing the one-sided alternative hypothesis ${\mathcal{H}}_{+}:\psi \sim {\mathcal{N}}^{+}(0,{0.4}^2)$ versus the null hypothesis ${\mathcal{H}}_0:\psi =0$. Figures inspired by JASP (jasp-stats.org).

Fig. 4 Sensitivity analysis for the Bayesian one-sided test. The Bayes factor ${\text{BF}}_{+0}$ is a function of the prior standard deviation σ. Figure inspired by JASP.

Fig. 5 Scatterplot of amygdalar activity and perceived stress in 13 patients with PTSD. Data extracted from Figure 5 in Tawakol et al. (2017), with help of Jurgen Rusch, Philips Research Eindhoven.

Table 3 Variable names and their description.

Variable	Description
Perceived stress scale	Participant score on the PSS
Amygdalar activity	Intensity of amygdalar resting state activity

Table 4 Raw data as extracted from Figure 5 in Tawakol et al. (2017), with help of Jurgen Rusch, Philips Research Eindhoven.

Perceived	Amygdalar
Stress Scale	Activity
12.0103	5.2418
32.0350	6.8601
22.0296	6.4402
20.0079	5.4620
24.0155	5.4439
24.0155	5.3349
24.0155	5.4216
26.0082	5.5176
28.0120	5.1615
21.9872	4.7114
21.9872	4.1844
20.0138	4.3079
16.0088	3.3015

Fig. 6 Confidence intervals and one-sided tests for the Pearson correlation as a function of the confidence coefficient. The vertical lines represent the confidence intervals (confidence coefficient on lower axis), and the curve represents the value that is just rejected by the one-sided test (α on the upper axis).

Fig. 7 van Doorn, Gronau, and Wagenmakers’ Bayesian analysis of the amygdala dataset. The left panel shows the result of estimating the Pearson correlation coefficient ρ under ${\mathcal{H}}_1$ with a two-sided uniform prior. The right panel shows the result of testing ${\mathcal{H}}_0:\rho =0$ versus the one-sided alternative hypothesis ${\mathcal{H}}_{+}:\rho \sim U[0,1]$. Figures from JASP.

Fig. 8 Sensitivity analysis for the Bayesian one-sided correlation test. The Bayes factor ${\text{BF}}_{+0}$ is a function of the prior width parameter $1/a$ from the stretched Beta(a,a) distribution. Figure from JASP.

Table 5 Overview of the approaches and results of the research teams.

Research team	DataSet I: Cetirizine and birth defects	DataSet II:Amygdalar Activity
Lakens and Hennig	Frequentist test of equivalence 10% equivalence region Data deemed inconclusive	Frequentist correlation, p =.047 Concerns about multiple comparisons Data deemed inconclusive
Morey and Homer	Frequentist logistic model, p =.287 Observational, so possible confounds Data deemed inconclusive	Frequentist correlation, $p=.047$ Small n means assumptions unverifiable Is the effect specific for amygdala? Data deemed inconclusive
Gronau, Van Doorn, and Wagenmakers	Default Bayes factor ${\text{BF}}_{01}=1.6$ Evidence “not worth more than a bare mention” Data deemed inconclusive	Default Bayes factor ${\text{BF}}_{10}=2$ Small n Data deemed inconclusive
Gelman	Bayesian analysis needs good prior Data likely to be inconclusive A key question is who takes the drug in the population	Bayesian analysis needs good prior Problems with generalizing to population Data likely to be inconclusive

Tawakol, A., Ishai, A., Takx, R. A. P., Figueroa, A. L., Ali, A., Kaiser, Y., Truong, Q. A., Solomon, C. J. E., Calcagno, C., Mani, V., Tang, C. Y., Mulder, W. J. M., Murrough, J. W., Hoffmann, U., Nahrendorf, M., Shin, L. M., Fayad, Z. A., and Pitman, R. K. (2017), “Relation Between Resting Amygdalar Activity and Cardiovascular Events: A Longitudinal and Cohort Study,” The Lancet, 389:834–845. DOI: 10.1016/S0140-6736(16)31714-7.

 PubMed Web of Science ®Google Scholar