Beyond the p-value: Bayesian Statistics and Causation

ABSTRACT

Statistical paradigms limit the perspective and tools social work researchers use to study the world and answer questions impacting people and policy. Currently, quantitative social work researchers overwhelmingly rely on the frequentist paradigm of statistics. This paper discusses foundational differences between the frequentist and Bayesian statistical paradigms, describes basic concepts of Bayesian analysis, compares Bayesian and frequentist statistical analysis for a sample social work problem, and introduces two types of causal analyses built on Bayesian statistical thinking: counterfactual causality, and causality based on work by computer scientist Judea Pearl. Implications for social work research are discussed.

KEYWORDS:

• Quantitative methods
• causality
• Bayesian statistics
• social work

Acknowledgments

The authors thank Dr. Adam Sales for feedback on the paper during revision, and the “Causality group” brought together by Dr. Michael Marder which inspired this paper.

Notes

1. This is a non-exhaustive list of strengths and benefits of Bayesian statistics. For further discussion within the context of medical device clinical trials, see the FDA (2010) guidance document (especially section 2.6, pp. 7–8), which discusses how Bayesian statistics handles issues related to missing data, sample size, and optional stopping.

2. Within Bayesian statistics, one method for updating the prior probability of a hypothesis is through use of a Bayes factor (BF). Accordingly, BFs are said to represent a measure of evidence (see Kass & Raftery, 1995 for an overview), which would seem to contradict Royall’s framework outlined here, which warrants comment. First, statistical tools (e.g., p-values, confidence intervals, Bayes factors) can be used in multiple ways. Within the Neyman-Pearson school of frequentist inference, p-values are used in a decision-making procedure (i.e. acceptance-rejection of the null hypothesis according to some pre-specified alpha-level). According to the Fisherian school, the p-value may serve as a measure of the strength of evidence. The interpretation of such tools then, lies in part within the aims of the individual researcher. The second point is procedural. BFs may be calculated for some data, full-stop (representing a measure of evidence, not unlike the likelihood ratio); or they may be utilized as part of a full Bayesian analysis, to arrive at updated posterior probabilities. In the former scenario, BFs may be utilized as a measure of evidence, while in the latter they are more of a means by which one completes a Bayesian analysis.

3. In contrast, with continuous sampling, the Bayesian analyst’s prior probabilities will typically “wash out” over time (Sober, 2008, p. 25).

4. From an ethical point of view, it is better for members of the control group to receive the best available intervention, other than the one being studied.