Using Bayes Factors to Test Hypotheses in Developmental Research

Abstract

This article discusses the concept of Bayes factors as inferential tools that can serve as an alternative to null hypothesis significance testing in the day-to-day work of developmental researchers. A Bayes factor indicates the degree to which data observed should increase (or decrease) the credibility of one hypothesis in comparison to another. Bayes factor analyses can be used to compare many types of models but are particularly helpful when comparing a point null hypothesis to a directional or nondirectional alternative hypothesis. A key advantage of this approach is that a Bayes factor analysis makes it clear when a set of observed data is more consistent with the null hypothesis than the alternative. Bayes factor alternatives to common tests used by developmental psychologists are available in easy-to-use software. However, we note that analysis using Bayes factors is a less general approach than Bayesian estimation/modeling, and is not the right tool for every research question.

Notes

¹ A hypothesized parameter value of zero is sometimes termed the “nil” or “nil-null” hypothesis to acknowledge the possibility of a null hypothesis of a nonzero effect; in this article for convention’s sake we will use the generic term null hypothesis to refer to a hypothesis of a zero effect.

² Technically, the tail-area probability of the test statistic.

³ This said, the noninformative nature of nonsignificant results is obviously only one of many reasons for the presence of publication bias. Whenever publication is contingent on what a study finds (rather than the quality of its methods), a biased literature will result; no statistical test can form a complete solution to this problem.

⁴ Specifically, testing a point hypothesis using Bayesian estimation is conceptually difficult because it requires the prior specification to take the form of a mixture between a point mass and a continuous probability distribution, and computationally difficult because it typically requires term-based model specification in programming languages such as Stan (Carpenter et al., in press) or JAGS (Plummer, 2003).

⁵ Some readers may notice that a Bayes factor analysis is thus similar in structure to a frequentist likelihood ratio test (see Glover & Dixon, 2004). The primary practical difference is that a frequentist likelihood ratio test typically compares the likelihood of the data if the true parameter value were zero to the likelihood of the data if the true parameter value were the same as the sample estimate (thus testing an alternative hypothesis that was formed after seeing the data). In contrast, a Bayes factor analysis tests an alternative hypothesis that is specified prior to the data analysis, and that spreads prior credibility over a range of values.

⁶ Throughout the article we have placed the H₁ hypothesis in the numerator of the Bayes factor equation, and H₀ in the denominator. This is purely a matter of convention; it would be just as legitimate to express the Bayes factor with H₁ in the denominator and H₀ in the numerator.

⁷ Technically it is the tail-area probability of the test statistic—rather than the probability of the data itself—which is calculated in NHST.

⁸ The Cauchy distribution has two parameters (location and scale), the values of which are 0 and 1, respectively, in the unit Cauchy distribution. The first quartile of a given Cauchy distribution occurs at location - scale, and the third quartile falls at location + scale.