Bayesian evaluation of informative hypotheses in SEM using Mplus: A black bear story

Abstract

Half in jest we use a story about a black bear to illustrate that there are some discrepancies between the formal use of the p-value and the way it is often used in practice. We argue that more can be learned from data by evaluating informative hypotheses, than by testing the traditional null hypothesis. All criticisms of classical null hypothesis testing aside, the best argument for evaluating informative hypotheses is that many researchers want to evaluate their expectations directly, but have been unable to do so because the statistical tools were not yet available. It will be shown that a Bayesian model selection procedure can be used to evaluate informative hypotheses in structural equation models using the software Mplus. In the current paper we introduce the methodology using a real-life example taken from the field of developmental psychology about depressive symptoms in adolescence and provide a step-by-step description so that the procedure becomes more comprehensible for applied researchers. As this paper illustrates, this methodology is ready to be used by any researcher within the social sciences.

Keywords:

• Structural equation modelling
• Bayes factor
• Informative hypothesis
• Order restricted inference
• Mplus
• Depression

View correction statement:

Corrigendum

Notes

¹Although an inequality constraint between two parameters can be tested using one-sided testing or contrast testing, it becomes difficult or even impossible to test more complicated hypotheses between multiple parameters. See Van de Schoot, Hoijtink, Mulder et al (2011) for a discussion about the differences between one-sided testing, contrast testing and informative hypothesis testing.

²As pointed out by one of the reviewers, not all methodologists would agree on the choice of our null hypothesis. The black bear decision problem is based on just two states of nature: Either there is a bear or there is no bear. Thus, both hypotheses are so-called simple hypotheses and the traditional approach rests on the idea that one of the two possible type I or type II errors is more important (e.g., Bickel & Doksum, 2006, p. 216). In the black bear story, the more important error undoubtedly would be to continue hiking while there is a bear. Thus, contrary to what we claim, the appropriate null hypothesis in this example should be “There is a bear”. In choosing a very small significance level, the action “continue hiking” would only be taken if there are observations very inconsistent with this null hypothesis. However, we choose our null hypothesis to resemble the daily practice of analysing null hypotheses where generally the null hypothesis tested is “nothing is going on”, that is parameters are equal to zero or to other parameters. Therefore, our black bear hypothesis also states “nothing is going on, there is no black bear hiding”.

³Note that, as was shown by Van de Schoot et al. (2012), the methodology described here is limited to a maximum of six connected parameters. For example, the hypothesis β₁ > β₂ > β₃ > β₄ > β₅ > β₆ > β₇ is not allowed, but the hypothesis β₁ > β₂ > β₃ > β₄ > β₅ > β₆ & β₇ > β₈ > β₉ is allowed. The latter hypothesis consists of two sets of ordered parameters and the largest set does not exceed the limit of six parameters.

⁴With simple order restricted hypotheses we mean only restrictions of the form β₁ > β₂, thereby excluding equality constraints and functions of parameters, for example (β₁ – β₂) > (β₃ – β₄), or (2*β₁) > β₂.

⁵Note that, as was shown by Van de Schoot et al. (2012; see also Hoijtink, 2012), a minimum number of iterations has to be specified, see Table 1. In Mplus this can be done using FBITERATIONS = … . As said earlier, only six parameters can be used with this methodology. This has to do with the huge number of iterations needed to get stable results for a hypothesis with seven or more connected parameters.

Additional information

Notes on contributors

Rens van de Schoot

The first author was supported by a grant from the Netherlands organization for scientific research: NWO-VENI-451-11-008