A Bayes Factor for Replications of ANOVA Results

ABSTRACT

With an increasing number of replication studies performed in psychological science, the question of how to evaluate the outcome of a replication attempt deserves careful consideration. Bayesian approaches allow to incorporate uncertainty and prior information into the analysis of the replication attempt by their design. The Replication Bayes factor, introduced by Verhagen and Wagenmakers (2014), provides quantitative, relative evidence in favor or against a successful replication. In previous work by Verhagen and Wagenmakers (2014), it was limited to the case of t-tests. In this article, the Replication Bayes factor is extended to F-tests in multigroup, fixed-effect ANOVA designs. Simulations and examples are presented to facilitate the understanding and to demonstrate the usefulness of this approach. Finally, the Replication Bayes factor is compared to other Bayesian and frequentist approaches and discussed in the context of replication attempts. R code to calculate Replication Bayes factors and to reproduce the examples in the article is available at https://osf.io/jv39h/.

KEYWORDS:

• ANOVA designs
• Bayes factor
• Replications

Acknowledgments

I thank Farid Anvari, André Beauducel, Nicholas Coles, Peder Isager, Anne Scheel, Daniël Lakens, Eric-Jan Wagenmakers and one anonymous reviewer for their constructive remarks and comments, that helped to improve the clarity and structure of article.

Conflicts of Interest

None.