Conducting Bayesian-Classical Hybrid Power Analysis with R Package Hybridpower

Abstract

There are several approaches to incorporating uncertainty in power analysis. We review these approaches and highlight the Bayesian-classical hybrid approach that has been implemented in the R package hybridpower. Calculating Bayesian-classical hybrid power circumvents the problem of local optimality in which calculated power is valid if and only if the specified inputs are perfectly correct. hybridpower can compute classical and Bayesian-classical hybrid power for popular testing procedures including the t-test, correlation, simple linear regression, one-way ANOVA (with equal or unequal variances), and the sign test. Using several examples, we demonstrate features of hybridpower and illustrate how to elicit subjective priors, how to determine sample size from the Bayesian-classical approach, and how this approach is distinct from related methods. hybridpower can conduct power analysis for the classical approach, and more importantly, the novel Bayesian-classical hybrid approach that returns more realistic calculations by taking into account local optimality that the classical approach ignores. For users unfamiliar with R, we provide a limited number of RShiny applications based on hybridpower to promote the accessibility of this novel approach to power analysis. We end with a discussion on future developments in hybridpower.

Keywords:

• Power
• experimental design and analysis
• Bayesian statistics
• statistical software
• R language

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes

1 The R package hybpridpower can be downloaded from GitHub at https://github.com/JoonsukPark/RHybrid. Every code snippet in this article can be found in the file examples.R under the R folder at the package root, which is fully reproducible and contains other examples. Also, examples.R in the same folder provides example code for other statistical tests that are not described in this article.

2 There are currently four RShiny applications that implement the Bayesian-classical hybrid approach for the independent samples t-test, the sign test, the fixed-effects one-way ANOVA, and the repeated measures one-way ANOVA. See https://joonsukpark.shinyapps.io/ttest/, https://joonsukpark.shinyapps.io/signtest/, https://joonsukpark.shinyapps.io/feanova/, and https://joonsukpark.shinyapps.io/rmanova/, respectively. By default, the t-test and the sign test RShiny applications will reproduce the examples described here.

3 For clarity, we denote a power value as a number below 1 and an area under the curve as a percentage.

4 Such a store of inputs is called an instance in hybridpower and computer science in general.

5 Such a routine is called a method in hybridpower. Methods are almost identical to functions except that methods are always attached to another variable with $in R.

6 The current implementation only supports balanced designs, and we expect future implementations to include imbalanced designs.

7 The prior can be specified to follow other distributions, and we provide a more detailed discussion on choosing the form of the prior distribution in the example on the sign test.

8 Unlike the classical approach, where the effect size can be specified only in the scale of d within hybridpower, the Bayesian-classical hybrid approach can also take unstandardized effect sizes when the standard deviation of the outcome, sigma, is specified. By default, sigma (the standard deviation of the population) is set to 1 such that the scale of the mean is in Cohen’s d. Note that prior_sigma is distinct from sigma; prior_sigma pertains to uncertainty about the unknown mean whereas sigma pertains to the expected variation in the actual data. Thus, they represent different types of uncertainties.

9 The mean of a Beta distribution, $\text{Beta}(a,b),$ is given by $\frac{a}{a+b}.$ For Beta(3,5), the mean is equal to $\frac{3}{3+5}=.375.$

10 Example code and vignettes of other designs that have been implemented in hybridpower is available at https://github.com/JoonsukPark/RHybrid.