Blending Bayesian and Classical Tools to Define Optimal Sample-Size-Dependent Significance Levels

Figures & data

Table 1 Bayes factor for all possible results in a clinical trial with two arms of size n = 8 each. Cells in boldface make up the region ${\Psi }^{*},$ and the observed Bayes factor is shown in boldface italics. See text.

	y
x	0	1	2	3	4	5	6	7	8	Sum
0	4.765	2.382	1.112	0.476	0.183	0.061	0.017	0.003	4e-04	9
1	2.382	2.541	1.906	1.173	0.611	0.267	0.093	0.024	0.003	9
2	1.112	1.906	2.052	1.710	1.166	0.653	0.29	0.093	0.017	9
3	0.476	1.173	1.710	1.866	1.633	1.161	0.653	0.267	0.061	9
4	0.183	0.611	1.166	1.633	1.814	1.633	1.166	0.611	0.183	9
5	0.061	0.267	0.653	1.161	1.633	1.866	1.710	1.173	0.476	9
6	0.017	0.093	0.29	0.653	1.166	1.710	2.052	1.906	1.112	9
7	0.003	0.024	0.093	0.267	0.611	1.173	1.906	2.541	2.382	9
8	4e-04	0.003	0.017	0.061	0.183	0.476	1.112	2.382	4.765	9
Sum	9	9	9	9	9	9	9	9	9	81

Fig. 1 Optimal averaged type-I (solid gray line), type-II (dotted line), and total (solid black line) error probabilities as functions of the number of patients n in each arm of a two-arm medical study.

Table 2 Optimal averaged error probabilities $\alpha ({\delta }^{*})$ and $\beta ({\delta }^{*})$ for comparison of two proportions for various arm sizes n₁ and n₂ in a two-arm medical study. Calculations were performed with a = b.

n1	n2	α	β	n1	n2	α	β	n1	n2	α	β
10	10	0.1639	0.4050	60	50	0.0626	0.2652	90	30	0.0707	0.2804
20	10	0.1318	0.3939	60	60	0.0591	0.2572	90	40	0.0648	0.2608
20	20	0.0995	0.3651	70	10	0.1130	0.3675	90	50	0.0575	0.2506
30	10	0.1159	0.3900	70	20	0.0865	0.3132	90	60	0.0550	0.2401
30	20	0.1045	0.3333	70	30	0.0727	0.2876	90	70	0.0529	0.2323
30	30	0.0997	0.3070	70	40	0.0645	0.2717	90	80	0.0493	0.2281
40	10	0.1250	0.3703	70	50	0.0603	0.2593	90	90	0.0468	0.2240
40	20	0.0868	0.3357	70	60	0.0575	0.2501	100	10	0.1111	0.3627
40	30	0.0850	0.3029	70	70	0.0539	0.2446	100	20	0.0818	0.3079
40	40	0.0706	0.2968	80	10	0.1130	0.3648	100	30	0.0684	0.2795
50	10	0.1126	0.3761	80	20	0.0834	0.3122	100	40	0.0617	0.2601
50	20	0.0883	0.3240	80	30	0.0704	0.2847	100	50	0.0559	0.2479
50	30	0.0767	0.2992	80	40	0.0634	0.2671	100	60	0.0538	0.2368
50	40	0.0718	0.2817	80	50	0.0603	0.2530	100	70	0.0512	0.2291
50	50	0.0667	0.2718	80	60	0.0553	0.2455	100	80	0.0483	0.2238
60	10	0.1097	0.3741	80	70	0.0531	0.2380	100	90	0.0467	0.2188
60	20	0.0860	0.3193	80	80	0.0508	0.2327	100	100	0.0449	0.2150
60	30	0.0765	0.2903	90	10	0.1131	0.3626
60	40	0.0689	0.2747	90	20	0.0810	0.3114

Fig. 2 Type-I (solid gray lines), type-II (dashed black lines), and total (dot-dashed black lines) error probabilities as functions of sample size n for tests of $\mathbf{H}:\mu =0$ on a normally distributed variable with variance 1 and unknown mean $\mu , \mathcal{N}(\mu ,1),$ with priors for the mean $\mu \sim \mathcal{N}(m,100)$ with m = 0 (left) and m = 10 (right).

Fig. 3 Bayes factor for $\mathcal{N}(0,2)$ vs. Cauchy, arising from a test of a normal variance with hypotheses $\mathbf{H}: {\sigma }^2=2$ vs. $\mathbf{A}:{\sigma }^2\ne 2$.

Fig. 4 Type-I (solid gray line), type-II (dotted line), and total (solid black line) error probabilities as functions of the sample size n for the Hardy–Weinberg equilibrium hypothesis.