Bayesian Multivariate Logistic Regression for Superiority and Inferiority Decision-Making under Observable Treatment Heterogeneity

Figures & data

Figure D1. Bivariate superiority and inferiority spaces for the All, Any, and Compensatory ($\mathit{w}=0.50,0.50$) rules.

Figure D2. Traceplot of MCMC chains for the application of Bayesian multivariate logistic regression to the IST data.

Figure D3. Comparison of CATEs and their standard deviations per interval of blood pressure after stratified multivariate analysis (mB) and multivariate logistic regression (mLR). Each interval reflects one standard deviation.

Table D1. Example of required sample sizes (n) for analysis with correlated data and anticipated probabilities to conclude superiority when sample size computations use the true correlation ($p_T$) vs. assume uncorrelated dependent variables ($p_U$) under four different data-generating mechanisms (DGMs).

	$\rho <0$			$\rho =0$			$\rho >0$
DGM	n	$p_T$	$p_U$	n	$p_T$	$p_U$	n	$p_T$	$p_U$
	All rule
S2	307	0.801	0.801	307	0.801	0.801	307	0.801	0.801
L2	77	0.801	0.801	77	0.803	0.803	76	0.803	0.808
S3	307	0.800	0.800	307	0.801	0.801	307	0.801	0.801
L3	79	0.801	0.800	79	0.804	0.804	77	0.803	0.812
	Any rule
S2	76	0.801	0.825	81	0.803	0.803	85	0.802	0.783
L2	27	0.811	0.862	31	0.807	0.807	35	0.801	0.756
S3	51	0.807	0.850	57	0.804	0.804	64	0.805	0.760
L3	18	0.821	0.918	23	0.809	0.809	29	0.804	0.714
	Compensatory rule
S2	53	0.798	0.845	61	0.801	0.801	68	0.799	0.760
L2	18	0.807	0.884	23	0.794	0.794	29	0.799	0.714
S3	24	0.796	0.923	37	0.804	0.804	49	0.802	0.699
L3	5	0.826	0.996	14	0.798	0.798	24	0.807	0.608

Table D2. Parameters of average treatment effects (ATEs) in the trial and conditional average treatment effects (CATEs) in a subpopulation for two outcome variables.

		ATE			CATE
ES		$({\delta }_1,{\delta }_2)$	$\delta (\mathbf{w})$	${\rho }_{{\theta }^k,{\theta }^l}$	$({\delta }_1,{\delta }_2)$	$\delta (\mathbf{w})$	${\rho }_{{\theta }^k,{\theta }^l}$
1.1	D	(0.000, 0.000)	0.000	−0.160	(0.000, 0.000)	0.000	−0.200
				0.030			0.000
				0.220			0.200
1.2	C	(0.000, 0.000)	0.000	−0.163	(0.000, 0.000)	0.000	−0.207
				0.028			0.002
				0.219			0.208
2.1	D	(0.000, 0.000)	0.000	−0.154	(0.250, 0.150)	0.225	−0.200
				0.037			0.000
				0.229			0.200
2.2	C	(0.000, 0.000)	0.000	−0.157	(0.116, 0.069)	0.104	−0.206
				0.036			0.003
				0.228			0.207
3.1	D	(0.100, 0.000)	0.075	−0.152	(0.300, 0.200)	0.275	−0.200
				0.040			0.000
				0.232			0.200
3.2	C	(0.100, 0.000)	0.075	−0.155	(0.196, 0.093)	0.170	−0.205
				0.038			0.003
				0.231			0.206
4.1	D	(0.350, 0.000)	0.263	−0.197	(0.200, 0.000)	0.150	−0.200
				0.000			0.000
				0.197			0.200
4.2	C	(0.350, 0.000)	0.263	−0.197	(0.288, 0.000)	0.216	−0.202
				0.000			0.000
				0.197			0.202

Es = Effect size, D = Discrete covariate, C = Continuous covariate.

Table D3. Required sample sizes to evaluate the average treatment effect (ATE) and conditional treatment effect (CATE) for two outcome variables.

		All			Any			Compensatory
ES	${\rho }_{k^{\theta },l^{\theta }}$	ATE	CATE	Sub	ATE	CATE	Sub	ATE	CATE	Sub
1.1	$<0$	–	–	500	–	–	500	–	–	500
	$\approx 0$	–	–	500	–	–	500	–	–	500
	$>0$	–	–	500	–	–	500	–	–	500
1.2	$<0$	–	–	342	–	–	342	–	–	342
	$\approx 0$	–	–	342	–	–	342	–	–	342
	$>0$	–	–	342	–	–	342	–	–	342
2.1	$<0$	–	136	500	–	45	500	–	32	500
	$\approx 0$	–	136	500	–	48	500	–	36	500
	$>0$	–	136	500	–	51	500	–	40	500
2.2	$<0$	–	658	$\mathbf{342}$	–	215	342	–	154	342
	$\approx 0$	–	649	$\mathbf{342}$	–	229	342	–	175	342
	$>0$	–	644	$\mathbf{342}$	–	245	342	–	196	342
3.1	$<0$	–	77	500	381	29	191	309	21	155
	$\approx 0$	–	77	500	385	31	193	349	23	175
	$>0$	–	76	500	387	33	194	388	26	194
3.2	$<0$	–	358	$\mathbf{342}$	379	81	130	307	56	105
	$\approx 0$	–	358	$\mathbf{342}$	383	86	131	347	65	119
	$>0$	–	356	$\mathbf{342}$	386	91	132	386	73	132
4.1	$<0$	–	–	500	28	93	$\mathbf{14}$	22	73	$\mathbf{11}$
	$\approx 0$	–	–	500	28	93	$\mathbf{14}$	25	83	$\mathbf{13}$
	$>0$	–	–	500	28	94	$\mathbf{14}$	28	93	$\mathbf{14}$
4.2	$<0$	–	–	342	28	43	$\mathbf{10}$	22	34	$\mathbf{8}$
	$\approx 0$	–	–	342	28	44	$\mathbf{10}$	25	39	$\mathbf{9}$
	$>0$	–	–	342	28	44	$\mathbf{10}$	28	43	$\mathbf{10}$

Sub = expected size of subsample.

Bold-faced subsamples are smaller than required for estimation of the CATE.

Table D4. Bias in average treatment differences of effect size (ES) 4.1 and 4.2 by decision rule.

		All rule
		uLR	mB	mLR
ES	${\rho }_{{\theta }^k,{\theta }^l}$	(${\delta }^1,{\delta }^2$)	(${\delta }^1,{\delta }^2$)	(${\delta }^1,{\delta }^2$)
4.1	$<0$	(0.000, 0.000)	(−0.002, 0.001)	(0.000,−0.002)
	$\approx 0$	(0.000, 0.000)	(0.000, 0.000)	(−0.001,−0.001)
	$>0$	(−0.001, 0.000)	(−0.001,−0.001)	(−0.001, 0.000)
4.2	$<0$	(−0.002, 0.000)	(0.002,−0.001)	(0.000,−0.002)
	$\approx 0$	(−0.001,−0.001)	(−0.001, 0.001)	(−0.002,−0.001)
	$>0$	(−0.001,−0.001)	(0.001,−0.001)	(−0.001, 0.001)
		Any rule
		uLR	mB	mLR
ES	${\rho }_{{\theta }^k,{\theta }^l}$	(${\delta }^1,{\delta }^2$)	(${\delta }^1,{\delta }^2$)	(${\delta }^1,{\delta }^2$)
4.1	$<0$	(−0.013, 0.001)	(−0.001, 0.005)	(−0.024,−0.011)
	$\approx 0$	(−0.009,−0.002)	(0.001, 0.001)	(−0.023,−0.016)
	$>0$	(−0.006, 0.001)	(0.002, 0.004)	(−0.028,−0.007)
4.2	$<0$	(−0.018,−0.009)	(−0.005, 0.003)	(−0.031,−0.019)
	$\approx 0$	(−0.014, 0.003)	(−0.002,−0.001)	(−0.032,−0.011)
	$>0$	(−0.018,−0.002)	(−0.001, 0.005)	(−0.030,−0.008)
		Compensatory rule
		uLR	mB	mLR
ES	${\rho }_{{\theta }^k,{\theta }^l}$	$\delta (\mathbf{w})$	$\delta (\mathbf{w})$	$\delta (\mathbf{w})$
4.1	$<0$	−0.006	0.000	−0.030
	$\approx 0$	−0.012	0.000	−0.019
	$>0$	−0.004	−0.006	−0.015
4.2	$<0$	−0.018	−0.003	−0.040
	$\approx 0$	−0.017	−0.003	−0.029
	$>0$	−0.013	0.003	−0.024

uLR = univariate logistic regression.

mB = multivariate Bernoulli.

mLR = multivariate logistic regression.

Table D5. Proportions of superiority decisions for ATEs with two outcome variables by data-generating mechanism, correlation, and decision rule.

	$\rho <0$			$\rho =0$			$\rho >0$
ES	uLR	mB	mLR	uLR	mB	mLR	uLR	mB	mLR
Rule = All
1.1	0.000	0.004	0.000	0.000	0.005	0.001	0.004	0.005	0.007
1.2	0.003	0.002	0.001	0.000	0.005	0.002	0.006	0.006	0.005
2.1	0.000	0.001	0.003	0.002	0.004	0.004	0.006	0.005	0.008
2.2	0.002	0.003	0.000	0.007	0.002	0.005	0.003	0.005	0.010
3.1	0.064	0.051	0.046	0.066	0.046	0.056	0.054	0.043	0.046
3.2	0.051	0.048	0.055	0.050	0.057	0.050	0.049	0.052	0.061
4.1	0.059	0.051	0.042	0.059	0.044	0.044	0.052	0.046	0.053
4.2	0.051	0.058	0.045	0.045	0.041	0.051	0.044	0.049	0.053
Rule = Any
1.1	0.052	0.046	0.054	0.060	0.064	0.060	0.059	0.047	0.050
1.2	0.054	0.055	0.043	0.035	0.042	0.050	0.038	0.053	0.049
2.1	0.063	0.053	0.059	0.055	0.044	0.045	0.052	0.049	0.049
2.2	0.059	0.055	0.062	0.059	0.045	0.062	0.046	0.048	0.060
3.1	$\mathbf{0.807}$	$\mathbf{0.802}$	$\mathbf{0.789}$	$\mathbf{0.810}$	$\mathbf{0.812}$	$\mathbf{0.806}$	$\mathbf{0.796}$	$\mathbf{0.787}$	$\mathbf{0.791}$
3.2	$\mathbf{0.814}$	$\mathbf{0.790}$	$\mathbf{0.807}$	$\mathbf{0.819}$	$\mathbf{0.811}$	$\mathbf{0.791}$	$\mathbf{0.811}$	$\mathbf{0.803}$	$\mathbf{0.815}$
4.1	$\mathbf{0.804}$	$\mathbf{0.756}$	$\mathbf{0.781}$	$\mathbf{0.816}$	$\mathbf{0.775}$	$\mathbf{0.787}$	$\mathbf{0.808}$	$\mathbf{0.780}$	$\mathbf{0.777}$
4.2	$\mathbf{0.790}$	$\mathbf{0.749}$	$\mathbf{0.793}$	$\mathbf{0.806}$	$\mathbf{0.774}$	$\mathbf{0.770}$	$\mathbf{0.781}$	$\mathbf{0.754}$	$\mathbf{0.785}$
Rule = Compensatory
1.1	0.049	0.056	0.054	0.059	0.069	0.050	0.076	0.047	0.048
1.2	0.045	0.041	0.056	0.045	0.040	0.051	0.063	0.047	0.055
2.1	0.053	0.040	0.053	0.069	0.054	0.048	0.076	0.051	0.053
2.2	0.051	0.048	0.054	0.059	0.040	0.061	0.057	0.048	0.058
3.1	$\mathbf{0.757}$	$\mathbf{0.821}$	$\mathbf{0.813}$	$\mathbf{0.824}$	$\mathbf{0.802}$	$\mathbf{0.815}$	$\mathbf{0.815}$	$\mathbf{0.801}$	$\mathbf{0.794}$
3.2	$\mathbf{0.779}$	$\mathbf{0.804}$	$\mathbf{0.838}$	$\mathbf{0.802}$	$\mathbf{0.811}$	$\mathbf{0.804}$	$\mathbf{0.836}$	$\mathbf{0.801}$	$\mathbf{0.815}$
4.1	$\mathbf{0.794}$	$\mathbf{0.795}$	$\mathbf{0.774}$	$\mathbf{0.805}$	$\mathbf{0.799}$	$\mathbf{0.810}$	$\mathbf{0.858}$	$\mathbf{0.781}$	$\mathbf{0.790}$
4.2	$\mathbf{0.759}$	$\mathbf{0.786}$	$\mathbf{0.771}$	$\mathbf{0.820}$	$\mathbf{0.792}$	$\mathbf{0.806}$	$\mathbf{0.815}$	$\mathbf{0.798}$	$\mathbf{0.792}$

uLR = Univariate logistic regression.

mB = Multivariate Bernoulli.

mLR = Multivariate logistic regression.

Bold-faced entries have effect sizes that should lead to a superiority conclusion.

Table D6. Proportions of superiority decisions for CATEs with two outcome variables by data-generating mechanism, correlation, and decision rule.

	$\rho <0$			$\rho =0$			$\rho >0$
ES	mB	mLR-S	mLR-V	mB	mLR-S	mLR-V	mB	mLR-S	mLR-V
Rule = All
1.1	0.002	–	0.000	0.006	–	0.001	0.009	–	0.004
1.2	0.000	0.000	0.001	0.004	0.002	0.004	0.007	0.003	0.004
2.1	$\mathbf{0.999}$	–	$\mathbf{0.997}$	$\mathbf{0.999}$	–	$\mathbf{0.998}$	$\mathbf{1.000}$	–	$\mathbf{0.999}$
2.2	$\mathbf{0.484}$	$\mathbf{0.873}$	$\mathbf{0.998}$	$\mathbf{0.537}$	$\mathbf{0.854}$	$\mathbf{1.000}$	$\mathbf{0.529}$	$\mathbf{0.880}$	$\mathbf{1.000}$
3.1	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$
3.2	$\mathbf{0.790}$	$\mathbf{0.972}$	$\mathbf{1.000}$	$\mathbf{0.801}$	$\mathbf{0.979}$	$\mathbf{1.000}$	$\mathbf{0.804}$	$\mathbf{0.982}$	$\mathbf{1.000}$
4.1	0.050	–	0.040	0.042	–	0.036	0.045	–	0.048
4.2	0.051	0.045	0.054	0.052	0.053	0.059	0.046	0.056	0.060
Rule = Any
1.1	0.054	–	0.050	0.064	–	0.039	0.051	–	0.052
1.2	0.053	0.038	0.054	0.057	0.055	0.056	0.063	0.048	0.048
2.1	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$
2.2	$\mathbf{0.933}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{0.913}$	$\mathbf{0.999}$	$\mathbf{1.000}$	$\mathbf{0.904}$	$\mathbf{0.999}$	$\mathbf{1.000}$
3.1	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$
3.2	$\mathbf{0.932}$	$\mathbf{0.999}$	$\mathbf{1.000}$	$\mathbf{0.939}$	$\mathbf{0.998}$	$\mathbf{1.000}$	$\mathbf{0.899}$	$\mathbf{0.999}$	$\mathbf{1.000}$
4.1	$\mathbf{0.251}$	–	$\mathbf{0.266}$	$\mathbf{0.251}$	–	$\mathbf{0.242}$	$\mathbf{0.233}$	–	$\mathbf{0.230}$
4.2	$\mathbf{0.336}$	$\mathbf{0.508}$	$\mathbf{0.181}$	$\mathbf{0.305}$	$\mathbf{0.522}$	$\mathbf{0.183}$	$\mathbf{0.308}$	$\mathbf{0.512}$	$\mathbf{0.174}$
Rule = Compensatory
1.1	0.061	–	0.047	0.076	–	0.033	0.048	–	0.039
1.2	0.040	0.040	0.043	0.062	0.057	0.056	0.057	0.046	0.048
2.1	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$
2.2	$\mathbf{0.980}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{0.969}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{0.945}$	$\mathbf{0.999}$	$\mathbf{1.000}$
3.1	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$	$\mathbf{1.000}$	–	$\mathbf{1.000}$
3.2	$\mathbf{0.951}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{0.953}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{0.945}$	$\mathbf{1.000}$	$\mathbf{1.000}$
4.1	$\mathbf{0.283}$	–	$\mathbf{0.326}$	$\mathbf{0.292}$	–	$\mathbf{0.319}$	$\mathbf{0.287}$	–	$\mathbf{0.316}$
4.2	$\mathbf{0.390}$	$\mathbf{0.504}$	$\mathbf{0.190}$	$\mathbf{0.354}$	$\mathbf{0.534}$	$\mathbf{0.183}$	$\mathbf{0.359}$	$\mathbf{0.537}$	$\mathbf{0.232}$

mB = Multivariate Bernoulli.

mLR-S = Multivariate logistic regression –sample.

mLR-V = Multivariate logistic regression –value.

Bold-faced entries have effect sizes that should lead to a superiority conclusion.

Table D7. Average and conditional average treatment effects (ATE and CATE respectively) and their posterior probabilities (pp) in the IST data, by interval of blood pressure (Bp). Superiority or inferiority was concluded when $>$ or $<$ respectively.

Method	$\mathit{\delta }(\text{Bp})$	pp	Any	All	$\delta (\mathit{w},\text{Bp})$	pp	Comp
ATE ($-\infty <\text{Bp}<\infty$)			$n_{\text{H + A}}=1859,$ $n_{\mathrm{A}}=3798$
mB	(0.005, −0.015)	(0.859, 0.151)	–	–	−0.010	0.182	–
mLR	(0.004, −0.014)	(0.825, 0.152)	–	–	−0.010	0.178	–
CATE ($-\infty <\text{Bp}<-1\text{SD}$)			$n_{\text{H + A}}=316,$ $n_{\mathrm{A}}=620$
mB	(−0.001, 0.066)	(0.459, 0.972)	–	–	0.049	0.970	–
mLR	(0.012, 0.043)	(0.932, 0.963)	–	–	0.035	0.972	–
CATE ($+1\text{SD}<\text{Bp}<\infty$)			$n_{\text{H + A}}=290,$ $n_{\mathrm{A}}=646$
mB	(−0.009, −0.052)	(0.214, 0.070)	–	–	−0.041	0.063	–
mLR	(−0.003, −0.081)	(0.330, 0.001)	$>$	–	−0.062	0.001	$>$

mB = Multivariate Bernoulli analysis.

mLR = Multivariate logistic regression.

Table D8. Conditional average treatment effects in the IST data, by value of blood pressure (Bp). Superiority or inferiority was concluded when $>$ or $<$ respectively.

Value	$\mathit{\delta }(\text{Bp})$	pp	Any	All	$\delta (\mathit{w},\text{Bp})$	pp	Comp
−3 SD	(0.029, 0.110)	(0.922, 0.994)	$<$	–	0.090	0.996	$<$
−2 SD	(0.017, 0.068)	(0.930, 0.985)	–	–	0.055	0.989	$<$
−1 SD	(0.009, 0.026)	(0.927, 0.908)	–	–	0.022	0.929	–
+1 SD	(−0.001, −0.056)	(0.421, 0.002)	$>$	–	−0.042	0.002	$>$
+2 SD	(−0.004, −0.097)	(0.294, 0.001)	$>$	–	−0.074	0.001	$>$
+3 SD	(−0.007, −0.137)	(0.263, 0.001)	$>$	–	−0.104	0.001	$>$

Table D9. Example of means of the prior distribution of regression coefficients.

	$q=1$	$q=2$	$q=3$	$q=4$
${\beta }_0^q$	−0.000	0.766	0.766	0.000
${\beta }_1^q$	0.000	0.000	0.000	0.000
${\beta }_2^q$	1.902	0.781	1.121	0.000
${\beta }_3^q$	−3.804	−1.562	−2.241	0.000

Table D10. Parameters of average treatment effects (ATEs) in the trial and conditional average treatment effects (CATEs) in a subpopulation for tree outcome variables.

		ATE			CATE
ES		$({\delta }_1,{\delta }_2,{\delta }_3)$	$\delta (\mathbf{w})$	${\rho }_{{\theta }^k,{\theta }^l}$	$({\delta }_1,{\delta }_2,{\delta }_3)$	$\delta (\mathbf{w})$	${\rho }_{{\theta }^k,{\theta }^l}$
1.1	D	(0.000, 0.000, 0.000)	0.000	−0.160	(0.000, 0.000, 0.000)	0.000	−0.200
				0.030			0.000
				0.220			0.200
3.1	D	(0.100, 0.000, 0.100)	0.075	−0.152	(0.300, 0.200, 0.300)	0.275	−0.200
				0.040			0.000
				0.232			0.200

ES = Effect size, D = Discrete covariate.

Table D11. Required sample sizes to evaluate the average treatment effect (ATE) and conditional treatment effect (CATE) for three outcome variables.

		All			Any			Compensatory
ES	${\rho }_{k^{\theta },l^{\theta }}$	ATE	CATE	Sub	ATE	CATE	Sub	ATE	CATE	Sub
1.1	$<0$	–	–	500	–	–	500	–	–	500
	$\approx 0$	–	–	500	–	–	500	–	–	500
	$>0$	–	–	500	–	–	500	–	–	500
3.1	$<0$	–	79	500	234	20	117	153	9	77
	$\approx 0$	–	79	500	255	23	128	218	14	109
	$>0$	–	78	500	276	26	138	284	19	142

Sub = expected size of subsample.

Table D12. Proportions of superiority decisions for three outcome variables by data-generating mechanism, correlation, and decision rule.

		$\rho <0$		$\rho =0$		$\rho >0$
ES	Type	mB	mLR	mB	mLR	mB	mLR
Rule = All
1.1	ATE	0.000	0.000	0.000	0.001	0.004	0.002
3.1	ATE	0.045	0.045	0.058	0.048	0.044	0.058
1.1	CATE	0.000	0.000	0.000	0.000	0.001	0.000
3.1	CATE	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$
Rule = Any
1.1	ATE	0.049	0.056	0.045	0.050	0.046	0.056
3.1	ATE	$\mathbf{0.814}$	$\mathbf{0.822}$	$\mathbf{0.796}$	$\mathbf{0.775}$	$\mathbf{0.815}$	$\mathbf{0.775}$
1.1	CATE	0.047	0.050	0.042	0.037	0.063	0.032
3.1	CATE	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$
Rule = Compensatory
1.1	ATE	0.048	0.068	0.050	0.052	0.052	0.063
3.1	ATE	$\mathbf{0.781}$	$\mathbf{0.826}$	$\mathbf{0.788}$	$\mathbf{0.757}$	$\mathbf{0.787}$	$\mathbf{0.776}$
1.1	CATE	0.051	0.043	0.056	0.029	0.053	0.035
3.1	CATE	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$	$\mathbf{1.000}$

mB = Multivariate Bernoulli.

mLR = Multivariate logistic regression.

Bold-faced entries should lead to a superiority conclusion.