Bayesian variable selection via a benchmark in normal linear models

Figures & data

Figure 1. Posterior plots with the prostate cancer data.

Table 1. Values of $d_i$ and BIC$\left(m\right)$ in prostate cancer example.

	lcavol	lweight	svi	age	lbph	gleason	lcp	pgg45
$d_i$	1.00	0.98	0.93	0.80	0.71	0.61	0.56	0.54
BIC$\left(m\right)$	−0.38	−0.47	−0.54	−0.49	−0.46	−0.41	−0.33	−0.26

Table 2. Results for simulation Study 1.

n	p	$\frac{\parallel {\text{β}}_0\parallel }{{\sigma }_0}$	Method	Model size	Sensitivity	Specificity	PMSE	Rate
50	10	3	Proposed	2.973 (0.501)	0.955 (0.117)	0.984 (0.051)	8.022 (1.831)	0.784
			Lasso	6.032 (2.114)	0.998 (0.028)	0.566 (0.301)	8.306 (1.797)	0.104
			B-lasso	10.00 (0.000)	1.000 (0.000)	0.000 (0.000)	8.303 (1.771)	0.000
		5	Proposed	3.096 (0.356)	0.995 (0.041)	0.984 (0.048)	4.665 (0.966)	0.884
			Lasso	6.015 (2.090)	1.000 (0.011)	0.569 (0.299)	4.955 (1.044)	0.102
			B-lasso	10.00 (0.000)	1.000 (0.000)	0.000 (0.000)	4.947 (1.017)	0.000
		10	Proposed	3.123 (0.366)	1.000 (0.000)	0.982 (0.052)	2.341 (0.481)	0.890
			Lasso	5.939 (2.095)	1.000 (0.000)	0.580 (0.299)	2.502 (0.531)	0.110
			B-lasso	10.00 (0.000)	1.000 (0.000)	0.000 (0.000)	2.503 (0.515)	0.000
50	100	3	Proposed	2.891 (1.072)	0.819 (0.227)	0.996 (0.007)	9.379 (2.731)	0.327
			Lasso	17.73 (10.80)	0.990 (0.058)	0.848 (0.111)	9.758 (2.405)	0.004
			B-lasso	100.0 (0.000)	1.000 (0.000)	0.000 (0.000)	15.74 (2.935)	0.000
		5	Proposed	3.272 (0.950)	0.936 (0.142)	0.995 (0.008)	5.231 (1.665)	0.538
			Lasso	17.60 (10.50)	0.999 (0.021)	0.849 (0.108)	5.771 (1.440)	0.003
			B-lasso	100.0 (0.000)	1.000 (0.000)	0.000 (0.000)	11.86 (2.083)	0.000
		10	Proposed	3.313 (0.681)	0.996 (0.038)	0.997 (0.007)	2.374 (0.542)	0.761
			Lasso	18.58 (10.66)	1.000 (0.000)	0.839 (0.110)	2.925 (0.716)	0.002
			B-lasso	100.0 (0.000)	1.000 (0.000)	0.000 (0.000)	8.850 (1.322)	0.000
100	500	3	Proposed	3.150 (0.672)	0.970 (0.096)	1.000 (0.001)	7.588 (1.171)	0.750
			Lasso	39.08 (28.90)	1.000 (0.000)	0.928 (0.058)	9.343 (2.052)	0.000
			B-lasso	500.0 (0.000)	1.000 (0.000)	0.000 (0.000)	15.93 (1.957)	0.000
		4	Proposed	3.230 (0.601)	0.997 (0.033)	1.000 (0.001)	5.535 (0.763)	0.820
			Lasso	40.73 (28.65)	1.000 (0.000)	0.924 (0.058)	7.091 (1.580)	0.000
			B-lasso	500.0 (0.000)	1.000 (0.000)	0.000 (0.000)	13.68 (1.615)	0.000
		5	Proposed	3.180 (0.500)	1.000 (0.000)	1.000 (0.001)	4.416 (0.611)	0.860
			Lasso	42.16 (28.65)	1.000 (0.000)	0.921 (0.058)	5.693 (1.261)	0.000
			B-lasso	500.0 (0.000)	1.000 (0.000)	0.000 (0.000)	12.29 (1.400)	0.000
		6	Proposed	3.150 (0.479)	1.000 (0.000)	1.000 (0.001)	3.676 (0.507)	0.890
			Lasso	41.87 (28.24)	1.000 (0.000)	0.922 (0.057)	4.737 (1.065)	0.000
			B-lasso	500.0 (0.000)	1.000 (0.000)	0.000 (0.000)	11.34 (1.257)	0.000

Notes: Numbers in parentheses are standard deviations; the true model size is 3; model size, sensitivity, specificity, and PMSE are defined in (8(8) $\begin{array}{rl}\text{model size}& =\text{number of selected covariates }\end{array}$(8) )–(11(11) $\begin{array}{rl}\mathrm{P}\mathrm{M}\mathrm{S}\mathrm{E}& =\frac{\parallel {\mathit{y}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}-\overline{y}-\mathit{X}\stackrel{\mathbf{ˆ}}{\text{β}}{\parallel }^2}{n}\end{array}$(11) ); rate = rate of selecting exactly three important covariates.

Table 3. Results for simulation Study 2.

		Group selection				Variable selection
Case	Method	Size	Sens	Spec	Rate	Size	Sens	Spec	Rate	PMSE
I	Proposed	3.14	1.00	0.95	0.89	6.29	1.00	1.00	0.79	3.57
		(0.46)	(0.00)	(0.15)		(0.63)	(0.00)	(0.01)		(0.52)
	glasso	3.85	1.00	0.72	0.38	46.93	1.00	0.51	0.00	5.42
		(0.89)	(0.00)	(0.30)		(17.90)	(0.00)	(0.21)		(0.89)
	gbridge	3.59	1.00	0.80	0.64	22.46	1.00	0.80	0.00	4.15
		(0.94)	(0.00)	(0.31)		(21.53)	(0.00)	(0.26)		(0.91)
	sglasso	5.50	1.00	0.17	0.04	14.66	1.00	0.90	0.01	4.56
		(0.78)	(0.00)	(0.26)		(4.97)	(0.01)	(0.06)		(0.76)
II	Proposed	1.96	0.96	1.00	0.89	3.06	0.70	1.00	0.24	13.95
		(0.32)	(0.14)	(0.03)		(1.10)	(0.24)	(0.01)		(4.49)
	glasso	3.39	1.00	0.83	0.04	33.9	1.00	0.69	0.00	17.15
		(0.64)	(0.00)	(0.08)		(6.39)	(0.00)	(0.07)		(3.84)
	gbridge	4.80	1.00	0.65	0.12	26.1	0.98	0.77	0.00	16.25
		(1.47)	(0.00)	(0.18)		(8.57)	(0.07)	(0.09)		(3.66)
	sglasso	3.89	1.00	0.76	0.35	8.14	0.90	0.95	0.05	14.13
		(2.18)	(0.00)	(0.27)		(4.77)	(0.15)	(0.05)		(3.16)

Notes: Numbers in parentheses are standard deviations; under variable selection, size = model size, sen = sensitivity, spec = specificity, defined in (8(8) $\begin{array}{rl}\text{model size}& =\text{number of selected covariates }\end{array}$(8) )–(10(10) $\begin{array}{rl}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}& =\frac{\begin{array}{c}\text{number of removed unimportant}\\ \text{covariates}\end{array}}{p-3}\end{array}$(10) ); under group selection, size, sen, spec are defined by (8(8) $\begin{array}{rl}\text{model size}& =\text{number of selected covariates }\end{array}$(8) )–(10(10) $\begin{array}{rl}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}& =\frac{\begin{array}{c}\text{number of removed unimportant}\\ \text{covariates}\end{array}}{p-3}\end{array}$(10) ) with covariates replaced by groups; PMSE is defined by (11(11) $\begin{array}{rl}\mathrm{P}\mathrm{M}\mathrm{S}\mathrm{E}& =\frac{\parallel {\mathit{y}}_{\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}}-\overline{y}-\mathit{X}\stackrel{\mathbf{ˆ}}{\text{β}}{\parallel }^2}{n}\end{array}$(11) ); rate = rate of selecting exactly the true numbers of important groups and covariates.

Table 4. Results based on 100 random splits for the prostate cancer example.

	Selection rate for each covariate
Method	lcavol	lweight	svi	age	lbph	gleason	lcp	pgg45
Proposed	1.00	0.87	0.91	0.00	0.08	0.00	0.00	0.00
Lasso	1.00	0.99	1.00	0.51	0.88	0.46	0.23	0.75
B-lasso	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
		Average model size				PMSE
Proposed		2.860 (0.377)				0.594 (0.189)
Lasso		5.820 (1.553)				0.600 (0.216)
B-lasso		8.000 (0.000)				0.580 (0.188)

Note: Numbers in parentheses are standard deviations.

Table 5. Results based on 100 random splits for the CCT8 example.

	Selection rate for five SNPs with the highest rates
Method	rs965951	rs2245431	rs2832321	rs16983706	rs468619
Proposed	0.73	0.48	0.44	0.28	0.34
Lasso	0.97	0.94	0.82	0.75	0.84
B-lasso	1.00	1.00	1.00	1.00	1.00
	average model size			PMSE
Proposed	3.394 (2.571)			0.035 (0.021)
Lasso	58.98 (22.93)			0.034 (0.016)
B-lasso	200.0 (0.000)			0.064 (0.037)

Note: Numbers in parentheses are standard deviations.

Table 6. Results based on 100 random splits for the OWB example.

Method	Individual demographic variable selection
	Age	Treatment Months	Years of edu
Proposed	0.00	0.02	0.00
glasso	0.00	0.79	0.01
gbridge	0.00	0.04	0.00
sglasso	0.00	0.15	0.00
	group selection
	Personality	Physical health	Psych health	Spiritual health
	size=5	size=4	size=6	size=12
Proposed	0.00	0.00	1.00	0.21
glasso	0.01	0.90	1.00	0.20
gbridge	0.00	0.00	1.00	0.92
sglasso	0.03	0.46	1.00	0.69
	Active coping	Passive coping	Social support	Self-efficacy
	size=7	size=5	size=4	size=11
Proposed	0.04	0.05	1.00	0.05
glasso	0.33	0.02	1.00	0.43
gbridge	0.02	0.06	0.81	0.26
sglasso	0.60	0.13	1.00	0.68
	Selection within psychological health group
	ttraiAnx	tstatAnx	tbodimg	tcesd	tthinkbc
Proposed	1.00	0.98	0.79	0.64	0.70
glasso	1.00	1.00	1.00	1.00	1.00
gbridge	1.00	1.00	1.00	1.00	1.00
sglasso	1.00	1.00	0.99	1.00	0.32
	Selection within social support	Selection within spiritual health
	tcommnow	sprview6
Proposed	1.00	0.20
glasso	1.00	0.20
gbridge	0.81	0.91
sglasso	1.00	0.68
	Average of	Average of selected
	selected groups	individual variables	PMSE
Proposed	2.370 (0.646)	5.500 (1.439)	2.789 (0.448)
glasso	4.690 (0.837)	23.99 (6.889)	2.787 (0.425)
gbridge	3.110 (0.601)	10.48 (2.172)	2.713 (0.452)
sglasso	24.740 (2.008)	10.70 (4.150)	2.837 (0.424)

Figure 2. The posteriors of regression coefficients in three groups.