Variable selection in finite mixture of median regression models using skew-normal distribution

Figures & data

Table 1. Three penalty functions are used for variable selection procedure.

			${\nu }_1=0.35$			${\nu }_1=0.5$
Penalty	Com.	n	C${}_{\left(\mathrm{S}\mathrm{D}\right)}$	IC${}_{\left(\mathrm{S}\mathrm{D}\right)}$	MSE$(\widehat{\text{β}}{)}_{\left(\mathrm{S}\mathrm{D}\right)}$	C${}_{\left(\mathrm{S}\mathrm{D}\right)}$	IC${}_{\left(\mathrm{S}\mathrm{D}\right)}$	MSE$(\widehat{\text{β}}{)}_{\left(\mathrm{S}\mathrm{D}\right)}$
LASSO	Com. 1	200	2.6500${}_{\left(0.0750\right)}$	0.0650${}_{\left(0.1890\right)}$	0.1677${}_{\left(1.2661\right)}$	2.8250${}_{\left(0.0750\right)}$	0.0550${}_{\left(0.1313\right)}$	0.1556${}_{\left(1.0248\right)}$
		400	2.8750${}_{\left(0.0500\right)}$	0.0250${}_{\left(0.0991\right)}$	0.0975${}_{\left(0.4506\right)}$	2.9500${}_{\left(0.0500\right)}$	0.0125${}_{\left(0.0770\right)}$	0.0849${}_{\left(0.3854\right)}$
		800	2.9500${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0000\right)}$	0.0550${}_{\left(0.1240\right)}$	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0406${}_{\left(0.0428\right)}$
	Com. 2	200	1.7750${}_{\left(0.0500\right)}$	0.0500${}_{\left(0.1206\right)}$	0.1485${}_{\left(0.9960\right)}$	1.7760${}_{\left(0.0500\right)}$	0.0500${}_{\left(0.1204\right)}$	0.1484${}_{\left(0.9980\right)}$
		400	1.9775${}_{\left(0.0000\right)}$	0.0150${}_{\left(0.0556\right)}$	0.0622${}_{\left(0.2210\right)}$	1.9975${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0701\right)}$	0.0833${}_{\left(0.3552\right)}$
		800	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0300${}_{\left(0.0728\right)}$	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0443${}_{\left(0.0462\right)}$
HARD	Com. 1	200	2.7500${}_{\left(0.0500\right)}$	0.0150${}_{\left(0.1210\right)}$	0.1335${}_{\left(0.3055\right)}$	2.8075${}_{\left(0.0500\right)}$	0.0010${}_{\left(0.0500\right)}$	0.1036${}_{\left(0.2840\right)}$
		400	2.9975${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0712${}_{\left(0.0850\right)}$	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0511${}_{\left(0.0701\right)}$
		800	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0441${}_{\left(0.0495\right)}$	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0301${}_{\left(0.0320\right)}$
	Com. 2	200	1.8750${}_{\left(0.0500\right)}$	0.0200${}_{\left(0.0500\right)}$	0.0623${}_{\left(0.2751\right)}$	1.8750${}_{\left(0.0500\right)}$	0.0100${}_{\left(0.0497\right)}$	0.0954${}_{\left(0.2700\right)}$
		400	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0371${}_{\left(0.0621\right)}$	2.0000${}_{\left(0.0000\right)}$	0.0050${}_{\left(0.0000\right)}$	0.0569${}_{\left(0.0602\right)}$
		800	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0202${}_{\left(0.0242\right)}$	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0296${}_{\left(0.0313\right)}$
SCAD	Com. 1	200	2.7650${}_{\left(0.0500\right)}$	0.0550${}_{\left(0.1313\right)}$	0.1389${}_{\left(0.3026\right)}$	2.7750${}_{\left(0.0500\right)}$	0.0175${}_{\left(0.0689\right)}$	0.0880${}_{\left(0.0901\right)}$
		400	2.9240${}_{\left(0.0000\right)}$	0.0150${}_{\left(0.0000\right)}$	0.0712${}_{\left(0.0954\right)}$	3.0000${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0016\right)}$	0.0496${}_{\left(0.0550\right)}$
		800	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0366${}_{\left(0.0424\right)}$	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0277${}_{\left(0.0297\right)}$
	Com. 2	200	1.9550${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0500\right)}$	0.0717${}_{\left(0.2505\right)}$	1.9550${}_{\left(0.0000\right)}$	0.0100${}_{\left(0.0524\right)}$	0.0927${}_{\left(0.1023\right)}$
		400	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0454${}_{\left(0.0524\right)}$	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0537${}_{\left(0.0689\right)}$
		800	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0197${}_{\left(0.0202\right)}$	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0315${}_{\left(0.0350\right)}$

Note: The first column indicates the penalty function used for variable selection method and the second column indicates the component.

Table 2. Simulation results of the parameters of scale, skewness and mixing proportion for ${\nu }_1$=0.5.

Parameters	Com.	n	MSE($\hat{\sigma }$)${}_{\left(\mathrm{S}\mathrm{D}\right)}$	MSE($\hat{\lambda }{)}_{\left(\mathrm{S}\mathrm{D}\right)}$	MSE($\hat{\nu }{)}_{\left(\mathrm{S}\mathrm{D}\right)}$
LASSO	Com. 1	200	0.0209${}_{\left(0.1656\right)}$	0.0350${}_{\left(0.1202\right)}$	0.0014${}_{\left(0.0045\right)}$
		400	0.0098${}_{\left(0.0650\right)}$	0.0149${}_{\left(0.0455\right)}$	0.0006${}_{\left(0.0031\right)}$
		800	0.0054${}_{\left(0.0075\right)}$	0.0040${}_{\left(0.0069\right)}$	0.0002${}_{\left(0.0014\right)}$
	Com. 2	200	0.0221${}_{\left(0.1705\right)}$	0.0342${}_{\left(0.1351\right)}$	0.0014${}_{\left(0.0046\right)}$
		400	0.0104${}_{\left(0.0655\right)}$	0.0140${}_{\left(0.0459\right)}$	0.0006${}_{\left(0.0031\right)}$
		800	0.0060${}_{\left(0.0081\right)}$	0.0042${}_{\left(0.0077\right)}$	0.0003${}_{\left(0.0012\right)}$
HARD	Com. 1	200	0.0081${}_{\left(0.0765\right)}$	0.0120${}_{\left(0.0153\right)}$	0.0012${}_{\left(0.0035\right)}$
		400	0.0030${}_{\left(0.0089\right)}$	0.0041${}_{\left(0.0058\right)}$	0.0006${}_{\left(0.0016\right)}$
		800	0.0015${}_{\left(0.0017\right)}$	0.0027${}_{\left(0.0035\right)}$	0.0003${}_{\left(0.0006\right)}$
	Com. 2	200	0.0090${}_{\left(0.0790\right)}$	0.0130${}_{\left(0.0147\right)}$	0.0012${}_{\left(0.0031\right)}$
		400	0.0035${}_{\left(0.0087\right)}$	0.0049${}_{\left(0.0055\right)}$	0.0006${}_{\left(0.0017\right)}$
		800	0.0018${}_{\left(0.0021\right)}$	0.0028${}_{\left(0.0031\right)}$	0.0003${}_{\left(0.0008\right)}$
SCAD	Com. 1	200	0.0087${}_{\left(0.0924\right)}$	0.0113${}_{\left(0.0255\right)}$	0.0013${}_{\left(0.0040\right)}$
		400	0.0038${}_{\left(0.0209\right)}$	0.0042${}_{\left(0.0097\right)}$	0.0006${}_{\left(0.0029\right)}$
		800	0.0017${}_{\left(0.0025\right)}$	0.0030${}_{\left(0.0046\right)}$	0.0003${}_{\left(0.0010\right)}$
	Com. 2	200	0.0091${}_{\left(0.0889\right)}$	0.0130${}_{\left(0.0270\right)}$	0.0013${}_{\left(0.0037\right)}$
		400	0.0034${}_{\left(0.0224\right)}$	0.0052${}_{\left(0.0104\right)}$	0.0007${}_{\left(0.0024\right)}$
		800	0.0018${}_{\left(0.0020\right)}$	0.0031${}_{\left(0.0049\right)}$	0.0003${}_{\left(0.0009\right)}$

Table 3. Varying skewness with n = 400 and ${\nu }_1=0.5$.

Penalty	Com.	λ	C	IC	MSE$\left(\widehat{\text{β}}\right)$
LASSO	Com. 1	0.5	2.9975${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0557\right)}$	0.0851${}_{\left(0.3558\right)}$
		1.5	2.9550${}_{\left(0.0500\right)}$	0.0175${}_{\left(0.0721\right)}$	0.0902${}_{\left(0.3506\right)}$
		3	2.9500${}_{\left(0.0000\right)}$	0.0125${}_{\left(0.0770\right)}$	0.0849${}_{\left(0.3854\right)}$
	Com. 2	−0.5	1.9750${}_{\left(0.0500\right)}$	0.0225${}_{\left(0.0725\right)}$	0.0803${}_{\left(0.3724\right)}$
		−1.5	1.9975${}_{\left(0.0000\right)}$	0.0125${}_{\left(0.0750\right)}$	0.0770${}_{\left(0.3550\right)}$
		−3	1.9975${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0701\right)}$	0.0833${}_{\left(0.3552\right)}$
HARD	Com. 1	0.5	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0015\right)}$	0.0547${}_{\left(0.0684\right)}$
		1.5	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0676${}_{\left(0.0778\right)}$
		3	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0511${}_{\left(0.0701\right)}$
	Com. 2	−0.5	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0050\right)}$	0.0604${}_{\left(0.0599\right)}$
		−1.5	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0685${}_{\left(0.0619\right)}$
		−3	2.0000${}_{\left(0.0000\right)}$	0.0050${}_{\left(0.0000\right)}$	0.0569${}_{\left(0.0602\right)}$
SCAD	Com. 1	0.5	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0025\right)}$	0.0536${}_{\left(0.0511\right)}$
		1.5	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0512${}_{\left(0.0504\right)}$
		3	3.0000${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0016\right)}$	0.0496${}_{\left(0.0550\right)}$
	Com. 2	−0.5	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0509${}_{\left(0.0661\right)}$
		−1.5	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0019\right)}$	0.0632${}_{\left(0.0604\right)}$
		−3.	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	0.0537${}_{\left(0.0689\right)}$

Table 4. Varying sample size n with ${\lambda }_1=3$, ${\lambda }_2=-3$ and $\nu =0.5$.

			FMMeR model		NMR model
Penalty	Com.	n	C${}_{\left(SD\right)}$	IC${}_{\left(SD\right)}$	C${}_{\left(SD\right)}$	IC${}_{\left(SD\right)}$
LASSO	Com. 1	200	2.8250${}_{\left(0.0750\right)}$	0.0550${}_{\left(0.1313\right)}$	2.8540${}_{\left(0.3550\right)}$	0.0500${}_{\left(0.0250\right)}$
		400	2.9500${}_{\left(0.0500\right)}$	0.0125${}_{\left(0.0770\right)}$	2.9200${}_{\left(0.0224\right)}$	0.0250${}_{\left(0.0000\right)}$
	Com. 2	200	1.7760${}_{\left(0.0500\right)}$	0.0500${}_{\left(0.1204\right)}$	1.7750${}_{\left(0.3395\right)}$	0.0650${}_{\left(0.0300\right)}$
		400	1.9975${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0701\right)}$	1.9240${}_{\left(0.0206\right)}$	0.0100${}_{\left(0.0000\right)}$
HARD	Com. 1	200	2.8075${}_{\left(0.0500\right)}$	0.0010${}_{\left(0.0500\right)}$	2.8200${}_{\left(0.1192\right)}$	0.0025${}_{\left(0.0575\right)}$
		400	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	3.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$
	Com. 2	200	1.8750${}_{\left(0.0500\right)}$	0.0100${}_{\left(0.0497\right)}$	1.7550${}_{\left(0.0097\right)}$	0.0150${}_{\left(0.0596\right)}$
		400	2.0000${}_{\left(0.0000\right)}$	0.0050${}_{\left(0.0000\right)}$	1.9975${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$
SCAD	Com. 1	200	2.7750${}_{\left(0.0500\right)}$	0.0175${}_{\left(0.0689\right)}$	2.8065${}_{\left(0.1428\right)}$	0.0155${}_{\left(0.0775\right)}$
		400	3.0000${}_{\left(0.0000\right)}$	0.0025${}_{\left(0.0016\right)}$	2.9750${}_{\left(0.0042\right)}$	0.0025${}_{\left(0.0000\right)}$
	Com. 2	200	1.9550${}_{\left(0.0000\right)}$	0.0100${}_{\left(0.0524\right)}$	1.7500${}_{\left(0.1300\right)}$	0.0200${}_{\left(0.0790\right)}$
		400	2.0000${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$	1.9950${}_{\left(0.0000\right)}$	0.0000${}_{\left(0.0000\right)}$

Figure 1. Histogram of the BMI.

Figure 2. Clustering results for the BMI data obtained. (a) The 4 clusters at level 3. (b) The ascending paths from the modes at level 3 to those at level 4 and the contours of the density estimate at level 4. (c) The 2 clusters at level 4. (d) The ascending paths from the modes at level 4 to the next level and the contours of the density estimate at the next level.

Table 5. Variable selection for BMI data set via FMMeR model.

	LASSO		HARD		SCAD
Covariates	Com.1	Com.2	Com.1	Com.2	Com.1	Com.2
${\mathit{x}}_1$			−0.7386		−0.7224
${\mathit{x}}_2$		0.0674
${\mathit{x}}_3$
${\mathit{x}}_4$	0.8752	0.7071	1.0808	0.7490	1.0726	0.7358
${\mathit{x}}_5$	0.0032	0.0069	0.0027	0.0027	0.0034	0.0016
${\mathit{x}}_6$		0.0501		0.0468		0.0424
${\mathit{x}}_7$	0.2773	0.5750	0.7060	0.6165	0.6848	0.5997
${\mathit{x}}_8$	0.2087	1.1808	0.6887	1.1974	0.6666	1.1324
${\mathit{x}}_9$	−0.0446	−0.1054	−0.0668	−0.1067	−0.0656	−0.1003
${\mathit{x}}_{10}$		−0.7357	−0.4254	−0.7508	−0.4060	−0.7007

Table 6. Variable selection for BMI data set via finite mixture of MODLR model.

	LASSO		HARD		SCAD
Covariates	Com.1	Com.2	Com.1	Com.2	Com.1	Com.2
${\mathit{x}}_1$			−0.7244		−0.7226
${\mathit{x}}_2$		0.0709
${\mathit{x}}_3$
${\mathit{x}}_4$	0.8790	0.6873	1.0554	0.7283	1.0496	0.7153
${\mathit{x}}_5$	0.0038	0.0070	0.0033	0.0026	0.0032	0.0019
${\mathit{x}}_6$		0.0486		0.0447		0.0410
${\mathit{x}}_7$	0.2690	0.6033	0.7884	0.6480	0.7628	0.6284
${\mathit{x}}_8$	0.2062	1.2133	0.7931	1.2285	0.7621	1.1663
${\mathit{x}}_9$	−0.0454	−0.1079	−0.0714	−0.1089	−0.0687	−0.1027
${\mathit{x}}_{10}$		−0.7595	−0.5188	−0.7734	−0.4929	−0.7256

Table 7. Variable selection for BMI data set via NMR model.

	LASSO		HARD		SCAD
Covariates	Com.1	Com.2	Com.1	Com.2	Com.1	Com.2
${\mathit{x}}_1$	−0.9520		−0.9582
${\mathit{x}}_2$	−0.0457	0.0824	−0.0451	0.0824
${\mathit{x}}_3$	0.0754		0.0759
${\mathit{x}}_4$	0.9064	0.6228	0.9067	0.6227	0.0794	0.0693
${\mathit{x}}_5$	0.0049	0.0081	0.0049	0.0081	0.0003	0.0003
${\mathit{x}}_6$	−0.019	0.0438	−0.0189	0.0437		0.0038
${\mathit{x}}_7$	0.8690	0.6757	0.8703	0.6761	0.0244	0.0634
${\mathit{x}}_8$	0.7615	1.2918	0.7639	1.2921	0.0195	0.1164
${\mathit{x}}_9$	−0.0729	−0.1134	−0.0731	−0.1134	0.0068	0.0008
${\mathit{x}}_{10}$	−0.4855	−0.8169	−0.4876	−0.8172	−0.0137	−0.0884