Applying multilevel regression weighting when only population margins are available

Figures & data

Table 1. Weighting and complex estimation strategies considered in the simulation study.

Weighting and complex estimation strategy	Description
Unweighted	Unweighted estimation
Postratification	Standard poststratification without a multilevel regression model
MLP	Multilevel regression and poststratification
Raking	Standard raking without a multilevel regression model
MRR	Multilevel regression and raking
MRPCS	Multilevel regression and population cell size estimation using weighted least squares

Table 2. Comparisons of Monte-Carlo bias, Monte-Carlo variance (MCvar) and mean squared error (MSE).

Estimator	MCbias	MCvar	MSE
Scenario 1a
Unweighted	−0.2344	0.0002	0.0551
Poststratification	−0.0107	0.0002	0.0003
MRP	−0.0010	0.0001	0.0001
Raking	−0.0035	0.0002	0.0002
MRR	−0.0050	0.0002	0.0002
MRPCS	−0.0121	0.0002	0.0003
Scenario 1b
Unweighted	−0.2344	0.0002	0.0551
Poststratification	−0.0526	0.0002	0.0030
MRP	−0.0579	0.0002	0.0036
Raking	−0.0496	0.0002	0.0027
MRR	−0.0584	0.0002	0.0036
MRPCS	−0.0589	0.0002	0.0037
Scenario 2
Unweighted	−0.4661	0.0000	0.2173
Poststratification	−0.1886	0.0034	0.0390
MRP	−0.0685	0.0084	0.0131
Raking	−0.3206	0.0036	0.1064
MRR	−0.3350	0.0022	0.1144
MRPCS	−0.0719	0.0064	0.0116
Scenario 3
Unweighted	0.1174	0.0004	0.0142
Poststratification	0.0053	0.0006	0.0006
MRP	−0.0105	0.0003	0.0004
Raking	0.0516	0.0004	0.0031
MRR	−0.0104	0.0003	0.0004
MRPCS	−0.0104	0.0003	0.0005

Table A.1. Values for the parameters in the model that generates the survey variable of interest y that is depending on the weighting variables ${\mathbf{x}}_{\mathbf{1}}\dots {\mathbf{x}}_{\mathbf{4}}.$

Parameter	Category	Scenario 1	Scenario 2	Scenario 3
δ		−0.5	−0.5	−12
${\gamma }_{j[i]}^{x_1}$	j = 1	−7	−7	−0.2
	j = 2	0	0	0
	j = 3	−0.2	−0.2	−0.2
	j = 4	0.4	0.4	0.4
	j = 5	7	7	2
${\gamma }_{k[i]}^{x_2}$	k = 1	−7	−7	−1.5
	k = 2	0.4	0.4	0.4
	k = 3	0	0	0
	k = 4	−0.2	−0.2	−0.2
	k = 5	7	7	1.5
${\gamma }_{l[i]}^{x_3}$	l = 1	1.5	1.5	1.5
	l = 2	−0.4	−0.4	−0.4
	l = 3	0	0	0
	l = 4	0.9	0.9	0.9
${\gamma }_{v[i]}^{x_4}$	v = 1	0.1	0.1	–
	v = 2	0.1	0.1	–
	v = 3	0.1	0.1	–
	v = 4	0	0	–
	v = 5	0.1	0.1	–
${\gamma }^{x_5}$	–	–	–	0.8

Table A.2. Values for the parameters in the inclusion model generating the inclusion propensities $\mathbf{\omega }$ that are depending on the weighting variables ${\mathbf{x}}_{\mathbf{1}}\dots {\mathbf{x}}_{\mathbf{4}}.$

Parameter	Category	Scenario 1	Scenario 2	Scenario 3
λ		−1	1e − 07	−14.5
${\xi }_{j[i]}^{x_1}$	j = 1	1	50	−0.1
	j = 2	0	0	0
	j = 3	−0.2	−0.2	−0.2
	j = 4	0.4	0.4	0.4
	j = 5	−1	−50	0.1
${\xi }_{k[i]}^{x_2}$	k = 1	0	0	0
	k = 2	−1	−55	−1e − 07
	k = 3	−1	−55	−1e − 07
	k = 4	−1	−50	−1e − 07
	k = 5	−1	−50	−1e − 07
${\xi }_{l[i]}^{x_3}$	l = 1	0	0	0
	l = 2	0.4	0.4	0.4
	l = 3	0.3	0.3	0.3
	l = 4	0.2	0.2	0.2
${\xi }_{v[i]}^{x_4}$	v = 1	1	50	–
	v = 2	0.1	0.1	–
	v = 3	0	0	–
	v = 4	0.2	0.2	–
	v = 5	−1	−50	–
${\xi }^{x_5}$		–	–	0.9
Weighting cells		500	500	100
Empty cells		217.82	390.97	3.75
		(43.6 %)	(78.2 %)	(3.7 %)

The table includes the numbers of the resulting weighting cells and the amount of empty weighting cells averaged over all simulation rounds.

Table A.3. Values for the parameters in the model that generates ${\mathbf{x}}_{\mathbf{5}}$ that is depending on the weighting variables ${\mathbf{x}}_{\mathbf{1}}\dots {\mathbf{x}}_{\mathbf{3}}.$

Parameter	Category	Scenario 3
a		15
$b_j^{x_1}$	j = 1	−0.2
	j = 2	0
	j = 3	−0.2
	j = 4	0.2
	j = 5	0.3
$b_k^{x_2}$	k = 1	−1
	k = 2	0.4
	k = 3	0
	k = 4	−0.2
	k = 5	1
$b_l^{x_3}$	l = 1	−0.0001
	l = 2	−0.0001
	l = 3	0
	l = 4	0.0001

Figure 1. Scenario 1a—Volunteer sample with moderately skewed inclusion.

Figure 2. Scenario 1b—Volunteer sample with moderately skewed inclusion and omitted variable in the weighting and multilevel regression models.

Figure 3. Scenario 2—Volunteer sample with a highly skewed inclusion.

Figure 4. Scenario 3—Volunteer sample with highly skewed inclusion. The multilevel model is enriched with information which cannot be included in the weighting model.

Table B.1. Cramér’s V for the association between the weighting variables ${\mathbf{x}}_{\mathbf{1}}\dots {\mathbf{x}}_{\mathbf{4}}$ and the survey variable of interest y, the association between the inclusion vector z and the survey variable of interest y, and the association between the weighting variables and the inclusion vector z.

Association	Scenario 1	Scenario 2	Scenario 3
$C{V}_{\mathbf{x}\mathbf{1},\mathbf{y}}$	0.68	0.68	0.40
$C{V}_{\mathbf{x}\mathbf{2},\mathbf{y}}$	0.59	0.59	0.32
$C{V}_{\mathbf{x}\mathbf{3},\mathbf{y}}$	0.14	0.14	0.23
$C{V}_{\mathbf{x}\mathbf{4},\mathbf{y}}$	0.12	0.12	0.63
$C{V}_{\mathbf{z},\mathbf{y}}$	0.28	0.51	0.18
$C{V}_{\mathbf{x}\mathbf{1},\mathbf{z}}$	0.32	0.51	0.27
$C{V}_{\mathbf{x}\mathbf{2},\mathbf{z}}$	0.33	0.77	0.10
$C{V}_{\mathbf{x}\mathbf{3},\mathbf{z}}$	0.08	0.11	0.09
$C{V}_{\mathbf{x}\mathbf{4},\mathbf{z}}$	0.24	0.36	0.46

Table B.2. Simulated margins for the weighting variables ${\mathbf{x}}_{\mathbf{1}}\dots {\mathbf{x}}_{\mathbf{4}}.$

Variable	Category	Scenario 1	Scenario 2	Scenario 3
${\mathbf{x}}_{\mathbf{1}}$	1	408.21	574.02	98.83
	2	155.44	165.17	175.05
	3	96.20	94.38	99.41
	4	255.65	149.94	240.59
	5	84.50	16.49	386.12
${\mathbf{x}}_{\mathbf{2}}$	1	482.35	853.54	224.57
	2	216.50	62.50	252.55
	3	175.31	25.46	232.10
	4	84.08	46.64	170.13
	5	41.76	11.86	120.66
${\mathbf{x}}_{\mathbf{3}}$	1	266.09	317.98	210.13
	2	288.97	268.90	267.14
	3	246.11	229.88	262.46
	4	198.82	183.24	260.27
${\mathbf{x}}_{\mathbf{4}}$	1	411.71	519.74
	2	229.30	219.88
	3	199.71	168.47
	4	128.58	80.58
	5	30.70	11.33