Are there differences in responses to social identity questions in face-to-face versus telephone interviews? Results of an experiment on a longitudinal survey

Figures & data

Table 1. Sample selection.

	Number of cases excluded	Eligible sample size
Adults in wave 2 enumerated households	0	2026
Non-respondents	156	1870
Proxy respondents	82	1788
Switchers	219	1569
New entrants at wave 2 or not interviewed in wave 1	117	1452
Survey error	34	1418

Figure 1. Identity questions in wave 2 of the Understanding Society Innovation Panel.

Table 2. Comparison of socio-demographic characteristics of face-to-face and telephone samples.^a

	Face-to-face	Telephone	χ^2 test p-value
Female?	56.7%	55.7%	0.71
Born in UK?	92.7%	94.0%	0.33
Ethnic group is non-White?^b	5.9%	4.9%	0.44
Has access to a car?	81.4%	85.9%	0.02
Number of own children in household	0.4	0.4
Birth cohort
1900–1940	21.3%	20.1%	0.61
1941–1950	16.5%	17.1%	0.77
1951–1960	15.4%	20.4%	0.02
1961–1970	20.6%	20.1%	0.85
1971–1980	15.6%	12.5%	0.10
1981–1990	7.1%	7.3%	0.91
1991–2000	3.5%	2.5%	0.23
Educational qualifications
First degree or higher	21.6%	24.1%	0.27
School completion	32.7%	32.4%	0.90
Other	26.4%	27.3%	0.72
No qualifications	19.3%	16.2%	0.14
Owns house?	77.8%	81.6%	0.08
Religion
None	47.0%	47.0%	0.98
Christian	45.4%	46.6%	0.66
Other religion	7.5%	6.4%	0.41
Current health status
Very good health	50.3%	52.3%	0.45
Good Health	46.5%	40.7%	0.03
Poor Health	3.2%	7.0%	0.00
Marital Status
Never Married	15.1%	13.5%	0.41
Married	65.2%	70.9%	0.02
Separated, divorced or widowed	19.7%	15.5%	0.04
Current activity status
Employed	55.9%	59.3%	0.21
Unemployed	3.2%	2.5%	0.41
Retired	27.9%	27.3%	0.80
Other	12.9%	11.0%	0.26
Region of residence
London	9.4%	7.4%	0.18
East and South	31.7%	36.5%	0.06
Midlands	16.5%	16.8%	0.88
North	30.3%	24.8%	0.02
Wales	5.7%	5.9%	0.87
Scotland	6.4%	8.6%	0.13
Number of observations	564	854

^a Excludes respondents who switched interview modes from their initial assignments.

^b Aggregate of all 2011 ethnic group categories, where respondents do choose one of the White groups.

Table 3. Tests of differences in item non-response by interview mode.^a

	Item non-response^b			Tests of mode effects		Number of item non-response cases in combined sample (N = 1418)
	F2F (%)	Telephone (%)	All (%)	χ^2 test	Exact Fisher’s test
				p-values	p-values
Occupation/profession	3.02	1.17	1.91	0.013	0.016	27
Ethnic or racial background	3.48	1.46	2.29	0.821	1.000	11
Religion	2.55	0.90	1.53	0.673	0.651	4
National identity	0.71	0.82	0.78	0.696	1.000	9
Political beliefs	0.36	0.23	0.28	0.176	0.222	6
Family	0.53	0.70	0.64			0
Father’s ethnic group^c	0.71	0.23	0.42	0.061	0.073	16
Marital or partnership status	0.00	0.00	0.00	0.097	0.168	9
Gender	1.78	0.70	1.13	0.101	0.125	22
Age and life stage	1.07	0.35	0.64	0.770	0.747	9
Education	0.89	1.99	1.55	0.766	1.000	2
Sexual orientation	0.71	0.59	0.64	0.372	0.492	20

Note: Statistically significant differences at the 5% level are indicated in italics.

^a Excludes respondents who switched interview modes from their initial assignments.

^b Item non-response is measured by the proportion of respondents who respond to these questions with ‘don’t know’ or ‘refuse’. Most of these are ‘don’t know’ responses, with only a handful of ‘refusals’.

^c ‘Your mother’s ethnic group’ was asked of only those whose mother’s race/ethnicity was different from that of her father and so there were only a handful of cases who were asked this question. So, we have omitted this question from the analysis.

Table 4. Estimates of identity response from ordered logit using only mode effect as covariate.^a

	Estimated coefficient of interview mode	p-value	Predicted probability of identity responses
			Face-to-face			Telephone
			Important	Not very important	Not at all important	Important	Not very important	Not at all important
Occupation/profession	0.09	0.46	0.66	0.17	0.16	0.68	0.16	0.15
Ethnic or racial background	−0.15	0.18	0.56	0.30	0.14	0.52	0.32	0.16
Religion	−0.09	0.47	0.38	0.36	0.26	0.36	0.37	0.27
National identity	−0.15	0.28	0.75	0.21	0.05	0.72	0.23	0.06
Political beliefs	0.38	0.00	0.34	0.46	0.21	0.43	0.42	0.15
Family	−0.38	0.34	0.98	0.01	0.01	0.97	0.02	0.01
Father’s ethnic group^b	0.06	0.61	0.47	0.32	0.21	0.48	0.32	0.20
Marital or partnership status	0.01	0.95	0.72	0.16	0.12	0.72	0.16	0.12
Gender	−0.24	0.10	0.80	0.14	0.06	0.76	0.16	0.08
Age and life stage	−0.19	0.11	0.62	0.31	0.08	0.57	0.34	0.09
Education	0.22	0.13	0.71	0.24	0.05	0.75	0.20	0.04
Sexual orientation	−0.09	0.46	0.65	0.24	0.10	0.63	0.25	0.11
Occupation	0.15	0.37	0.68	0.16	0.16	0.71	0.15	0.14
Profession	0.04	0.80	0.65	0.18	0.17	0.66	0.18	0.16

Note: Mode effects that are statistically significant at 1% level of significant are indicated in bold.

^a Excludes respondents who switched interview modes from their initial assignments. Predicted probabilities have been computed by averaging the predicted probabilities of all observations.

^b ‘Your mother’s ethnic group’ was asked of only those whose mother’s race/ethnicity was different from that of her father and so there were only a handful of cases who were asked this question. So, we have omitted this question from the analysis.

Table 5. Estimates of identity response from ordered logit using with mode effect and additional characteristics as covariates.^a

	Estimated coefficient of interview mode	p-value	Predicted probability of identity responses
			Face-to-face			Telephone
			Important	Not very important	Not at all important	Important	Not very important	Not at all important
Occupation/profession	0.09	0.53	0.69	0.18	0.13	0.71	0.17	0.12
Ethnic or racial background	−0.15	0.22	0.57	0.30	0.13	0.53	0.32	0.15
Religion	−0.13	0.36	0.33	0.50	0.17	0.30	0.51	0.19
National identity	−0.12	0.38	0.76	0.20	0.04	0.74	0.22	0.05
Political beliefs	0.34	0.00	0.34	0.47	0.19	0.42	0.44	0.14
Family	−0.38	0.43	1.00	0.00	0.00	1.00	0.00	0.00
Father’s ethnic group b	0.08	0.48	0.47	0.34	0.19	0.49	0.33	0.17
Marital or partnership status	−0.08	0.61	0.80	0.14	0.05	0.79	0.15	0.06
Gender	−0.22	0.14	0.81	0.13	0.06	0.78	0.16	0.07
Age and life stage	−0.17	0.16	0.62	0.31	0.07	0.58	0.34	0.08
Education	0.11	0.45	0.77	0.20	0.03	0.79	0.18	0.03
Sexual orientation	−0.13	0.30	0.67	0.24	0.09	0.64	0.26	0.10
Occupation	0.23	0.27	0.69	0.17	0.13	0.74	0.15	0.11
Profession	0.00	0.99	0.68	0.19	0.12	0.68	0.19	0.12

Note: Mode effects that are statistically significant at 1% level of significance are indicated in bold.

^a Excludes respondents who switched interview modes from their initial assignments. Additional covariates included in the model are those listed in Table 2. Predicted probabilities have been computed by averaging the predicted probabilities of all observations.

^b ‘Your mother’s ethnic group’ was asked of only those whose mother’s race/ethnicity was different from that of her father and so there were only a handful of cases who were asked this question. So, we have omitted this question from the analysis.