Backfire effect of salient information on vaccine take-up experimental evidence from scared-straight intervention in rural northern Nigeria

Figures & data

Table 1. Balance tests (mean)

	Scared-straight flipcharts	Control flipcharts	Difference (p = value)
	(1)	(2)	(3)
Baseline characteristics
Age	25.231	25.271	0.899
Muslim	0.483	0.511	0.118
No education	0.478	0.489	0.632
Primary school	0.241	0.256	0.490
Secondary school or more	0.278	0.255	0.269
Has paid work	0.446	0.435	0.666
Earning per capita per day	5615.647	6110.985	0.234
Not married	0.153	0.148	0.770
Pregnant	0.165	0.198	0.094
Have children	0.781	0.754	0.204
Ever used clinic	0.736	0.732	0.820
Distance to health clinic (km)	1.717	1.689	0.132
Transportation costs (both way)	128.395	121.805	0.466
Opportunity costs (both way)	4.229	4.052	0.792
Received tetanus vaccine before	0.407	0.404	0.895
Perception
Likely to contract tetanus	0.366	0.394	0.233
Number of people die of tetanus (0–100)	30.859	29.534	0.249
Worried about tetanus	0.331	0.379	0.035
Tetanus is bad	0.427	0.45	0.344
Important to protect from tetanus	0.476	0.515	0.105
Vaccine efficacy	22.899	21.569	0.273
Heart rate	86.414	87.518	0.130

Notes: These are based on the analysis sample of 1,660 women. “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Worried about tetanus,” “Tetanus is bad,” and “Important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). The estimates are with village fixed effects.

Table 2. Priming effect by vaccine experience (OLS regression)

	Received Vaccine
	Regression with covariates (1)	Regression with covariates and health facility fixed effects (2)	Regression with covariates and village fixed effects (3)
Scared-straight flipcharts	−0.055**	−0.061**	−0.040*
	(0.026)	(0.023)	(0.023)
Scared-straight * Received vaccine before	0.088*	0.095**	0.051
	(0.045)	(0.043)	(0.040)
Received vaccine before	−0.086***	−0.052*	−0.010
	(0.030)	(0.029)	(0.030)
r2	0.077	0.062	0.024
p-values of F test: (Scared-straight + Scared-straight * Received vaccine before = 0)	0.379	0.358	0.758

Notes: These are based on the analysis sample of 1,524 women after dropping observations with missing values. Robust standard errors clustered by villages (80 villages) are in parentheses. The mean of dependent variable under control group is 0.778. Covariates are proportion of respondents with scared-straight flipcharts in village (column 1 and 2), woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, past utilization of the assigned health clinic, and the belief measurements before the intervention listed in Table 1 (Perceptions). Control mean of dependent variable is the mean under control flipcharts and Received vaccine before = 0. * significant at 10%; ** significant at 5%; *** significant at 1%

Table 3. Priming effect by past vaccine experience and cash incentives (OLS regression)

Received Vaccine
	Never received tetanus vaccine before N = 987		Received tetanus vaccine before N = 673
Sample:	Regression without heterogeneous effects (1)	Regression with heterogenous effects (2)	Regression without heterogenous effects (3)	Regression with heterogeneous effects (4)
Scared-straight flipcharts	−0.050**	−0.072*	−0.021	0.053
	(0.022)	(0.039)	(0.036)	(0.069)
C300 * Scared-straight		0.029		−0.092
		(0.060)		(0.082)
C800 * Scared-straight		0.052		−0.143*
		(0.048)		(0.084)
C300		0.178***		0.269***
		(0.042)		(0.053)
C800		0.256***		0.326***
		(0.041)		(0.056)
r2	0.030	0.126	0.021	0.119
p-values of F test: Scared-straight + Scared-straight * C300 = 0		0.262		0.469
Scared-straight + Scared-straight * C800 = 0		0.525		0.094
p-values (equality): (2) = (4) for Scared-straight flipcharts			0.467
(2) = (4) for Scared-straight flipcharts				0.092
(2) = (4) for Scared-straight + C300 * Scared-straight				0.953
(2) = (4) for Scared-straight + C800 * Scared-straight				0.211
Mean of dependent var under control	0.778	0.629	0.706	0.500

Notes: Robust standard errors clustered by villages (80 villages) are in parentheses. Covariates are woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic. All regressions include covariates and village fixed effects. Control mean of dependent variable is the mean under control flipcharts and C5. * significant at 10%; ** significant at 5%; *** significant at 1%

Table 4. Priming effect on risk perceptions (OLS regression)

	Likely to contract tetanus Number of people	Number of people die of tetanus (0–100)	Worried about tetanus	Tetanus is bad	Important to protect from tetanus	Vaccine efficacy
	(1)	(2)	(3)	(4)	(5)	(6)
Scared-straight flipcharts	0.036	3.626**	0.146***	0.156***	0.130***	−0.089
	(0.026)	(1.747)	(0.038)	(0.035)	(0.035)	(1.595)
Scared-straight * Received vaccine before	−0.049	−2.646	−0.008	−0.044	−0.058	−2.665
	(0.042)	(2.284)	(0.051)	(0.043)	(0.039)	(2.775)
Received vaccine before	−0.022	0.313	0.125***	0.148***	0.153***	4.668*
	(0.045)	(2.329)	(0.039)	(0.035)	(0.037)	(2.440)
Adjusted p value for Scared-straight flipcharts (FDR)	0.207	0.062	0.001	0.000	0.001	0.956
Adjusted p value for interaction term (FDR)	0.408	0.408	0.875	0.408	0.408	0.408
N	1524	1523	1524	1524	1524	1520
r2	0.091	0.076	0.148	0.116	0.113	0.093
p values of F test: (Scared-straight + Scared-straight * Received vaccine before = 0)	0.658	0.490	0.001	0.001	0.017	0.233
Village FE	X	X	X	X	X	X
Covariates	X	X	X	X	X	X
Mean of dependent var under control	0.448	36.534	0.486	0.568	0.664	28.975

Notes: These are based on the analysis sample of 1,660 women. Missing observations in each column is due to missing values and invalid numbers in the dependent variable. Robust standard errors clustered by villages (80 villages) are in parentheses. “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Very worried about tetanus,” “Tetanus is very bad,” and “Very important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). Covariates are woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic, and the belief measurements before the intervention listed in Table 1 (Perceptions). Control mean of dependent variable is the mean under control flipcharts and Received vaccine before = 0. * significant at 10%; ** significant at 5%; *** significant at 1%

Table 5. Emotional responses to flipcharts (OLS regression)

	Flipcharts made me feel
	Frightened	Tense	Nervous	Heart rate
	(1)	(2)	(3)	(4)
Scared-straight flipcharts	0.306***	0.330***	0.313***	4.793***
	(0.038)	(0.030)	(0.037)	(0.973)
Scared-straight * Received vaccine before	0.071	0.102*	0.112**	3.543***
	(0.058)	(0.053)	(0.053)	(1.315)
Received vaccine before	0.032	0.051	−0.017	−0.367
	(0.045)	(0.042)	(0.036)	(1.168)
Adjusted p value for Scared-straight flipcharts (FDR)	0.000	0.000	0.000	0.000
Adjusted p value for interaction term (FDR)	0.224	0.073	0.073	0.034
N	1651	1651	1650	1442
r2	0.130	0.161	0.144	0.073
p values of F test: (Scared-straight + Scared-straight * Received vaccine before = 0)	0.000	0.000	0.000	0.000
Village FE	X	X	X	X
Covariates	X	X	X	X
Mean of dependent var under control	0.287	0.228	0.298	88.235

Notes: These are based on the analysis sample of 1,660 women. Missing observations in each column is due to missing values and invalid numbers in the dependent variable. Robust standard errors clustered by villages (80 villages) are in parentheses. All the dependent variables are dummy variables which take 1 if a respondent answers “very much” or “much” to the question: “How did you feel about the flipchart you were just shown? Feel frightened, feel tensed, feel nervous, feel uncomfortable?” after the flipcharts intervention. Covariates are woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic. Control mean of dependent variable is the mean under control flipcharts and Received vaccine before = 0. * significant at 10%; ** significant at 5%; *** significant at 1%

Table

Scared-straight flipchart	Control flipchart
Example
-Tetanus is very dangerous (esp for babies)	-Tetanus is very dangerous (esp for babies)
-Typical symptoms of tetanus	-Typical symptoms of tetanus
(1) Severe pain	(1) Severe pain
(2) Muscle spasm	(2) Muscle spasm

Source:Wikimedia Commons

Table

	Never received vaccine before (N = 987)	Received vaccine before (N = 673)	Difference (p = value)
	(1)	(2)	(3)
Baseline characteristics
Age	24.552	26.272	0.000
Muslim	0.513	0.473	0.312
No education	0.480	0.487	0.822
Primary school	0.240	0.260	0.351
Secondary school or more	0.277	0.253	0.346
Has paid work	0.398	0.504	0.001
Earning per capita per day	6576.643	4781.844	0.001
Not married	0.198	0.082	0.000
Pregnant	0.170	0.195	0.170
Have children	0.675	0.905	0.000
Ever used clinic	0.674	0.822	0.000
Distance to health clinic (km)	1.711	1.694	0.880
Perception
Likely to contract tetanus	0.333	0.446	0.003
Number of people die of tetanus (0–100)	29.035	31.999	0.099
Worried about tetanus	0.260	0.492	0.000
Tetanus is bad	0.372	0.533	0.001
Important to protect from tetanus	0.413	0.612	0.000
Vaccine efficacy	20.928	24.250	0.133
Heart rate	86.664	87.321	0.418

Notes: These are based on the analysis sample of 1,660 women. “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Very worried about tetanus,” “Tetanus is very bad,” and “Very important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). The estimates are with village fixed effects, with clustered standard error (village level).

Table

	Never received tetanus vaccine before (N = 987)			Received tetanus vaccine before (N = 673)
	Control flipcharts	Difference	p value	Control flipcharts	Difference	p value
	(1)	(2)	(3)	(4)	(5)	(6)
Baseline characteristics
Age	24.508	0.094	0.825	26.063	0.449	0.340
Muslim	0.496	0.036	0.152	0.459	0.028	0.320
No Education	0.485	−0.009	0.769	0.464	0.049	0.184
Primary school	0.225	0.032	0.276	0.266	−0.012	0.739
Secondary school or more	0.285	−0.018	0.533	0.270	−0.038	0.263
Has paid work	0.397	0.001	0.965	0.517	−0.029	0.439
Earning per capita per day	6382.979	407.563	0.514	4365.560	895.077	0.068
Not married	0.201	−0.006	0.815	0.094	−0.027	0.226
Pregnant	0.170	−0.001	0.983	0.152	0.093	0.005
Have children	0.680	−0.010	0.746	0.919	−0.030	0.212
Ever used clinic	0.671	0.005	0.846	0.828	−0.014	0.582
Distance to health clinic (km)	1.720	−0.019	0.428	1.706	−0.025	0.423
Transportation costs (both way)	131.957	−1.353	0.914	121.268	−10.393	0.415
Opportunity costs (both way)	4.251	−0.441	0.654	3.956	0.733	0.400
Received tetanus vaccine before	0.000	0.000		1.000	0.000
Perception
Likely to contract tetanus	0.323	0.021	0.483	0.434	0.024	0.522
Number of people die of tetanus	29.504	−0.991	0.503	32.498	−1.074	0.564
Worried about tetanus	0.233	0.057	0.048	0.481	0.023	0.537
Tetanus is bad	0.359	0.028	0.380	0.521	0.027	0.446
Important to protect from tetanus	0.383	0.065	0.046	0.600	0.027	0.456
Vaccine efficacy	22.246	−2.773	0.059	23.836	0.891	0.671
Heart rate	86.594	0.150	0.878	86.484	1.826	0.119

Notes: These are based on the analysis sample of 987 women who never received tetanus vaccine before (columns 1–3) and 673 women who received vaccine before (columns 4–6). “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Very worried about tetanus,” “Tetanus is very bad,” and “Very important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). The estimates are with village fixed effects.