Backfire effect of salient information on vaccine take-up experimental evidence from scared-straight intervention in rural northern Nigeria

ABSTRACT

Vaccination is the most cost-effective way to prevent mothers and infants from contracting tetanus. However, developing countries struggle with the persistent low take-up of vaccination. The low risk perceptions of disease can be one of the barriers to vaccination. One way to increase the risk perceptions of disease is to use salient loss-framed messages to highlight negative consequences of not getting vaccinated. We conducted a randomized controlled trial among 1,660 women in 80 villages in northeastern Nigeria. Respondents were randomly assigned to view one of two flipcharts: (1) control flipcharts, which contained written explanation about the severity of the disease, or (2) `scared-straight’ flipcharts that contain the salient information about the disease severity in addition to the written explanation about the severity of the disease. Additionally, respondents were provided randomly assigned amounts of cash incentives. The scared-straight intervention backfired among women with no previous experience of tetanus vaccination: it decreased their vaccine take-up by 3.7–6.1 percentage points, even though it increased their perceived risk of disease and their fear level. The negative effect of the scared-straight intervention is the most prevalent among women who received the lowest amount of cash incentive. Women without experience of tetanus vaccination might have responded to the scared-straight flipcharts by denying the information provided because the flipcharts were too frightening. The use of the scared-straight tactic is not recommended to aim for the improved take-up of vaccination in developing countries where people might face budget constraints for achieving desirable behaviors.

Trial Registration: The trial was registered at ISRCTN registry (ID: ISRCTN95083356).

KEYWORDS:

• Vaccination
• salient information
• fear appeals
• backfire
• Nigeria

Introduction

Vaccination is an extremely cost-effective way to improve health. For example, treating one case of measles costs 23 times the cost of one vaccination, and 24 USD is saved for every 1 USD spent on the diphtheria-tetanus-pertussis (DTP) vaccine¹. Despite the huge benefits of preventing diseases at low cost, the persistent low vaccine take-up remains a puzzle.²

One possible reason of the low vaccine take-up is the low-risk perceptions of the disease. The meta-analysis by Brewer et al. (2007)³ finds positive correlations between adult vaccination behaviors and risk perceptions. One way to increase the risk perceptions of disease is to use salient loss-framed messages to highlight negative consequences of not getting vaccinated. In this paper, the term “salient” signifies that information is not bland but emphasis is deliberately made on the severity of disease. The paper reports a field experiment designed to evaluate the effectiveness of providing salient information about the severity of disease in increasing tetanus vaccine take-up among women of childbearing age in rural Nigeria.

Loss-framed messages emphasize the negative consequences, such as the severity of disease of not adopting recommended behaviors, in this case vaccination, while gain-framed messages emphasize the positive effects, such as long-term health benefits. Findings about the effectiveness of loss-framed messages in improving health behaviors compared to gain-framed messages have been mixed. For example, while O’Keefe and Nan (2012)⁴ find no significant difference in persuasion between gain- and loss-framed messages, other researchers provide some evidence that loss-framed messages are more persuasive for vaccination behaviors.^5,6 For example, Abhyankar et al. (2008) find that loss-framed messages increase mothers’ intentions to get their children vaccinated against measles, mumps, and rubella (MMR).

Loss-framed messages emphasizing the disease severity are typically fear appeals. Fear appeals are persuasive messages that arouse fear. Extensive research has been done to examine the effectiveness of fear appeals in health communication.⁷ On one hand, many claimed that the fear appeals induce the desirable health behaviors. For example, Dillard and Anderson (2004)⁸ made respondents recognize the threat of influenza by emphasizing its severe symptoms, finding that comprehending the threats increased the take-up of influenza vaccine and stronger threats worked better. The recent meta-analysis of fear appeals by Tannenbaum et al. (2015)⁹ shows that fear appeals are broadly effective in influencing attitudes, intentions, and behaviors.

On the other hand, a significant amount of research shows that the fear appeals do not affect health behaviors, or can even backfire.¹⁰ For example, Jepson and Chaiken (1990)¹¹ show that the fear appeals adversely affect information processing, leading to delayed behaviors. The literature highlights the importance of perceived efficacy, which consists of response efficacy, the effectiveness of recommended response, and self-efficacy, one’s ability to perform a recommended response.^7,12,13 People can respond to a fearful message by denying it, avoiding it, or processing it with bias if they do not feel that they can avoid fearful consequences. To mitigate such defensive reactions to control fear, fear appeals need to include an efficacy message, a suggestion advising respondents to seek recommended health behaviors in order to avoid fearful consequences. The meta-analysis using strict inclusion criteria by Peters et al. (2013)¹⁴ provides evidence for the adverse effects of threatening information, when efficacy is not highlighted (low efficacy).

This study evaluates the differential effect of fear appeals on vaccine take-up according to previous experience of vaccination. As Health Belief Model¹⁵ implies, it is likely that women who have had experience of vaccination have high risk perceptions, while women who have never had vaccination have low risk perceptions. Then, we hypothesize that, while women with previous vaccination experience do not respond to the salient information of disease severity by increasing their vaccine take-up because they already have high perception of the disease risk, the fear appeals are effective in increasing the vaccine take-up among women with no such experience because their baseline risk perceptions are relatively low, and can be increased through the fear appeal intervention. Women with no experience of vaccination are the primary policy target.

We study tetanus toxoid vaccines, which are the most effective in preventing maternal and neonatal tetanus and available free of charge to pregnant women in most health clinics, but which do not attain high take-up in Nigeria. Nigeria is one of 25 countries where tetanus is still a major public health problem.¹⁶ Providing tetanus toxoid vaccines to mothers most effectively protects both mothers and newborn babies from tetanus. Fatality from neonatal tetanus reaches almost 100% without medical treatment, which is difficult to obtain in rural Africa.¹⁷ Furthermore, tetanus has salient symptoms that fit to our scared-straight intervention. The most apparent symptom is the muscle spasm with severe pain.

Methods

Sampling

Our study was conducted in Jada local government area (LGA) of Adamawa state in the northeastern region of Nigeria in March–May 2013. In Adamawa state, the rate of tetanus-toxoid vaccination (two or more injections) during the last pregnancy was 58.8%, which was much lower than in southern states, 75–80%.¹⁸ Three-stage sampling was employed for the sample. We first selected health clinics where respondents receive vaccination. As a second step, we sampled villages within the catchment area of each sampled clinic. Lastly, we sampled women within each sampled village.

In the first stage, ten health clinics, geographically spread across Jada LGA, were selected. Out of the total of 11 wards within Jada LGA, we focused on nine rural wards, each of which has one to five public health clinics. We selected the main health clinic from each ward, with an exception of one large ward where we selected two clinics (i.e., 10 clinics in total). The catchment areas of each clinic were defined by the primary healthcare development agency responsible for national immunization campaigns.

In the second stage, 80 villages from the catchment areas of these 10 clinics were selected. We selected villages if they had more than 10 households and the total number of villages within the catchment area did not exceed 15; if it did, priority was given to villages furthest from the clinic. We excluded small villages in order to efficiently collect a sufficient number of respondents.

In the third stage, women of childbearing age, who are the relevant population with regard to preventing neonatal tetanus, were selected. We selected one eligible woman between the ages of 15 and 35 from each household. The survey team visited all the households in each village to find out if there were any eligible women. A woman was ineligible if she had received a tetanus vaccination within 6 months prior to the survey (to avoid overdose). In cases where there was more than one eligible woman in a household, the first priority was given to pregnant women (based on self-report), followed by women who had never received a tetanus vaccination and women who had not received a tetanus vaccine in the past 6 months to avoid overdose (based on vaccination records on antenatal care card or self-report). If there was more than one woman who was eligible under the same priority, a participant was selected based on the alphabetical order of her first name. The analysis sample consists of 1,660 women.

Experimental design

To examine the effectiveness of the salient loss-framed information (also fear appeals or priming) on the vaccine take-up, we randomly provided salient loss-framed information using `scared-straight’ flipcharts for respondents in the treatment group, while those in the control group received the control flipcharts. We call the flipcharts that contain the salient information about the disease severity `scared-straight’ flipcharts. While the control flipcharts contained written explanation about the severity of the disease without any graphics, the scared-straight flipcharts additionally provided the graphical information to emphasize the severity of the disease. To control for the level of information on the disease, both flipcharts contained information about the disease, such as the causes and symptoms of tetanus. Written explanation was read out loud by interviewers to respondents. Appendix 1 illustrates the scared-straight and control flipcharts.

In addition to the scared-straight intervention, we also provided randomly assigned amounts of cash incentives to respondents and evaluate the differential effect of the priming intervention. In each village, the amount of cash incentives that was offered was randomly assigned to each respondent, as detailed below: 5 naira (approximately 3.3 US. cents, C5), 300 naira (2 US. dollars, C300), or 800 naira (5.3 US. dollars, C800). Transportation costs to visit health facilities are often argued to be one of the major barriers to the vaccine take-up (Phillips et al, 2017). If the budget constraint prevents women from visiting the clinic for vaccination, providing cash incentives can relax the constraint and increase the perceived self-efficacy. Thus, cash incentives can increase the effectiveness of the priming intervention by increasing the perceived self-efficacy. We selected the amounts of cash incentives based on the results of our pilot study to see which amount is likely to incentivize women’s behavioral change. We employed these three amounts to capture the potentially nonlinear impacts of cash incentives.

Randomization was done using a paper questionnaire with the type of flipcharts (scared-straight or control) and the amount of the cash incentive (C5, C300, or C800) preprinted in the middle of the page. The preprinting was based on a randomization scheme generated by a computer algorithm.

This study was part of a larger study, which includes additional study arms. In this paper, we do not include respondents under the additional study arms for analysis. Under the additional study arms, respondents could receive cash incentives if they came to the clinic, independent of their decision on vaccination (clinic visit condition). All the women under the clinic visit condition received the control flipcharts to be comparable with respondents under the vaccination condition who received the control flipcharts. We designed these arms to evaluate the psychological factors of receiving the vaccine by comparing the clinic visit condition and vaccination condition. In this paper, we exclude these additional arms with the clinic visit condition so that we can compare the scared-straight and control flipcharts under the vaccination condition.

Ethics approval

Ethics approval was obtained on November 2012 from University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board (HUM00063832). Oral and written informed consents were obtained from all the respondents prior to their participation in the study.

Procedures

The baseline paper-based questionnaire was administered by trained interviewers to all women in the sample to capture their prior knowledge, beliefs, and attitudes about tetanus and tetanus vaccination, as well as their own and their household’s baseline characteristics, such as demographics, health, and economic status, and their social networks.

Immediately after respondents had completed the baseline questionnaire, the intervention took place if the women agreed to participate in the intervention (all agreed). They were given information about tetanus using flipcharts as well as told how much they could receive if they visited the assigned health clinic within 7 days from the baseline survey. Respondents were then provided a voucher that they could redeem at the assigned health clinic. After the flipchart intervention, all respondents were asked a set of questions regarding flipcharts to capture the change in their perceptions about, understanding of, and attitudes toward vaccination.

The assignment of health clinic for each respondent was determined based on the village where she resided. The voucher indicated the respondent’s name, her unique ID assigned in the project, date of the intervention, name of the health clinic assigned to attend, and the amount of cash compensation to be provided (5, 300, or 800 naira) if she received the vaccine. At each health clinic, the endline questionnaire was administered only for the women who visited the clinic for the vaccination within one week of the baseline survey. Our enumerators verified the actual vaccination received by respondents and asked questions that were mainly about the transportation used to visit the clinic.

Outcomes

The primary outcome measure is the actual vaccine take-up at the assigned clinic. Other outcomes are subjective risk perceptions, which include: the perceived likelihood to contract tetanus; the number of people a respondent thinks die of tetanus out of 100 people; whether a respondent is very worried about tetanus; whether a respondent thinks that tetanus is very bad; whether a respondent thinks that it is very important to be protected from tetanus; and perceived vaccine efficacy. We define the vaccine efficacy as the difference between the hypothetical number of unvaccinated people, out of 100, whom a respondent thinks get tetanus and the number of vaccinated people, out of 100, who get tetanus.

We also use the emotional response, both subjective and objective, to the flipcharts intervention as outcome measures. The subjective measures of the emotional response are respondent’s perception that the flipcharts made her feel very frightened, tense, and nervous. The objective measure of the emotional response is respondent’s heart rate. Using the heart rate monitor, we measure the heartbeat per minute before and after the flipcharts intervention.

Statistical analysis

To evaluate the effect of scared-straight flipcharts on behavioral and perception changes, we estimate:

$y_{ij}=\alpha +{\beta }_{1}ScaredStraigh{t}_{ij}+{\beta }_{2}ScaredStraight\ast ReceivedVaccineBefor{e}_{ij}+{\beta }_{3}ReceivedVaccineBefor{e}_{ij}+X_{ij}\mu +{\upsilon }_j+{\epsilon }_{ij}$

where $y_{ij}$ is a dummy variable that takes 1 if a woman i in village j receives a tetanus vaccine; ScaredStraight = 1 if a woman i is assigned to the treatment group who were shown the scared-straight flipcharts; $ReceivedVaccineBefore$=1 if a woman i received a tetanus vaccine prior to our intervention. Although the dependent variable is a dummy variable, we use OLS regression because we estimate the interaction term, which is difficult to interpret using logistic regression. We control for various characteristics such as woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic. We control for the village fixed effects ${\upsilon }_j$. We also show results with health facility fixed effects because all the women in a village were instructed to visit the same health clinic to receive the tetanus vaccination. Results are qualitatively consistent with both fixed effects.

To evaluate the potentially heterogeneous effects of the scared-straight intervention according to the cash incentives offered to respondents, we estimate:

$y_{ij}=\alpha +{\beta }_{1}ScaredStraigh{t}_{ij}+{\beta }_{2}ScaredStraight\ast C{300}_{ij}+{\beta }_{2}ScaredStraight\ast C{800}_{ij}+X_{ij}\mu +{\upsilon }_j+{\epsilon }_{ij}$

for women with and without experience of tetanus vaccination, separately.

Results

Descriptive statistics and balance tests

The analysis is based on 1,660 women aged 15–35 years old for whom information about basic baseline characteristics and GPS coordinates are available, of which 987 women never received tetanus vaccine before, and the remaining 673 received the vaccine previously. Among 987 women without vaccination experience, 469 of them (47.5%) received the scared-straight intervention. Among 673 women with the experience, 313 women (46.5%) received the scared-straight intervention.

Respondents were on average 25 years old, 50% were Muslim and the other half were Christian. About half had not received any education, 24.8% had primary school education, and 26.7% had secondary school education or more. Many respondents (44.1%) were engaged in paid work such as selling agricultural produce, and the average amount of household earnings per capita in the previous month was about 5,850 naira (approximately 39 US dollars).

Fifteen percent had never been married, around 18% were pregnant at the time of the baseline survey, and about 77% had at least one child. The majority of respondents, 73.4%, had previously visited the health clinic to which they were assigned in this study. The mean distance from respondents’ houses to the assigned clinic was 1.7 kilometers; around 47% of respondents lived within 1.5 kilometers of the clinics. The mean one-way transportation costs to the clinic were around 125 naira.

More than one third of respondents (37.9%) thought that they were likely to contract tetanus; on average, respondents thought that 30 people out of 100 would die of tetanus, and substantial proportions of respondents (35–50%) felt worried about tetanus, thought that tetanus was bad, and believed that it was important to get protected from tetanus. On average, respondents thought that 22 people out of 100 could be saved from tetanus with vaccines (vaccine efficacy). The mean baseline heart rate was very high, 86.9 beats per minute.

This study focuses on the differential effect of salient information according to experience of tetanus vaccination. Appendix 2 shows the descriptive statistics according to experience of tetanus vaccination. Out of the total sample, 40.5% had received tetanus-toxoid vaccination at least once before the study. Women without vaccination experience were significantly younger, less likely to be married, less likely to have children, and less likely to have visited the health clinic. They were less likely to have a paid job, but higher household income per capita.

In terms of risk perceptions, we find that women without vaccination experience had consistently lower risk perceptions than women who had received vaccinations (Appendix 2); they were less likely to perceive the likelihood of the contraction risk of tetanus and less likely to perceive the severity of tetanus.

Table 1 shows the balance test of baseline characteristics and risk perceptions between women who were shown the scared-straight flipcharts and women who were shown the control flipcharts. Overall, the results indicate that the randomization performed well. Most variables are balanced across interventions including the prior tetanus vaccination experience except two variables. Respondents who received the scared-straight intervention were more likely to be pregnant, and more likely to be worried about tetanus. Appendix 3 shows the balance tests among the subgroup sample. There are some variables that are not balanced in each subsample. We control for these variables for regression analysis.

Table 1. Balance tests (mean)

	Scared-straight flipcharts	Control flipcharts	Difference (p = value)
	(1)	(2)	(3)
Baseline characteristics
Age	25.231	25.271	0.899
Muslim	0.483	0.511	0.118
No education	0.478	0.489	0.632
Primary school	0.241	0.256	0.490
Secondary school or more	0.278	0.255	0.269
Has paid work	0.446	0.435	0.666
Earning per capita per day	5615.647	6110.985	0.234
Not married	0.153	0.148	0.770
Pregnant	0.165	0.198	0.094
Have children	0.781	0.754	0.204
Ever used clinic	0.736	0.732	0.820
Distance to health clinic (km)	1.717	1.689	0.132
Transportation costs (both way)	128.395	121.805	0.466
Opportunity costs (both way)	4.229	4.052	0.792
Received tetanus vaccine before	0.407	0.404	0.895
Perception
Likely to contract tetanus	0.366	0.394	0.233
Number of people die of tetanus (0–100)	30.859	29.534	0.249
Worried about tetanus	0.331	0.379	0.035
Tetanus is bad	0.427	0.45	0.344
Important to protect from tetanus	0.476	0.515	0.105
Vaccine efficacy	22.899	21.569	0.273
Heart rate	86.414	87.518	0.130

Notes: These are based on the analysis sample of 1,660 women. “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Worried about tetanus,” “Tetanus is bad,” and “Important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). The estimates are with village fixed effects.

Effect of scared-straight flipcharts on vaccination take-up

Among women who had never received a tetanus vaccination before the intervention, showing scared-straight flipcharts backfired: it decreased the likelihood of receiving the tetanus vaccine at the clinic by 4.0–6.1 percentage points (Table 2). On the other hand, we found that the effect of priming was insignificant among women with experience of vaccines.

Table 2. Priming effect by vaccine experience (OLS regression)

	Received Vaccine
	Regression with covariates (1)	Regression with covariates and health facility fixed effects (2)	Regression with covariates and village fixed effects (3)
Scared-straight flipcharts	−0.055**	−0.061**	−0.040*
	(0.026)	(0.023)	(0.023)
Scared-straight * Received vaccine before	0.088*	0.095**	0.051
	(0.045)	(0.043)	(0.040)
Received vaccine before	−0.086***	−0.052*	−0.010
	(0.030)	(0.029)	(0.030)
r2	0.077	0.062	0.024
p-values of F test: (Scared-straight + Scared-straight * Received vaccine before = 0)	0.379	0.358	0.758

Notes: These are based on the analysis sample of 1,524 women after dropping observations with missing values. Robust standard errors clustered by villages (80 villages) are in parentheses. The mean of dependent variable under control group is 0.778. Covariates are proportion of respondents with scared-straight flipcharts in village (column 1 and 2), woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, past utilization of the assigned health clinic, and the belief measurements before the intervention listed in Table 1 (Perceptions). Control mean of dependent variable is the mean under control flipcharts and Received vaccine before = 0. * significant at 10%; ** significant at 5%; *** significant at 1%

Table 3 shows the effect of the scared-straight intervention according to the cash incentives offered to respondents. Among women without experience of the vaccination, the negative effect of the scared-straight intervention is the most prevalent among women who received the lowest amount of cash incentive (Table 3 column 2); showing the scared-straight flipcharts decreased the likelihood of the vaccine take-up by 7.2 percentage points if women were offered 5 naira and if they had never received the tetanus vaccine before. On the other hand, we did not observe any negative effect of the scared-straight intervention among women who were offered medium or the highest amount of cash incentives. We need to be cautious, however, to interpret these results because the marginal effects of the scared-straight intervention were not statistically different according to the amount of cash incentives.

Table 3. Priming effect by past vaccine experience and cash incentives (OLS regression)

Received Vaccine
	Never received tetanus vaccine before N = 987		Received tetanus vaccine before N = 673
Sample:	Regression without heterogeneous effects (1)	Regression with heterogenous effects (2)	Regression without heterogenous effects (3)	Regression with heterogeneous effects (4)
Scared-straight flipcharts	−0.050**	−0.072*	−0.021	0.053
	(0.022)	(0.039)	(0.036)	(0.069)
C300 * Scared-straight		0.029		−0.092
		(0.060)		(0.082)
C800 * Scared-straight		0.052		−0.143*
		(0.048)		(0.084)
C300		0.178***		0.269***
		(0.042)		(0.053)
C800		0.256***		0.326***
		(0.041)		(0.056)
r2	0.030	0.126	0.021	0.119
p-values of F test: Scared-straight + Scared-straight * C300 = 0		0.262		0.469
Scared-straight + Scared-straight * C800 = 0		0.525		0.094
p-values (equality): (2) = (4) for Scared-straight flipcharts			0.467
(2) = (4) for Scared-straight flipcharts				0.092
(2) = (4) for Scared-straight + C300 * Scared-straight				0.953
(2) = (4) for Scared-straight + C800 * Scared-straight				0.211
Mean of dependent var under control	0.778	0.629	0.706	0.500

Notes: Robust standard errors clustered by villages (80 villages) are in parentheses. Covariates are woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic. All regressions include covariates and village fixed effects. Control mean of dependent variable is the mean under control flipcharts and C5. * significant at 10%; ** significant at 5%; *** significant at 1%

The null effect of scared-straight flipcharts among women with experience of the vaccination is persistent, except among women who were offered the highest amount of cash incentives. If women with vaccination experience received the highest cash incentives (C800), then the priming intervention decreased the likelihood of vaccination by 14.3 percentage points more than when the priming intervention came with the lowest cash incentives, C5 (Table 3 column 4). The marginal effect of scared-straight flipcharts among women with the vaccination experience, however, was not statistically different from the null effect among women without vaccination experience with the highest cash incentives (Table 3 column 2 vs. 4).

Risk perceptions and emotional arousal

Table 4 shows the effect of the scared-straight intervention on the risk perceptions. We find no differential effect of the intervention on the risk perceptions according to women’s experience of the tetanus vaccine. The scared-straight intervention increased the risk perceptions, regardless of experience of the tetanus vaccination. Among women who had never received the tetanus vaccine, it increased the average number of people whom they thought would die of tetanus by 3.63, and it also increased the likelihood of them worrying about tetanus, of perceiving that tetanus is bad, and of perceiving that it is important to protect themselves from tetanus by 14.6, 15.6, and 13.0 percentage points respectively more than the control flipcharts (Table 4 columns 1–5).

Table 4. Priming effect on risk perceptions (OLS regression)

	Likely to contract tetanus Number of people	Number of people die of tetanus (0–100)	Worried about tetanus	Tetanus is bad	Important to protect from tetanus	Vaccine efficacy
	(1)	(2)	(3)	(4)	(5)	(6)
Scared-straight flipcharts	0.036	3.626**	0.146***	0.156***	0.130***	−0.089
	(0.026)	(1.747)	(0.038)	(0.035)	(0.035)	(1.595)
Scared-straight * Received vaccine before	−0.049	−2.646	−0.008	−0.044	−0.058	−2.665
	(0.042)	(2.284)	(0.051)	(0.043)	(0.039)	(2.775)
Received vaccine before	−0.022	0.313	0.125***	0.148***	0.153***	4.668*
	(0.045)	(2.329)	(0.039)	(0.035)	(0.037)	(2.440)
Adjusted p value for Scared-straight flipcharts (FDR)	0.207	0.062	0.001	0.000	0.001	0.956
Adjusted p value for interaction term (FDR)	0.408	0.408	0.875	0.408	0.408	0.408
N	1524	1523	1524	1524	1524	1520
r2	0.091	0.076	0.148	0.116	0.113	0.093
p values of F test: (Scared-straight + Scared-straight * Received vaccine before = 0)	0.658	0.490	0.001	0.001	0.017	0.233
Village FE	X	X	X	X	X	X
Covariates	X	X	X	X	X	X
Mean of dependent var under control	0.448	36.534	0.486	0.568	0.664	28.975

Notes: These are based on the analysis sample of 1,660 women. Missing observations in each column is due to missing values and invalid numbers in the dependent variable. Robust standard errors clustered by villages (80 villages) are in parentheses. “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Very worried about tetanus,” “Tetanus is very bad,” and “Very important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). Covariates are woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic, and the belief measurements before the intervention listed in Table 1 (Perceptions). Control mean of dependent variable is the mean under control flipcharts and Received vaccine before = 0. * significant at 10%; ** significant at 5%; *** significant at 1%

Among women who had experience of receiving the tetanus vaccine, the effect of the scared-straight intervention on risk perceptions was statistically the same as that among women who had never received the tetanus vaccine. However, the scared-straight intervention increased the risk perceptions among women without vaccine experience more than among women with experience, although differences were statistically insignificant (Table 4).

Table 5 shows the effect of scared-straight intervention on several measurements of emotional responses. Among women who had never received the tetanus vaccine, the scared-straight intervention increased the fear level; it increased the likelihood of being frightened, being tense, and being nervous by 30.6, 33.0, and 31.3 percentage points, respectively more than the control flipcharts (Table 5 columns 1–3). Furthermore, it increased the heart rate by 4.79 beats per minute more than the control flipcharts (column 4). Among women who had experience of the tetanus vaccination, the scared-straight intervention increased the fear level more than the control flipcharts did, and the increase in the fear level among them was larger than among women who had never had tetanus vaccination, although the difference of some measurements of the stated fear was statistically insignificant.

Table 5. Emotional responses to flipcharts (OLS regression)

	Flipcharts made me feel
	Frightened	Tense	Nervous	Heart rate
	(1)	(2)	(3)	(4)
Scared-straight flipcharts	0.306***	0.330***	0.313***	4.793***
	(0.038)	(0.030)	(0.037)	(0.973)
Scared-straight * Received vaccine before	0.071	0.102*	0.112**	3.543***
	(0.058)	(0.053)	(0.053)	(1.315)
Received vaccine before	0.032	0.051	−0.017	−0.367
	(0.045)	(0.042)	(0.036)	(1.168)
Adjusted p value for Scared-straight flipcharts (FDR)	0.000	0.000	0.000	0.000
Adjusted p value for interaction term (FDR)	0.224	0.073	0.073	0.034
N	1651	1651	1650	1442
r2	0.130	0.161	0.144	0.073
p values of F test: (Scared-straight + Scared-straight * Received vaccine before = 0)	0.000	0.000	0.000	0.000
Village FE	X	X	X	X
Covariates	X	X	X	X
Mean of dependent var under control	0.287	0.228	0.298	88.235

Notes: These are based on the analysis sample of 1,660 women. Missing observations in each column is due to missing values and invalid numbers in the dependent variable. Robust standard errors clustered by villages (80 villages) are in parentheses. All the dependent variables are dummy variables which take 1 if a respondent answers “very much” or “much” to the question: “How did you feel about the flipchart you were just shown? Feel frightened, feel tensed, feel nervous, feel uncomfortable?” after the flipcharts intervention. Covariates are woman’s age, age squared, religion (Muslim or not), highest education attained, marital status, pregnancy status, whether she has a child, whether she has a paid work, distance to the health clinic, and past utilization of the assigned health clinic. Control mean of dependent variable is the mean under control flipcharts and Received vaccine before = 0. * significant at 10%; ** significant at 5%; *** significant at 1%

Discussion

We find heterogeneous impacts of the scared-straight intervention. On one hand, women with experience of the tetanus vaccination did not increase their vaccine take-up due to the scared-straight intervention, although it increased their risk perceptions and fear level. However, this result is consistent with our hypothesis; because women with experience of the tetanus vaccination already had a high level of the risk perceptions of tetanus, as we see in Appendix 2, the salient information did not alter their vaccination behavior. Although the intervention increased their risk perceptions and fear level, it did not induce the behavioral change because the baseline level of risk perceptions was already high.

On the other hand, the scared-straight intervention backfired among women without experience of the tetanus vaccination, even though it increased their risk perceptions slightly more than it did for women with experience, and also increased their fear level although less than for women with experience. This result implies the possible low association between risk perceptions and vaccine acceptance, although previous studies have indicated their positive association.^19–21The result that the scared-straight intervention backfired among women without experience of the vaccination is puzzling because the increase in perceived disease risk and stated fear level does not coincide with the vaccine take-up.

As the changes in risk perceptions and emotional arousal did not explain the backfire, we explore other possible mechanisms of the backfire effects. The first possibility is denial. It is possible that women without experience of tetanus vaccination might have responded to the scared-straight flipcharts by denying the information provided because the flipcharts were too frightening. In the psychology literature, this phenomenon is known as denial.^13,14,22 This hypothesis of denial is consistent with results that the scared-straight intervention did not increase the stated fear level or even the heart rate among women without experience as much as it did among women with experience (Table 5). When a woman feels too scared of something, she might stop thinking about it completely, and thus we observe less change in the fear level after the intervention among women without experience of the vaccination than among women with experience.

Table 3 shows the suggestive evidence to support this `denial’ theory. Among women without experience of vaccination, the scared-straight intervention backfired more if they were offered the smallest amount of cash incentives than if they were offered the higher amount of incentives, although the difference is insignificant (Table 3 column 2). Literature in psychology often discusses the possibility of the backfire especially when the fearful messages are not accompanied by efficacy statements.⁹ Efficacy statements provide the solutions which eliminate or mitigate the fearful events. Although our intervention clearly provided the solution, the vaccine take-up, women who were offered the lowest cash incentives might have had low self-efficacy due to budget constraints, in other words, they could not afford to go to receive the vaccine at the health clinic. This low self-efficacy might have led to the denial.

On the other hand, among women with vaccine experience, the scared-straight intervention backfired if they were offered higher cash incentives (Table 3 column 4). This phenomenon might be related to the backfire of monetary incentives. We sometimes observe backfire of monetary incentives in the literature.²³ One potential mechanism through which monetary incentives backfire is signaling. Providing cash incentives to a specific task might imply that the task is not desirable. In African countries where basic health care is not easily available, even free vaccination causes general distrust against vaccination.²⁴ Although we found a strong and positive effect of cash incentives on the vaccine decision itself, the cash incentives might have interacted with information in a negative way. It is possible that women with vaccine experience were neutral about the frightening information because they might have been aware of the information from their experience. However, cash incentives attached to the information might have made them skeptical about vaccination due to the signaling that the cash incentives had.

Another possibility as to why the backfire is not explained by the stated risk perceptions nor by stated fear is that the stated risk perceptions and fear level did not reflect the actual level of risk perceptions and the fear level. However, this possibility is less likely because we captured the objective measure of the response to the fear, the heart rate, and there we found a similar effect of the scared-straight intervention as on the subjective measurement of the fear (Table 5).

Contributions to the literature

This paper makes contributions to the literature on loss-framed messages, fear appeals, and priming and the broad literature on information and behavioral interventions for health in developing countries. First, the paper is one of the first randomized control trials on the effectiveness of fear appeals in a developing country. Most research on fear appeal has been conducted in developed countries. For example, among 127 papers included in the meta-analysis by Tannenbaum et al. (2015),⁹ only a few cover developing countries.²⁵

Second, our study is one of the few experiments to show robust evidence for backfire of fear appeal by measuring the rigorous causal effect of fear appeals on actual vaccination behaviors instead of hypothetical behaviors, which are commonly examined in extant studies,⁹ as well as looking at heart rate, an objective measure of emotional response.²⁶

Third, our paper is the first, to our knowledge, to combine the priming intervention with cash incentives to reveal potential underlying mechanisms. In particular, we demonstrate that not only the effectiveness of fear appeal, but also its interactions with incentive intervention can be different according to the baseline risk perceptions of disease.

Limitations

The findings of the paper have limited external validity because of the non-representative sample, as well as the survey instrument that was not tested in other locations. About half of respondents in our study had no educational attainment. Although we provided respondents with oral explanation along with the written explanation of control flipcharts, their low literacy might have lowered their understanding of flipcharts. The (potential) lack of decision-making power among respondents might have limited their access to vaccination. This is unlikely to bias the estimated treatment effects because respondents’ decision-making power should be balanced across the treatment arms; this might have hindered their overall uptake of vaccination, however. We did not report the effect of baseline vaccine beliefs on the vaccine uptake, primarily because we found no difference in the effect of scared-straight flipcharts according to the vaccine beliefs. Although we have suggestive evidence that the backfire of the intervention is due to denial, we do not have direct evidence for this mechanism. Future work should explore more on mechanisms through which information interventions can influence health behaviors among the vulnerable population.

Conclusion

This paper examines the effect of salient information about the disease severity on the tetanus vaccine take-up among women of childbearing age in rural northeastern Nigeria.

We find the differential effect of salient information through the scared-straight intervention according to whether respondents had previous experience of tetanus vaccination or not. While the vaccine take-up was not affected by the salient information among women with experiences of tetanus vaccination, the scared-straight intervention decreased the likelihood of the vaccine take-up among women who had never received the tetanus vaccination.

This backfire effect of the scared-straight intervention is not explained by the changes in risk perceptions and emotional arousal because this salient information intervention increased the risk perceptions of tetanus and induced emotional response such as stated fear and the increase in the heart rate among all the respondents, regardless of their experiences of vaccination.

The potential reason of this backfire effect could be denial. Our results suggests that the salient information about the disease severity using the fear appeals backfired among women without any vaccine experience, with low risk perceptions about disease, because among respondents who had low self-efficacy the recipients of the information responded to the intervention by denying the information. Respondents might have the low self-efficacy if they faced budget constraints that prevented them from achieving the recommended outcomes.

Overall, we do not recommend the use of the scared-straight tactic to aim for the improved take-up of vaccination among our target; women who never had experiences of vaccination, in developing countries where people might face budget constraints for achieving desirable behaviors. The tactic was not effective even among women who had experience of vaccination.

Ethics approval

Ethics approval was obtained on November 2012 from University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board (HUM00063832). Oral and written informed consents were obtained from all the respondents prior to their participation in the study.

Acknowledgments

We are grateful to Abdullahi Belel and the Adamawa State Primary Health Care Development Agency for their cooperation and support throughout the project implementation. We give special thanks to Benjamin Fintan and the field team who devoted themselves to the project. This project was supported with research grants from the Japan Society for the Promotion of Science.

Disclosure of potential conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

SUPPLEMENTARY MATERIAL

Supplemental data for this article can be accessed on the publisher’s website.

Additional information

Funding

This project was supported with research grants from the Institute for Research on Women & Gender, the Rackham Graduate School, the Department of Afroamerican and African Studies, the Department of Economics, and the Center for the Education of Women at the University of Michigan; the Japan Society for the Promotion of Science [22223003]; and Yamada Scholarship Foundation.

Appendix 1.

Comparison of Flipcharts (Example)

Table

Scared-straight flipchart	Control flipchart
Example
-Tetanus is very dangerous (esp for babies)	-Tetanus is very dangerous (esp for babies)
-Typical symptoms of tetanus	-Typical symptoms of tetanus
(1) Severe pain	(1) Severe pain
(2) Muscle spasm	(2) Muscle spasm

Source:Wikimedia Commons

Appendix 2.

Differences in Characteristics by Vaccine Experience (Mean)

Table

	Never received vaccine before (N = 987)	Received vaccine before (N = 673)	Difference (p = value)
	(1)	(2)	(3)
Baseline characteristics
Age	24.552	26.272	0.000
Muslim	0.513	0.473	0.312
No education	0.480	0.487	0.822
Primary school	0.240	0.260	0.351
Secondary school or more	0.277	0.253	0.346
Has paid work	0.398	0.504	0.001
Earning per capita per day	6576.643	4781.844	0.001
Not married	0.198	0.082	0.000
Pregnant	0.170	0.195	0.170
Have children	0.675	0.905	0.000
Ever used clinic	0.674	0.822	0.000
Distance to health clinic (km)	1.711	1.694	0.880
Perception
Likely to contract tetanus	0.333	0.446	0.003
Number of people die of tetanus (0–100)	29.035	31.999	0.099
Worried about tetanus	0.260	0.492	0.000
Tetanus is bad	0.372	0.533	0.001
Important to protect from tetanus	0.413	0.612	0.000
Vaccine efficacy	20.928	24.250	0.133
Heart rate	86.664	87.321	0.418

Notes: These are based on the analysis sample of 1,660 women. “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Very worried about tetanus,” “Tetanus is very bad,” and “Very important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). The estimates are with village fixed effects, with clustered standard error (village level).

Appendix 3.

Balance tests (Mean)

Table

	Never received tetanus vaccine before (N = 987)			Received tetanus vaccine before (N = 673)
	Control flipcharts	Difference	p value	Control flipcharts	Difference	p value
	(1)	(2)	(3)	(4)	(5)	(6)
Baseline characteristics
Age	24.508	0.094	0.825	26.063	0.449	0.340
Muslim	0.496	0.036	0.152	0.459	0.028	0.320
No Education	0.485	−0.009	0.769	0.464	0.049	0.184
Primary school	0.225	0.032	0.276	0.266	−0.012	0.739
Secondary school or more	0.285	−0.018	0.533	0.270	−0.038	0.263
Has paid work	0.397	0.001	0.965	0.517	−0.029	0.439
Earning per capita per day	6382.979	407.563	0.514	4365.560	895.077	0.068
Not married	0.201	−0.006	0.815	0.094	−0.027	0.226
Pregnant	0.170	−0.001	0.983	0.152	0.093	0.005
Have children	0.680	−0.010	0.746	0.919	−0.030	0.212
Ever used clinic	0.671	0.005	0.846	0.828	−0.014	0.582
Distance to health clinic (km)	1.720	−0.019	0.428	1.706	−0.025	0.423
Transportation costs (both way)	131.957	−1.353	0.914	121.268	−10.393	0.415
Opportunity costs (both way)	4.251	−0.441	0.654	3.956	0.733	0.400
Received tetanus vaccine before	0.000	0.000		1.000	0.000
Perception
Likely to contract tetanus	0.323	0.021	0.483	0.434	0.024	0.522
Number of people die of tetanus	29.504	−0.991	0.503	32.498	−1.074	0.564
Worried about tetanus	0.233	0.057	0.048	0.481	0.023	0.537
Tetanus is bad	0.359	0.028	0.380	0.521	0.027	0.446
Important to protect from tetanus	0.383	0.065	0.046	0.600	0.027	0.456
Vaccine efficacy	22.246	−2.773	0.059	23.836	0.891	0.671
Heart rate	86.594	0.150	0.878	86.484	1.826	0.119

Notes: These are based on the analysis sample of 987 women who never received tetanus vaccine before (columns 1–3) and 673 women who received vaccine before (columns 4–6). “Likely to contract tetanus” is a binary variable; “Number of people who die of tetanus” is the number of people out of 100; “Very worried about tetanus,” “Tetanus is very bad,” and “Very important to be protected from tetanus” are binary variables. “Vaccine efficacy” is the difference between the hypothetical number of unvaccinated people whom each respondent thinks get tetanus and the number of vaccinated people who get tetanus (range: −100 to 100). The estimates are with village fixed effects.