Entertainment-Education for Better Health: Insights from a Field Experiment in India

Abstract

Entertainment-education has been touted as a potent delivery channel for health education campaigns. Yet, there is little evidence of its causal effects. This paper aims to fill the gap in the literature by using a field experiment in India to study two questions on the efficacy of health entertainment-education. First, can health entertainment-education, particularly through films that show role models and draw on emotions, lead to lasting, positive change in health knowledge and behavior? Second, can financial incentives for ex-post health literacy boost the effectiveness of health entertainment-education? The results show that health entertainment-education successfully increased health knowledge (e.g. knowledge about cleanliness and hygiene) by 16 percent. These gains persist almost one year later, although there were no observed impacts on health behaviors. Further, financial incentives do not appear to have any effects. These insights contribute to our knowledge of what works for health education in low-income settings, so that future education campaigns can be crafted with more meaningful impact.

Keywords:

• entertainment-education
• financial incentives
• health
• randomized controlled trial
• India

1. Introduction

Global public health has improved significantly in recent decades: child mortality has declined, maternal wellness has increased, and HIV/AIDS has become less prevalent (United Nations, 2019). Nevertheless, ensuring healthy lives for all—the third of the United Nations’ Sustainable Development Goals (SDGs) by 2030—continues to be a pressing challenge. For example, in India, the under age five mortality rate stands at 50 per 1,000 live births compared to an SDG target of 25. With 2.1 million people living with HIV, the country likewise has the third largest HIV epidemic in the world (Government of India, 2019).

One popular public policy tool to boost health outcomes is education interventions. Health education and information campaigns have long been widely used to promote healthy behaviors and prevent diseases. Yet, as the World Bank notes, “many of these campaigns are unconvincing, lack inspiring narratives, and are communicated through outmoded and uninteresting outlets such as billboards and leaflets” (Thewarapperuma, 2015, para. 2). It is perhaps unsurprising, then, that systematic reviews of such campaigns reveal zero or only modest effects on health behaviors, particularly in the long term. These null effects are evident across a range of health domains, including hand washing, sanitation, and sexual behavior (e.g. De Buck et al., 2017; McCoy et al., 2010).

Against this backdrop, entertainment-education has been touted as a potent and cost-effective channel for delivering health information. Entertainment-education refers to ‘the process of purposely designing and implementing a media message to both entertain and educate, in order to increase audience members’ knowledge about an educational issue, create favorable attitudes, shift social norms, and change overt behavior’ (Singhal, Cody, Rogers, & Sabido, 2003, p. 5). While much has been written about entertainment-education in the academic literature, there is surprisingly little evidence on its causal effects (World Bank, 2015). Especially in the context of public health, only a small number of studies have examined entertainment-education using rigorous methods such as randomized controlled trials (e.g. Banerjee, La Ferrara, & Orozco, 2020; Orozco-Olvera, Shen, & Cluver, 2019).

This paper aims to fill the literature gap by using a field experiment in India to study two related questions on health entertainment-education. First, can health entertainment-education, particularly through films, lead to lasting, positive change in health knowledge and behavior? Drawing on social learning and drama theory, health entertainment-education films show the main characters as positive role models, with dramatic narratives that seek to elicit emotional responses from viewers. Since entertainment-education is designed to affect both hearts and minds, this study tests whether it may have enduring effects over the long term.

Second, can monetary incentives for ex-post health literacy boost the effectiveness of health entertainment-education? In theory, financial rewards in an educational setting may be important to activate motivation and effort to learn, especially if self-control problems (e.g. procrastination) or lack of interest prevent participants from fully benefitting from the education program. At the same time, if the entertainment value of the films fully captures the interest and effort of the audience, financial rewards may not be necessary and may even crowd out intrinsic motivation. Hence, whether cash incentives have a role to play in entertainment-education is an open question.

The results show that entertainment-education successfully increased short-term health knowledge scores (e.g. knowledge about cleanliness and hygiene, breastfeeding, whether AIDS is curable) by 8 percentage points. This estimate is statistically and economically significant, representing a 16 percent change compared to the control group mean. Importantly, these knowledge gains persist almost one year later and exhibit little decay. Nevertheless, health entertainment-education had limited effects on health behaviors. The cash incentives for performance on a post-program health knowledge test likewise had little impact on health knowledge and behavior. Taken together, these insights contribute to our understanding of what works for health education in developing countries, so that future education campaigns can be crafted with more meaningful impact.

2. Experiment design

I conduct a randomized field experiment among urban poor individuals in Ahmedabad, a large city in Gujarat, India. The project was implemented in partnership with Saath (a non-government organization working on urban development) and targeted neighborhoods where Saath operated. There are three study arms: a group assigned to only health entertainment-education (henceforth, HEE); a group assigned to health entertainment education with cash incentives for health knowledge test performance (henceforth, HEEC); and a control group. Below, I discuss the study sample, interventions, and data collection in greater detail.

2.1. Sample recruitment and summary statistics

The sample was recruited as follows. After obtaining a geographic listing of all households in a neighborhood, surveyors visited approximately every fourth household on the list. In each visited household, the household members self-selected the main decision-maker, earning member, or earning member’s spouse as the respondent. The respondent was then invited to participate in a ‘life skills’ program, described as a program to help improve their lives. The content of the program was not mentioned at this point. The surveyor also informed respondents about where the program would be held and the time commitment required (i.e. five consecutive weekly sessions, two to three hours per session). Should the respondent agree to join, the surveyor noted the respondent’s contact information. The surveyor also recorded the days and times that the respondent can attend the program.¹ Around 20 percent of invited households declined to participate in the study.

The study sample comprises 887 respondents. Their baseline characteristics are reported in Table 1. Fifty-eight percent are women, almost all are married, and the average age is 38 years old. The sample has low educational attainment: only 45 and 3 percent completed elementary and secondary school, respectively. Additionally, respondents’ baseline health knowledge is far from perfect (e.g. only 39 percent are aware that AIDS is incurable), and healthy behaviors are not very evident (e.g. only 1 percent of households report drinking filtered or boiled water).

Table 1. Baseline summary statistics

	Full sample (N = 887)			Means by study arm
	Mean	SD	CV	HEE (N = 218)	HEEC (N = 224)	Control (N = 445)	F-test p-value
Panel A. Household (HH) variables
HH size	5.90	2.56	0.43	5.81	5.98	5.89	0.999
HH monthly income (Rs.)	7024.09	5096.80	0.73	6533.33	6944.51	7304.57	0.147
HH monthly income per capita (Rs.)	1247.38	773.70	0.62	1193.02	1228.34	1283.60	0.165
HH has TV	0.79	0.41		0.80	0.81	0.78	0.487
HH has bicycle	0.41	0.49		0.40	0.38	0.43	0.419
HH has scooter/motor	0.19	0.40		0.22	0.21	0.17	0.110
HH has computer	0.02	0.12		0.01	0.01	0.02	0.657
HH has mobile phone	0.84	0.37		0.82	0.83	0.84	0.419
HH has nonfarm enterprise	0.26	0.44		0.25	0.29	0.24	0.334
HH has water connection	0.78	0.42		0.82	0.73	0.78	0.156
Panel B. Respondent Variables
Woman	0.58	0.49		0.56	0.60	0.59
Age	38.47	9.10	0.24	38.58	39.14	38.08	0.467
Married	0.98	0.14		1.00	0.97	0.98	0.055*
Hindu	0.82	0.39		0.79	0.84	0.82	0.340
Completed elementary school	0.45	0.50		0.48	0.43	0.45	0.953
Completed secondary school	0.03	0.17		0.04	0.03	0.03	0.773
Microfinance client	0.48	0.50		0.48	0.49	0.48
Has inconsistent time preferences	0.49	0.50		0.46	0.50	0.50	0.642
Is risk averse	0.17	0.38		0.15	0.20	0.17	0.719
Has any income during last 12 months	0.75	0.43		0.78	0.74	0.73	0.505
Panel C. Health Variables
Knows AIDS is not curable	0.39	0.49		0.40	0.39	0.38	0.945
Knows nightblindness is curable	0.28	0.45		0.25	0.33	0.28	0.173
Knows should wash hands after defecation	0.92	0.27		0.93	0.92	0.92	0.717
Drinks filtered or boiled water	0.01	0.09		0.01	0.01	0.01	0.827
Uses a water vessel with tap	0.03	0.16		0.04	0.02	0.03	0.752

Notes: Data from baseline survey. In Column 3, the Coefficient of Variation (CV) is the ratio of the standard deviation to the mean and is calculated for continuous variables. The last column reports the p-value of the joint hypothesis test $H_0:{\beta }_1=0,{\beta }_2=0,$ where the regression is the same specification used for the impact analysis (Equation (1)(1) $$\begin{array}{cc}{Y}_{\mathit{\text{is}}}={\beta }_0\hfill & +{\beta }_1{\mathit{\text{HEE}}}_{\mathit{\text{is}}}+{\beta }_2{\mathit{\text{HEEC}}}_{\mathit{\text{is}}}+{\varphi }_s+{ϵ}_{\mathit{\text{is}}},\hfill \end{array}$$(1) ) and the dependent variable is a baseline outcome. The p-values are not reported for two baseline outcomes—i.e. the dummy for woman and the dummy for whether the respondent is a microfinance client—because they are contained in the strata fixed effects in the regression. ***p < 0.01, **p < 0.05, *p < 0.10.

To understand how the study sample compares to the average household in Ahmedabad subdistrict, I use information from the 2011 Census of India. The census reports the proportion of households in the subdistrict who have a TV (0.84), bicycle (0.49), scooter or motorcycle (0.53), computer (0.22), and mobile phone (0.63). Contrasted with the study’s baseline data, these census statistics illustrate that the study sample covers some of the poorest families. Asset ownership is generally lower in the study sample than in the overall subdistrict; for example, the proportion of sample households with a scooter or motorcycle is only 0.19 (Table 1, Panel A). However, mobile phone ownership is much higher in the sample, with a proportion of 0.84. These patterns align with the project’s collaboration with Saath for sample recruitment. Indeed, Saath works closely with disadvantaged areas, and they typically contact beneficiaries of their projects via mobile phones.

The study sample was randomly assigned into three study arms: 25 percent to HEE, 25 percent to HEEC, and 50 percent to control.² Randomization was carried out at the individual level and stratified by whether the respondent was a Saath microfinance client, along with gender and neighborhood. The last column of Table 1 shows that randomization was successful. The HEE, HEEC, and control groups are balanced across different household and health characteristics (Panels A and C). Furthermore, although the respondent for the household was self-selected, respondent characteristics are largely balanced (Panel B).³ Only one variable, marital status, exhibits imbalance. In the next two subsections, I describe the health entertainment-education intervention provided to HEE and HEEC, as well as the cash incentives intervention given to HEEC.

2.2. Health entertainment-education

The first intervention explores the potential of entertainment-education to influence health knowledge and outcomes. HEE and HEEC respondents were assigned to watch health entertainment-education films at a nearby program center, once a week for five weeks (the control study arm was assigned to a placebo film program, discussed below). Each health film screening focused on one of the following topics: (1) cleanliness and hygiene; (2) midwives; (3) maternal and child health; (4) condoms, AIDS, and syphilis; and (5) night blindness (nyctalopia). These films were produced by either UNICEF South Asia or the Voluntary Health Association of India (VHAI), a local non-profit organization supporting grassroots health and development programs.

Supplementary Material A gives an overview of the program modules and film content. UNICEF and VHAI deliberately developed the films to simultaneously entertain and educate audiences about healthy behaviors, diseases, and preventive healthcare. Using storytelling with relatable and empathetic characters, the films aimed to immerse viewers into the narrative, which weaved in health messages as a central component of the plot. These messages, in turn, were designed to increase viewers’ knowledge about health issues and change their health attitudes, beliefs, and practices.

To illustrate the films’ approach to entertainment-education, consider the module for maternal and child health, which includes a video called ‘Saving a Life.’ This film intends to educate viewers about diarrhea, a leading cause of death in children under five years old (WHO., 2017). The film is part of UNICEF’s Meena series, which chronicles the adventures of its eponymous protagonist, a young South Asian girl who lives in the village with her family and pet parrot. Research in public health communication has heralded Meena as a successful example of entertainment education (e.g. McKee, Aghi, Carnegie, & Shahzadi, 2004), though no existing study has identified its causal effects.

In ‘Saving a Life,’ Meena’s auntie arrives for a visit. Later that evening, a terrible storm came, and Meena’s baby sister, Rani, had diarrhea. Auntie remembered her own child who died of diarrhea at one year old. No one in the household knew what to do, and the health worker lived far away. Meena suggested to seek the advice of their teacher. She braved thunder and heavy rain to go to their teacher’s house. The teacher explained that diarrhea can be dangerous for babies if they lose a lot of water, so they should be fed lots of liquid. Meena brings this information back to her parents, and they started giving liquids to baby Rani. Auntie heavily doubted that this is the right thing to do. The next day, baby Rani was better, and the storm had stopped. Meena is recognized for her efforts, and even Auntie had to agree that Meena saved Rani’s life.

Several theories from the pedagogy, sociology, psychology, and communication literatures explain why entertainment-education can influence health literacy and behavior (Sood, Menard, & Witte, 2003). The narratives of the films in this study draw on two theoretical constructs: social learning and drama theory. Social learning emphasizes modeling and suggests that learning can take place by observing others and the consequences of their behavior (Bandura, 1977). In line with this theory, the films depict the main characters such as Meena as positive role models: they demonstrate healthy values and behavior, and they receive praise. Meanwhile, characters such as Meena’s auntie serve as transitional models, representing the audience: they begin with doubt but eventually become convinced. By playing out this scenario, the films can guide viewers and cultivate their self-efficacy, that they too can improve their own health knowledge and actions (Bandura, 2003).

Next, drama theory highlights the importance of evoking viewers’ emotions through confrontation, drama, conflict, and resolution (Kincaid, 2002). The theory holds that an emotional response builds connections with the characters. This emotional connection allows the audience to re-conceptualize the problem in the drama into their own lives, thereby inducing them to resolve the problem in a similar manner as depicted in the story. Emotion is central in all the films used in this study: for example, in the case of Meena saving baby Rani’s life, the narrative is dramatic because it involves potential harm to a child. Since all films were developed locally, the contexts are familiar and culturally relevant to the audience. The on-screen characters can seem like friends and neighbors, making it easy for viewers to empathize with them.

I now turn to the placebo program provided to the control arm. Like health entertainment-education, this program consisted of five weekly sessions of two to three hours each. However, the placebo films dealt with themes unrelated to health—i.e. financial planning and services, such as budgeting.⁴ The films were produced by the project team using standard materials, and importantly, were not designed using entertainment-education (e.g. the videos followed a lecture style).

In practice, the health and placebo film programs were implemented with the exact same format and logistics. To watch the films, participants in all study arms received free transportation to and from the program center. All received Rs. 50 (approximately US$ 1) for each session they attended, as a token of gratitude for participation. The sessions were implemented in a class of around 20 people, where everyone had identical education program assignment (i.e. health or placebo).⁵ The classes convened at the same time weekly together with a skilled facilitator, who answered questions and led an interactive class discussion about the film content.

The control condition in this study involves a placebo (instead of no intervention at all) to hold potential Hawthorne-type effects constant. Such effects may be particularly important in this setting because the training was quite extensive. For example, attending a five-week program could decrease the available time and effort that participants have to obtain clean drinking water for their homes. The placebo therefore helps to ensure that all subjects experienced similar disruption in their everyday lives due to the weekly, 2–3 hour-long meetings. Moreover, the placebo is critical so that differences in outcomes can be attributed to health entertainment-education, rather than any other aspect of intervention delivery (e.g. group discussion, classroom peer effect).

Although my experiment does not directly compare entertainment-education with more traditional health information campaigns (e.g. flyers, billboards), it is worth noting that a large literature has already examined the latter. Systematic reviews have documented that traditional information interventions—from handwashing and sanitation to HIV prevention—have limited or no effects, especially in the long term (e.g. De Buck et al., 2017; McCoy et al., 2010; World Bank, 2018). At the same time, much less is known about the causal impacts of entertainment-education: most existing research relies on observational empirical designs, and few employ randomized controlled trials (World Bank, 2015). Taking stock of this literature, an experiment design with fewer treatment arms and concentrates on health entertainment-education versus a placebo would not only be more cost effective, but also have higher statistical power for a given sample size.

2.3. Cash incentives for health knowledge test performance

The HEEC study arm was assigned to a second intervention: a cash incentive based on their score in a health knowledge test three weeks after the final film screening. Comparing HEE and HEEC then tests whether monetary rewards can enhance the effectiveness of health entertainment-education. The incentive amount was Rs. 10 (US$ 0.20) for each correct answer. There were eight questions, so the maximum possible reward was Rs. 80 (US$ 1.60). Such an amount is not trivial in this context: it represents about half of the average daily wage of casual urban workers during the study period (ILO., 2018). Hence, to control for potential income effects, non-HEEC respondents received the same cash incentive, but for correct answers to test questions unrelated to health.⁶

The use of cash incentives in this study relates closely to two strands of economics research. The first is the literature on conditional cash transfers (CCTs), a social protection program widely implemented in many low- and middle-income countries since the 1990s (Parker & Todd, 2017). In CCTs for human capital accumulation, parents receive financial payments if they invest in their children’s education. CCTs have been shown to be highly effective at increasing school enrollment, attendance, and completion (Baird, Ferreira, Özler, & Woolcock, 2014; Fiszbein & Schady, 2009; García & Saavedra, 2022). However, there is less evidence on their impact on test scores (Baird et al., 2014; Snilstveit et al., 2015), and the results are mixed (Baird, McIntosh, & Özler, 2019; Millán, Barham, Macours, Maluccio, & Stampini, 2019). Prior research has suggested that the mixed findings may be due to the lack of teachers and other inputs to translate higher attendance into learning (García & Saavedra, 2022). By examining learning impacts in a context with a well-functioning education supply side, the present paper helps to expand the evidence base on CCTs.

The second relevant literature strand concerns financial incentives in the economics of education (e.g. Angrist, Bettinger, Bloom, King, & Kremer, 2002; Fryer, 2011; Guryan, Kim, & Park, 2016; Jackson, 2010). Like studies of CCTs, the effect of monetary rewards given to students for academic performance is mixed, across the primary, secondary, and post-secondary levels (Gneezy, Meier, & Rey-Biel, 2011). For instance, Bettinger (2012) shows that among elementary school students in the US, cash incentives for standardized test performance led to higher scores for math but not reading, science, and social science. Angrist and Lavy (2009) find that in a sample of Israeli high school pupils, a financial award positively impacted passing rates in a matriculation exam in the subsample of girls alone. Studying college students, Leuven, Oosterbeek, and Van der Klaauw (2010) report that a large incentive for successful completion of first-year requirements had statistically insignificant effects in the full sample. The impact of financial incentives on student achievement thus appears to be context dependent. In this study, I aim to shed light on the efficacy of cash rewards in a health entertainment-education setting.

Why might financial incentives improve health knowledge test scores? The related literatures on CCTs and economics of education highlight three behavioral constraints that cash rewards may help overcome: future discounting, lack of information, and low motivation. As Baird et al. (2014) explain, two arguments for attaching conditions to CCTs are that parents heavily discount future consumption, or they are ill-informed about the returns to schooling. This reasoning applies to the present study as well: participants may not invest enough effort to learn from the program because they are too impatient to work for benefits that do not materialize immediately (e.g. improved health), or they lack information about the value of better health knowledge. It is also possible that even with the entertainment-education format, participants have insufficient interest in the films and fail to exert optimal effort to learn from the program. In this case, financial rewards can provide extra motivation to learn, thereby increasing effort (e.g. Kremer, Miguel, & Thornton, 2009).

From a practical perspective, the cash rewards studied in this paper can be implemented by organizations such as VHAI, which produced several of the films in the health entertainment-education program, or Saath, an urban development group and the project’s implementation partner. Providing financial incentives for test performance is in line with many CCT programs around the world: a recent review by García and Saavedra (2022) finds that 40 percent of CCTs include a requirement for grade promotion or achievement. Ultimately, the goal of training programs at VHAI, Saath, and other comparable institutions is to impart knowledge, not just ensure attendance. The present study contributes insights into whether cash incentives can help achieve this objective.

Finally, it is important to mention three points that influence the interpretation of the cash incentives. First, it may be that the rewards were not large enough to encourage additional effort: all participants already received Rs. 50 (US$ 1) for each attended session (see Section 2.2), and the cash incentives only added Rs. 80 (US$ 1.60) for learning from the health films. Moreover, the sample consists only of respondents who agreed to participate in a ‘life skills’ program (see Section 2.1), and may have already had high interest and motivation to learn. Second, all participants were discreetly informed of their cash incentives assignment during the first program session, and were reminded at the start of the remaining sessions.⁷ While it is possible that these reminders destroy intrinsic motivation, there is no evidence to suggest that this is the case.⁸ And third, since HEE and HEEC participants were present in the same class, there may be spillover effects. Nevertheless, the direction of the bias from these spillovers is unclear (e.g. HEEC participants may have been more or less motivated in knowing that their neighbors are not receiving the same cash incentives).

2.4. Survey data collection, implementation, and attrition

Original data were collected from study subjects in three survey rounds: (1) a baseline prior to the film screenings; (2) a health knowledge test three weeks after the final film session (hereafter, midline); (3) an endline approximately 10 months after the final session, to measure health knowledge and behavior. All survey questions were designed to be linked directly to the health entertainment-education content. Health knowledge was measured through simple questions such as ‘True/False: Water strained through a cloth is safe for drinking.’ Similarly, health behaviors were measured through self-reported indicators, such as whether the respondent has been tested for AIDS. Supplementary Materials B and C list the health knowledge and behavior questions, respectively.

To facilitate field work logistics, the execution of the study was divided into four waves. Participants were recruited from neighborhoods that were mutually exclusive across waves. Hence, each wave represents a separate iteration of participant recruitment, baseline, randomization, film screenings, midline, and endline (e.g. Wave 1 was completed from recruitment to endline, then Wave 2 was carried out). These separate iterations were necessary given the capacity of the field staff and classroom infrastructure, and taking into account the comprehensive nature of a five-week education program.

Table 2 reports, per wave, the number of study participants as well as the survey attrition rates. There is no attrition in the baseline survey because it was implemented during the same household visit as sample recruitment (i.e. individuals were invited to participate in the study, and if they agreed, the enumerator proceeded with the baseline). Midline data are unavailable for 15 percent of the sample: this is driven primarily by Wave 1, during which the health knowledge test was not administered to the control arm because of a field error.⁹ In the endline, 839 of the 887 subjects were interviewed, corresponding to 5 percent attrition. Supplementary Material D shows that there is no strong evidence of differential attrition in levels and baseline characteristics, and that the results are robust to excluding all Wave 1 data from the midline analysis.

Table 2. Study participants by wave and survey attrition (missing data)

		% Allocation random assignment			% of study participants with missing data
	No. of study participants	HEE	HEEC	Control	Baseline	Midline	Endline
Wave 1	191	21%	27%	52%	0%	53%	3%
Wave 2	282	26%	25%	50%	0%	5%	7%
Wave 3	160	25%	26%	49%	0%	4%	4%
Wave 4	254	26%	24%	50%	0%	4%	7%
Total	887	25%	25%	50%	0%	15%	5%

Notes: The study was conducted in four waves. Each wave refers to a separate iteration of participant recruitment, baseline, randomization, film screenings, midline, and endline. For each wave, the table indicates the number of study participants, their allocation to the study arms, and the percentage of study participants with missing data for each survey.

2.5. Empirical estimation

With the randomized design, I obtain causal effects of the treatments by estimating the Ordinary Least Squares (OLS) regression (1) $$\begin{array}{cc}{Y}_{\mathit{\text{is}}}={\beta }_0\hfill & +{\beta }_1{\mathit{\text{HEE}}}_{\mathit{\text{is}}}+{\beta }_2{\mathit{\text{HEEC}}}_{\mathit{\text{is}}}+{\varphi }_s+{ϵ}_{\mathit{\text{is}}},\hfill \end{array}$$(1) where the subscripts i and s refer to individuals and randomization strata, respectively. The variables HEE and HEEC are mutually exclusive dummies for the treatment arms. HEE is for the treatment of health entertainment-education alone. HEEC is for the treatment combination of health entertainment-education with cash incentives for performance in the health knowledge test. Following the approach of prior studies (e.g. Özler et al., 2018), the only controls in the above equation are randomization strata fixed effects, ϕ_s. The regression thus provides unadjusted estimates, allowing for transparency and avoiding concerns regarding specification search (Lin, 2013).

The ϕ_s are defined by gender, neighborhood, and Saath microfinance client status.¹⁰ The ϕ_s absorb wave effects because neighborhoods do not overlap across waves. The ϕ_s also increase the precision of standard errors (Bruhn & McKenzie, 2009). Because randomization was at the individual level, I use Huber-White robust standard errors (Abadie, Athey, Imbens, & Wooldridge, 2023).¹¹

The estimates of interest from Equation (1)(1) $$\begin{array}{cc}{Y}_{\mathit{\text{is}}}={\beta }_0\hfill & +{\beta }_1{\mathit{\text{HEE}}}_{\mathit{\text{is}}}+{\beta }_2{\mathit{\text{HEEC}}}_{\mathit{\text{is}}}+{\varphi }_s+{ϵ}_{\mathit{\text{is}}},\hfill \end{array}$$(1) are twofold. First, the estimates of β₁ and β₂ represent the impact of the HEE and HEEC treatments, respectively, relative to the control group. Second, the marginal impact of the cash incentives is ${\beta }_2-{\beta }_1.$ I conduct the two-sided hypothesis test $H_0:{\beta }_1={\beta }_2$ to assess whether the difference between HEE and HEEC is statistically significant. Throughout the analysis, I focus on intent-to-treat effects. This is because the film program consisted of five weekly sessions of two to three hours each, and it is not possible or realistic to compel all participants to complete all sessions. Still, attendance was quite high: 88 percent of the sample attended at least one session, and 77 percent completed all five.¹²

3. Results

I examine three sets of outcomes: (1) short-term health knowledge from the midline survey, three weeks after the final film screening; (2) longer-term health knowledge measured in an endline survey 10 months later; and (3) self-reported health behaviors from the same endline. All health knowledge and behavior outcomes I consider are grounded in, and chosen according to, the program modules.

3.1. Effects on short-term health knowledge

Table 3 presents estimates of Equation (1)(1) $$\begin{array}{cc}{Y}_{\mathit{\text{is}}}={\beta }_0\hfill & +{\beta }_1{\mathit{\text{HEE}}}_{\mathit{\text{is}}}+{\beta }_2{\mathit{\text{HEEC}}}_{\mathit{\text{is}}}+{\varphi }_s+{ϵ}_{\mathit{\text{is}}},\hfill \end{array}$$(1) for short-term health knowledge outcomes.¹³ The dependent variables are the proportion of correct answers to the given set of test questions. The exact question wording is shown in Supplementary Material B, which also reports results where outcomes are dummies for correct answers to individual questions (Supplementary Material Table B1).¹⁴

Table 3. Effects on short-term health knowledge

	All topics	By topic
	(1)	(2)	(3)	(4)
		Cleanliness and hygiene	Midwives, maternal & child health	Condom, AIDS, syphilis
HEE	0.083***	0.083***	0.092***	0.065*
	(0.015)	(0.022)	(0.021)	(0.034)
HEEC	0.093***	0.075***	0.116***	0.067*
	(0.015)	(0.024)	(0.022)	(0.035)
F-test, HEE = HEEC
F-statistic	0.402	0.107	0.995	0.003
p-value	0.526	0.744	0.319	0.959
Control Mean	0.530	0.392	0.603	0.521
Control SD	0.157	0.267	0.243	0.351
N	755	755	755	755

Notes: Outcomes are from a health knowledge test three weeks after the final film screening. The dependent variables are the proportion of correct answers in all test questions (Column 1) or in a given subset of questions by topic (Columns 2 to 4). Regression results for individual test questions are presented in Supplementary Material Table B1. HEE is a dummy for assignment to only health entertainment-education. HEEC is a dummy for assignment to health entertainment-education with cash incentives for test performance. Robust standard errors. All regressions include dummies for randomization strata, where strata are defined by gender, neighborhood, and microfinance client status. ***p < 0.01, **p < 0.05, *p < 0.10.

I find economically and statistically significant effects of HEE on the overall short-term health knowledge score. Compared to the control arm, which answered roughly half of all questions correctly, HEE increased scores by 8.3 percentage points, amounting to a 16 percent change (Column 1). To investigate if these overall effects are driven by specific modules, I examine subsets of test questions depending on the topic. In fact, respondents have better proficiency in all subjects due to the HEE treatment (Columns 2–4). Similar to HEE, HEEC also had positive and statistically significant effects on all outcomes (Columns 1–4).

Next, I compare HEEC and HEE to shed light on whether incorporating cash rewards into HEE boosts its effects on health literacy. The coefficient estimates on HEEC tend to be somewhat larger than that for HEE, but the differences are small and not statistically significant. For example, the marginal impact of cash incentives on the overall short-term health knowledge score is only 1 percentage point, with a p-value of 0.526 (Column 1). Similarly, for performance across different topics, I cannot reject the null hypothesis that HEE and HEEC have similar effects (Columns 2–4).

3.2. Effects on longer-term health knowledge

Table 4 explores whether HEE and HEEC participants are still more health literate than the control group in the longer term, 10 months after the program. As with the short-run health knowledge effects in the previous table, the dependent variables here represent the proportion of correct answers. Supplementary Material B lists the individual longer-term health knowledge test questions, and Supplementary Material Tables B2–B3 reports the analysis of each individual test question.

Table 4. Effects on longer-term health knowledge

	All topics	By topic
	(1)	(2)	(3)	(4)	(5)
		Cleanliness and hygiene	Midwives, maternal & child health	Condom, AIDS, syphilis	Night blindness
HEE	0.068***	0.037**	0.095***	0.046***	0.162***
	(0.013)	(0.016)	(0.025)	(0.015)	(0.033)
HEEC	0.057***	−0.002	0.061***	0.039**	0.181***
	(0.013)	(0.015)	(0.023)	(0.016)	(0.031)
F-test, HEE = HEEC
F-statistic	0.587	4.521	1.537	0.169	0.259
p-value	0.444	0.034	0.215	0.681	0.611
Control Mean	0.540	0.712	0.543	0.532	0.396
Control SD	0.148	0.210	0.292	0.178	0.361
N	839	839	839	839	839

Notes: Outcomes are from an endline survey conducted approximately 10 months after the final film screening. The dependent variables are the proportion of correct answers in all health knowledge questions (Column 1) or in a given subset of questions by topic (Columns 2 to 5). Regression results for individual questions are presented in Supplementary Material Tables B2 and B3. HEE is a dummy for assignment to only health entertainment-education. HEEC is a dummy for assignment to health entertainment-education with cash incentives for test performance. Robust standard errors. All regressions include dummies for randomization strata, where strata are defined by gender, neighborhood, and microfinance client status. ***p < 0.01, **p < 0.05, *p < 0.10.

The results show that the effects of HEE continue to be large, positive, and statistically significant. This is again true not only for the total score (Column 1), but also for scores by module (Columns 2–5). Further, the effect size remains comparable to the short-term estimates, suggesting little decay of health knowledge over time. In the long term, those in HEE still had higher overall performance by 6.8 percentage points relative to the control group’s total score of 54 percent. This represents a difference of around 13 percent, which is similar in magnitude to the short-term knowledge effect of around 16 percent (Table 3, Column 1).

Since health knowledge was measured using simple questions (e.g. ‘Washing hands with sand is just as effective as washing with soap. Is this true or false?’), a key issue to consider is whether the persistent positive effects are truly remarkable. One might imagine that it is unlikely for respondents to forget the answers to straightforward, binary concepts. Nevertheless, prior studies have shown that such knowledge can in fact vanish over time. For example, in South Africa, Adam et al. (2021) conduct a field experiment to evaluate the impact of a breastfeeding video intervention. The authors measure maternal knowledge of breastfeeding using only true-false questions. The authors find a statistically significant improvement in knowledge one month after the intervention. However, there were no significant differences between treatment and control at the 5-month follow up. Here, the lack of treatment effects is due to both a slight decrease in knowledge scores in the treatment group, as well as a slight increase in the scores of the control group.

Another related study is by Minamoto et al. (2012), who use before-and-after comparisons to investigate the effects of a health education program in Bangladesh on intestinal helminths. The authors measured knowledge using binary questions such as ‘Are worms good for health’? The authors find that 18 months after the program, educated households exhibited a significant increase in health knowledge. For example, 27 percent of households correctly answered the question ‘Is consumption of sweets associated with worms?’ compared to the baseline percentage of 7 percent. These educated households also have better sanitation practices (e.g. ownership of latrines, keeping a clean house). More than three years after the program, the better practices remained, but the knowledge gains were erased: indeed, the knowledge scores were even lower than baseline levels. This study thus illustrates that health knowledge of even the simplest matters can dissipate.

I now consider the marginal effects of the cash incentives by comparing HEE and HEEC. Across all outcome variables, the magnitude of ${\beta }_2-{\beta }_1$ is between –0.039 and 0.019, with p-values ranging from 0.034 to 0.681 (Columns 1–5). Indeed, if anything, the marginal effects appear to be negative. In particular, for health knowledge on cleanliness and hygiene (Column 2), cash rewards seem to offset the positive effects of health entertainment-education, so that HEEC has virtually no effect.¹⁵ Still, none of the other differences are statistically significant. Hence, from an overall perspective, the long-term results echo the short-term findings: cash incentives for health knowledge test performance does not seem to enhance the efficacy of HEE.

One potential explanation for the lack of difference between HEE and HEEC is the study sample composition. To recruit participants, the project invited individuals to a ‘life skills’ training program, and only those who agreed to participate are included in the sample (see Section 2.1). This implies that compared to the general population, the sample likely has higher motivation and interest in learning. Therefore, it is plausible that the cash incentive of Rs. 80 (US$ 1.80) for each correct answer was not substantial enough to cultivate further engagement.

Another possible explanation is the attendance payments provided in the study (see Section 2.2). All subjects received Rs. 50 (US$1) for each session they attended, which were intended to be only modest gestures of gratitude for participation. But if these attendance payments already influenced knowledge outcomes, and if financial rewards exhibit diminishing marginal effects, then cash incentives for test performance may not have been sufficiently potent (or even able) to induce additional changes. This idea is similar to Athey et al. (2021), who find that in the context of contraceptives, a tailored counseling session increased take up to a sufficiently high level that offering price discounts had no further impact.

In this study, the US$1 attendance payment may have already motivated participants to exert full effort to learn from the program. If so, the additional cash reward for test scores would have made little to no difference, consistent with the findings. Further, the attendance payments may have resulted in income effects as seen in previous research (e.g. Özler et al., 2020). Participants who attended all five sessions would have received Rs. 250 (US$ 5), i.e. 20 percent of the sample average monthly per capita income in the sample. This amount could have fostered respondents’ health knowledge by providing time and resources to synthesize the information from the health films. I am unable to rule out that the attendance payments resulted in null differences between HEE and HEEC, given that all study arms included such payments.

3.3. Health behavior outcomes

Given that HEE and HEEC successfully improved health knowledge, to what extent did they bring about changes in health behavior? I examine this question in Table 5, where the dependent variables in Column 2 onwards are dummies for whether the respondent implements a given health practice (e.g. drinks filtered or boiled water; washes hands with soap; has been tested for HIV). Additionally, I construct an aggregate of these indicators by taking the total number of all such practices implemented by the respondent, which is the outcome in Column 1.

Table 5. Effects on health behaviors

	All topics	Cleanliness and hygiene			Midwives, maternal & child health		Condoms, AIDS, syphilis		Night-blindness
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
	Total no. of behaviors adopted	Drinks filtered or boiled water	Uses a water vessel with tap	Washes hands with soap at least thrice daily	Would go for ante-natal care if self/spouse pregnant	Would go for hospital delivery if self/spouse pregnant	Has been tested for HIV in the last 3 months	Uses condom during intercourse	Eats more nutritious foods to prevent night-blindness
HEE	0.238***	0.016	−0.014	0.066	−0.007	−0.010	0.001	0.041	0.144***
	(0.089)	(0.017)	(0.017)	(0.041)	(0.012)	(0.015)	(0.022)	(0.037)	(0.036)
HEEC	0.173**	0.015	−0.015	−0.016	−0.005	0.017	0.024	0.019	0.135***
	(0.085)	(0.014)	(0.017)	(0.042)	(0.012)	(0.014)	(0.024)	(0.036)	(0.036)
F-test, HEE = HEEC
F-statistic	0.392	0.009	0.001	2.799	0.022	2.760	0.669	0.245	0.047
p-value	0.531	0.923	0.974	0.095	0.883	0.097	0.414	0.621	0.829
Control Mean	3.082	0.027	0.051	0.645	0.986	0.969	0.060	0.217	0.128
Control SD	0.943	0.161	0.220	0.479	0.120	0.175	0.238	0.413	0.335
N	839	839	839	839	839	839	839	839	839

Notes: Outcomes are from an endline survey conducted approximately 10 months after the final film screening. In Column 1, the dependent variable is the total number of all behaviors that the respondent adopted (up to 8). The dependent variables in Column 2 onwards are dummies for whether the respondent adopted the given behavior. HEE is a dummy for assignment to only health entertainment-education. HEEC is a dummy for assignment to health entertainment-education with cash incentives for test performance. Robust standard errors. All regressions include dummies for randomization strata, where strata are defined by gender, neighborhood, and microfinance client status. ***p < 0.01, **p < 0.05, *p < 0.10.

I do not find strong evidence that HEE or HEEC improved health behaviors. Relative to control, HEE and HEEC increased the total number of healthy behaviors by 0.24 and 0.17, respectively (Column 1)—particularly because they are more likely to report eating more nutritious foods to prevent nightblindness (Column 9).¹⁶ While this effect on nightblindness is statistically significant, it is but one outcome that changed due to the intervention, and there are no effects on other behaviors (Columns 2–8). It is therefore difficult to conclude that HEE and HEEC led to generally healthier practices. Moreover, all behavior outcomes are based on self-reports, so the positive effects of HEE and HEEC on eating more nutritious foods may be unreliable.

Turning now to the cash incentives, I find similar results as with health knowledge: the marginal effects of monetary rewards on health behaviors are overall statistically insignificant, and the difference between HEE and HEEC are generally close to zero. Taking all behavior outcomes together, the marginal impact of cash incentives has a large p-value of 0.531 (Column 1). In other words, cash rewards do not seem to induce positive behavior change in health entertainment-education.

3.4. Additional results and robustness

Supplementary Material E presents additional results. First, I estimate Equation (1)(1) $$\begin{array}{cc}{Y}_{\mathit{\text{is}}}={\beta }_0\hfill & +{\beta }_1{\mathit{\text{HEE}}}_{\mathit{\text{is}}}+{\beta }_2{\mathit{\text{HEEC}}}_{\mathit{\text{is}}}+{\varphi }_s+{ϵ}_{\mathit{\text{is}}},\hfill \end{array}$$(1) with measures of program attendance as the outcome variable. The results indicate that HEE and HEEC have similar attendance rates, but the control group exhibits statistically significantly higher attendance than HEE and HEEC. Second, I interact HEE and HEEC with education and gender to examine heterogeneity in effects, and find that the interactions are not statistically significant.

Supplementary Material F shows that the results are robust to the following concerns: (1) that the placebo does not provide an appropriate counterfactual; (2) multiple inference adjustment; (3) statistical power in estimating the marginal effect of incentives; (4) alternative regression specifications and functional forms; and (5) alternative methods of constructing the overall health knowledge score.

4. Conclusion

In this paper, I examine whether health entertainment-education results in lasting change in health knowledge and behavior, and I also explore the potential of monetary incentives to further increase the effectiveness of health entertainment-education. My study has three main findings: (1) health entertainment-education through films successfully increases health literacy; (2) almost one year after the course ended, the knowledge improvements persist, but there is no evidence that it translated into better health behavior; and (3) augmenting health entertainment-education with financial rewards on a health knowledge test generally had no impact.

This study has several limitations. First, my analysis includes only few ultimate health outcomes (e.g. child health, adult sexual health), and all are self-reported. Prior research has shown that self-reported health measures are inherently biased (Hebert, Clemow, Pbert, Ockene, & Ockene, 1995). Though direct observations of behaviors may be more accurate (Davies, Mowbray, Martin, Smith, & Rubin, 2022), it is more costly to implement and was infeasible in this study. Likewise, it can still yield biased measures of real, unobserved health behaviors due to Hawthorne effects (Luby, Halder, Huda, Unicomb, & Johnston, 2011; Ram et al., 2010). Therefore, an important area for future research is to examine behavior effects using more reliable outcome measures.

Second, I am unable to rule out that the null effect of cash rewards was due to the study design (see Sections 2.1 and 2.3). Specifically, all study participants received payments for attending each film session. If these payments already motivated participants, it may be that the cash reward was not large enough to induce extra effort. Moreover, the sample consists only of respondents who agreed to participate in a ‘life skills’ program. This implies they likely already have great interest in learning, potentially rendering the financial incentives less potent.

Finally, since the sample comprises only those who agreed to join a training, the results may not generalize to other study settings. In a sample that is already motivated and drawn into a learning program, the effects of the ‘entertainment’ aspect of the films may have been dampened. In this case, the positive impacts of health entertainment-education estimated in this study would be a lower bound, and one might expect even larger knowledge gains in the broader population.

Despite these caveats, my study reveals insights that can inform the delivery of effective health education. Crucially, my study shows that although health entertainment-education boosted health knowledge especially in the longer-term, there were no significant health behavior changes. This finding speaks directly to the literature, which has argued that lack of knowledge may not be the only determinant of health behavior. For example, cultural norms (e.g. Skordis et al., 2019) and practical obstacles to compliance (e.g. Bennett, Asjad Naqvi, & Schmidt, 2015) may also play significant roles.

Health entertainment-education thus does not appear to be a silver bullet for better public health. If the policy goal is to improve health literacy, then my results suggest that health entertainment-education can be remarkably effective, even among highly interested and motivated beneficiaries. However, if the policy goal is to influence health behavior, then more comprehensive approaches—such as combining health entertainment-education with interventions addressing personal, social, and environmental constraints—may be necessary to transform health practices.

Acknowledgements

I am grateful to Saath for their partnership in this project; to Shawn Cole, Jeremy Shapiro, and Bilal Zia for their collaboration in the field experiment; and to Stuti Tripathi, Bhakti Shah, and the Center for Microfinance for excellent research and field work assistance. I thank Simon Galle, Tom Muir, Hege Johanne Magnussen, and participants at the Experimental Evidence in Education Economics workshop at KU Leuven and the 45th Annual Meeting of the Norwegian Association of Economists for useful comments. I also thank the editor and two referees for helpful suggestions that substantially improved the paper. An earlier version of this paper was circulated with the title ‘Delivering Effective Health Education in Developing Countries: Insights from a Field Experiment in India.’ The field experiment in this paper is registered in the AEA RCT Registry (ID # AEARCTR-000017) and received approval from Harvard University’s IRB (CUHS # F17663-109). Funding from the World Bank for the field experiment is gratefully acknowledged. All participants gave their informed consent prior to their inclusion in the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The replication package for this paper is available at https://doi.org/10.7910/DVN/EJ03JR.

Notes

1 All field staff were trained extensively to ensure adherence to project protocols. The respondent’s study arm assignment was revealed during the first program session.

2 While the results indicate that this allocation was sufficiently powered, an alternative allocation (i.e., 42 percent to control, 29 percent each to HEE and HEEC) would have improved power (Muralidharan, Romero, & Wüthrich, 2023; Özler, 2022).

3 The baseline did not collect data on whether the respondent is the main decision-maker, earning member, or earning member’s spouse. The closest available variable is a dummy for whether the respondent has any income in the last 12 months. This variable is balanced across the study arms.

4 Supplementary Material A details the placebo film content. Financial planning and services was chosen as the placebo topic because this study was integrated into a broader project to minimize field work costs (see Carpena, Cole, Shapiro, and Zia (2019) who study the effects of financial education on financial outcomes). As part of this broader project, the financial film group was randomized into additional financial-related treatments: (1) financial counseling; (2) goal setting; and (3) a cash reward for financial literacy. Subjects assigned to the latter intervention have been excluded from the present study to obtain a cleaner comparison between HEE, HEEC, and the control arm. In other words, this study’s control arm is a composite of the financial films, financial counseling, and goal setting interventions. Supplementary Material F shows that the results are robust to restricting the control arm to “pure” control subjects—i.e., those assigned to financial films, but not financial counseling, goal setting, or cash rewards for financial literacy.

5 After randomization, classes were formed by grouping subjects based on their education program assignment and the day/time availability they specified at recruitment (see Section 2.1). The latter was meant to give participants a feasible class time, increasing the likelihood of attendance.

6 An alternative approach to control for income effects would have been to provide non-HEEC respondents with the average winnings of HEEC respondents.

7 It is necessary to inform study participants of their cash incentives assignment because the intervention’s objective is to test whether the promise of monetary rewards can increase motivation to learn from the health program.

8 As will be shown in the results, differences between HEE and HEEC are generally not statistically significant. Moreover, if reminders did impact intrinsic motivation, it could be a level effect given that all participants received information and reminders about their cash incentives assignment.

9 Among Wave 1 participants (N = 191), midline data are missing for 102 people (53 percent) of which 100 (52 percent) belong to the control study arm.

10 Results are similar when excluding strata fixed effects in the regressions (see Supplementary Material F).

11 Results are similar when clustering standard errors at the wave-class level (see Supplementary Material F). Here, a wave refers to a separate iteration from recruitment to endline, and a class refers to a group of participants with identical education program assignment (i.e., health or placebo), who met at the same day and time every week for the film screenings (see Section 2). In each wave, there were roughly 15 classes: 5 for health films and 10 for financial films. In practice, the number of participants per class was similar, at around 20 persons. However, because this study was integrated into a broader field project, some participants in the financial films group were assigned to additional cash incentives for financial literacy. These subjects—corresponding to 1/3 of the full project sample—have been excluded from the present study to obtain a cleaner comparison between the HEE, HEEC, and control arms.

12 Supplementary Material E presents estimates of Equation (1)(1) $$\begin{array}{cc}{Y}_{\mathit{\text{is}}}={\beta }_0\hfill & +{\beta }_1{\mathit{\text{HEE}}}_{\mathit{\text{is}}}+{\beta }_2{\mathit{\text{HEEC}}}_{\mathit{\text{is}}}+{\varphi }_s+{ϵ}_{\mathit{\text{is}}},\hfill \end{array}$$(1) with attendance as the outcome variable.

13 The sample size here is only 755 because of missing data from Wave 1 control group (see Section 2.4). Supplementary Material D shows that the results are similar when excluding all Wave 1 respondents from the regression sample.

14 To manage the survey length, the midline did not cover night blindness, and it was instead covered in the endline.

15 Financial incentives may have demotivated participants to learn “Cleanliness and Hygiene” because they were already well-versed in this concept. At baseline, 92–93 percent in each study arm were aware they should wash hands after defecation (Table 1, Panel C). Similarly, at endline, the control group exhibited the highest average score in “Cleanliness and Hygiene” across all modules. Research on the psychology of motivation contends that cash rewards perceived as controlling or affecting autonomy can diminish intrinsic motivation (Deci & Ryan, 1985; Frey & Oberholzer-Gee, 1997). Given participants’ existing proficiency in “Cleanliness and Hygiene,” it may be that they viewed the rewards as controlling or a threat to their autonomy; the rewards would then result in detrimental effects, in line with findings in prior psychological research.

16 The nightblindness outcome is constructed from the open-ended question “Can you list the precautions you take to ensure that you or your family member doesn’t contract nightblindness?” (see Supplementary Material C). Almost 60 percent of interviewees did not have a response to this question, and any text responses were hand-coded to create the indicator variable. Apart from eating more nutritious foods, some examples of other responses to this question are keeping eyes clean, reducing alcohol intake, and not watching TV—all of which were not relevant based on the content of the health education films.