Empowering youth for social change through the Shujaaz multimedia platform in Kenya

ABSTRACT

The Shujaaz media seek to improve the lives of Kenyan youth by norming and stimulating positive health behaviours, and incentivizing income generation activities. This study evaluates the impact of Shujaaz using data collected over a 2-year period from a cohort of 700 youth aged 15–24 years. Multivariate correlated random-effects regression models were used to estimate the impact of exposure to analogue (comic and radio) and digital Shujaaz media (social media and SMS) on attitudes, norms and behaviours around family planning and income generation. Shujaaz analogue media were associated with intermediate outcomes; digital media were associated with a 18.1 percentage point increase in ever using condoms and a 19.0 percentage point increase in recommending the use of condoms to friends and partners. Additionally, both analogue and digital media were associated with improved income-generating outcomes. Importantly, exposure to digital media was associated with a KSH 2,096 (US$20.9) increase in monthly income.

KEYWORDS:

• East African youth
• social media
• sexual and reproductive health
• youth income generation

Background

Youth are critical for East African development, especially in light of the fast-approaching demographic window (Bloom, Kuhn, & Prettner, 2017; Gichuhi & Nasiyo, 2016; Kinaru-Mbae & Chatterjee, 2017). In Kenya, with almost 39% percent of the reproductive age population being 24 years old or younger (Kenya National Bureau of Statistics, & ICF Macro, 2015), youth need opportunities to build human capital through developing marketable skills that enable them to generate income, and through controlling their fertility (AFIDEP, 2017; Facilitating Fertility Decline to Maximise on the Window of Opportunity, 2015). The present study examines how the Shujaaz multimedia platform aids Kenyan youth in achieving those goals.

Shujaaz as an omni-channel behaviour change strategy

Shujaaz is a public-interest media platform established in 2010 in Kenya. The overall focus of the Shujaaz strategy is to serve as a platform for young people to form a community, where they can discuss pressing issues, especially sensitive or taboo topics, such as money, sex, relationships, and contraception. Shujaaz uses an integrated omni-channel design that includes a free-of-charge nationally distributed comic (705,000 copies a month), a weekly syndicated radio show, and digital media (Facebook, SMS, Twitter, WhatsApp) to tell stories of fictional characters and real people representative of youth from different parts of the country as they encounter and resolve challenging life and health issues. Improving sexual and reproductive health outcomes and providing income-generation skills to youth in Kenya are at the centre of the Shujaaz strategy. At a glance, these two goals might seem independent, yet youth with greater level of financial realization have demonstrated higher intentions to use family planning (FP) methods as well (Nunes Hildebrand & Rodriguez Veloso, 2012).

The transtheoretical model of health behaviour change

Based on the Transtheoretical Model (TTM) of health behaviour, the Shujaaz strategy recognizes that behaviour change is a process by stages, rather than a single event (Prochaska & DiClemente, 1982). The TTM model posits that change unfolds over time, progressing through a series of stages, including: pre-contemplation, contemplation, preparation and action. The Shujaaz strategy targets individuals situated at each of these stages of change, offering a mix of information and practical tools to help them achieve sexual and reproductive health and to improve income-generation skills.

During the pre-contemplation stage, it is hypothesized that individuals do not intend to take action in the near future, either because they are uninformed, under-informed, or misinformed about the pros and cons of specific behaviours (Glanz, Rimer, & Viswanath, 2008). For those situated at this stage, Shujaaz exposes participants to stories highlighting the relevance of proper planning on health-related issues in order to achieve their income and development goals; this is the stage of ‘norming’ contraception use as an element of ‘smart planning’. During the contemplation stage, it is hypothesized that youth intend to change their behaviour but get stuck evaluating the pros and cons of their actions. At this stage, Shujaaz comic and radio stories dispel myths and misconceptions, while digital media conversations enable youth to discuss specific information about available contraceptive methods and available financial options to engage in financial transactions. In the preparation stage, individuals are ready to take action and are already willing to hold conversations with health providers, peers, parents, counsellors or other actors who could help them achieve the change (Glanz et al., 2008). Such conversations can happen on the Shujaaz social media platforms, on other platforms, or offline. Finally, individuals at the action stage are already making specific modifications towards the desired behaviour change, including contraceptive use and post-action behaviours such as recommending the use of modern contraceptive methods.

Analogue versus digital media channels

Different media channels are postulated to play distinct roles in guiding youth through the stages of change, with analogue and digital media characterized by distinct strengths. On the one hand, most analogue media (especially print) allow only uni-directional communication, in which the media professional presents media consumers with one message at a time. The ‘one-professional-voice’ is often viewed as more trustworthy than the multiple voices of digital media, while the consistency of the ‘voice’ also contributes to establishing a sense of belonging with analogue-media professionals (Deliso, 2006). In contrast, digital media enables multi-directional communication and expanded conversations, in which media consumers also serve as media producers by providing feedback on stories; sharing, modifying and adapting messages; and conducting conversations among peers (Stephen & Galak, 2010).

In market studies, empirical evidence reveals that media strategies relying on analogue channels have stronger impacts on brand awareness and trust, while strategies delivered through digital channels are more likely to have positive impacts on brand image (Bruhn, Schoenmueller, & Schäfer, 2012; Stephen & Galak, 2010). In this case, the Shujaaz strategy is designed so that stories appear first in the nationally distributed print comic to promote awareness and trust and are then expanded and reflected upon using the other media channels: the weekly radio show, monthly events, and a range of digital platforms. Because of the wider distribution of and access to analogue media in Kenya (and the lesser availability of digital and social media), we hypothesize that:

H1: Awareness of Shujaaz will be built primarily by initial exposure to Shujaaz interventions through analogue media, followed by exposure to digital media.

While it is anticipated that the combination of analogue and digital media unlocks the greatest benefits for the audience in terms of attitudinal, normative or behavioural outcomes (Deliso, 2006; Freeman, Potente, Rock, & McIver, 2015), currently however, the evidence base for such an approach is lacking, and there is still the need for rigorous explorations of the impact of a combination of analogue and digital media in a single Behaviour Change Communication (BCC) strategy. To date, most reported interventions have focused on using and assessing the impact of only one type of media – either analogue or digital – but not both in one coherent strategy (Hamm et al., 2014; Moorhead et al., 2013; Adewuyi & Adefemi, 2016; Freeman et al., 2015). By examining the Shujaaz media experience, this paper aims to fill the evidence gap on the effectiveness of a joint analogue/digital media platform by examining how Shujaaz users progress from first exposure to analogue media, followed by deeper engagement with Shujaaz digital platforms, and ultimately to changes in normative and behavioural outcomes. Therefore, we hypothesize the following:

H2: Exposure to both analogue and digital media will be positively associated with normative and behavioural outcomes throughout the stages of change.

H3: The longitudinal effects of exposure to Shujaaz media on intermediate and final outcomes will vary significantly by channel of exposure (analogue vs. digital).

Finally, economic models suggest that labour market conditions can play a role in influencing marriage, childbearing, and the use of contraception. Considering that childbearing is traditionally more intensive in women’s time, the opportunity cost of having children is higher for women with greater income earning opportunities, leading to postponements in marriage and childbearing (Jensen, 2012). On the other hand, for both genders, when youth struggle to make a living, they are less able to prioritize the avoidance of pregnancy over immediate financial needs (Jennings et al., 2017). While Shujaaz targets the improvement of sexual and reproductive health among youth directly, it can also affect these outcomes indirectly through the promotion of income-generating skills and the subsequent increase in income. Therefore, our final hypothesis can be defined as:

H4: Shujaaz operates to improve family planning outcomes by increasing income.

Methods

Sample design

This study was originally intended as a nationally-representative cross-sectional survey that then became the baseline sample for a panel study. Data for the panel were collected through face-to-face interviews. The target random sample of youth was achieved using a multi-stage sampling procedure. Districts in the country were clustered into urban and rural strata and then selected within each stratum using a probability proportional to population size (PPPS) approach. To select households within clusters, interviewers established a minimum of four landmarks in each PSU and then randomly (using a Kish Grid) selected two landmarks as the starting points of the random walk. A Kish Grid was used to randomly select among eligible household members (adults, 15–24 years old). Only one respondent was selected in each household. If a desired respondent was not at home, an interviewer made up to three call-backs before a replacement household was selected.

For the second round of the interviews in January-February 2017, all 2,011 respondents from the 2016 survey were contacted by phone to confirm their location and availability. The respondents immediately available were re-interviewed within 1–2 weeks of the call, depending on their location. The respondents who moved but disclosed their new location and confirmed availability for the survey were also interviewed within 1–2 weeks of the call. Respondents who moved and changed their phone number required additional effort to track. The team contacted their families and neighbours by secondary contact phone or in person to find the new address and phone number for the respondents and tried to locate them, using updated contact information. In total, 700 participants completed the second round of interviews that formed the panel. No differences in area of residence, education, age, marital status, and access to internet were found between those who completed the second round of interviews and those who did not. However, in regression models that estimated the likelihood of being in the panel sample, males were more likely to participate in the second round of the survey (by 7% points). Similarly, those participating in the second round of the survey were more likely (by 6% points) to report no independent income.

Interviewers

Interviewers for this study were selected from the age group close to that of the Shujaaz audience, i.e. mostly younger than 25 years of age with the exception of the supervisory staff. Interviewers were recruited from all regions in Kenya and were fluent in several local dialects. Training of interviewers was done centrally in Nairobi and took three days for each data collection year plus an additional day for practice. Training was conducted in English as all interviewers had post-secondary education and were fluent in English; however, each received a translated KiSwahili questionnaire for guidance, and each had opportunities to practice interviewing in local dialects.

Questionnaire

In the survey, interviewers collected information from respondents on basic socio-demographic characteristics, general media use (e.g., radio, TV, newspapers and other printed media, internet and social media), family planning attitudes and behaviours, attitudes towards agriculture as an income-generating activity, and money and finance. Questions were designed to measure the norms, attitudes conversations, knowledge and other components within the Transtheoretical Model of Change, following global practices in media audience and behaviour outcomes assessment (Abreu-Lopez, Gersen, & Mirzoyants-McKnight, 2016; Bicchieri, 2014; Kang, 2010; Morhart, Malar, Guevremont, Girardin,F., & Grohmann, 2015; Speizer, Calhoun, & Guilkey, 2018; World Health Organization and Center for Disease Control and Prevention, 2011).

Prior to the study launch, the questionnaire went through a round of internal and external reviews. The internal review was done by the Shujaaz Production and Social Media teams; the external interview was done by the Research Directors of Research Plus Africa, the company responsible for the data collection for this study. Small adaptations were made to the questionnaire based on the review process to ensure that the questions were sensitive to the cultural context of youth in Kenya and were clear to all respondents regardless of their age and educational attainment.

Ethical considerations

Participation in the survey was inclusive of respondents of both genders, socio-economic strata and ethnic backgrounds within the target age range of 15 to 24 years. Respondent participation was voluntary; each participant provided oral consent prior to the interview, and the consent was recorded in the computer assisted telephone interviewing system. For respondents aged 15–17 years old, interviewers obtained written guardian consent in line with the Child Protection Act of Kenya, in addition to the assent of the young person him/herself. All participants had the right to withdraw from the interview at any point. All data were kept confidential; individual questionnaires were identified by a numeric code only. Respondents’ names and contact information were secured in a separate dataset.

On behalf of Well Told Story, Research Plus Africa, the subcontractor in this study, obtained all required study permits from the National Commission for Science, Technology and Innovation (NACOSTI), which is the Kenyan equivalent to the human subject research approval by an Institutional Review Board.

Measures

Sexual and reproductive health outcomes

Eight different indicators were used to assess the process of adopting family planning, each one fitting within one of the stages of change (Table 1). At the pre-contemplation/contemplation stage, three indicators were used. First, family planning importance was assessed from two questions, including ‘How important is managing when you have children for you and for your future plans?’ defined as a binary variable, and agreement with the statement that ‘“smart hustlers” protect themselves from unwanted pregnancies,’ also defined as a binary variable. Family planning knowledge was defined as the number of contraceptive methods a respondent could spontaneously list as methods to prevent pregnancy. At the preparation stage, ‘conducting family planning conversations’ was defined as a set of binary variables informing any conversations with peers and any conversations with adults (including health providers and shopkeepers) about the use of contraceptives. At the action stage, three indicators were used. Ever use of condoms and ever use of modern contraception were defined as binary variables describing prevalence of ever use of the methods. In addition, recommending a condoms was also included in the analysis and assessed based on the question ‘Have you ever told your friends or partners they should use a condom?’

Table 1. The FP and Money Journeys by Stages of Change.

	Family Planning	Income generation/Money
Pre-contemplation/ Contemplation:	Report that managing when you have children is ‘very important’ for future plans ‘Strongly agree’ that ‘smart hustlers’ protect themselves from unwanted pregnancies Total number of FP methods known (condom, implant, patch, injectable, coil, pill, e-pill)	Number of types of financial activities with which respondent is familiar (budgeting; remittances; bill-paying; saving; borrowing; lending; insurance; investing; installment purchasing)
Preparation:	Ever had a conversation about FP with peers Ever had a conversation about FP with adults (including health professionals and shopkeepers)	Talked to anyone about the uses of mobile money Talked to any adults about the uses of mobile money
Action:	Ever used a condom Ever used modern contraception Ever recommended using a condoms to friends or partners Has at least one child	Ever used mobile money Ever engaged in financial activites Total monthly income (KSH 0, KSH 1–4800, KSH 4801+) Total monthly income (continuous) Any adult adopted mobile money based on conversations Anyone adopted mobile money based on conversations

Income-generating skills

Similarly, multiple indicators were used to measure progress towards the generation of skills to increase income, largely focused on the ability of mobile money to facilitate savings, borrowing and purchasing. Corresponding to the pre-contemplation/contemplation stage, respondents were asked about the number of types of financial activities with which they were familiar, including such activities as budgeting, saving, borrowing, lending and investing. At the preparation stage, respondents were asked about conversations they may have had regarding income-generating activities or the use of mobile money (i.e., digital financial services) for multiple purposes. At the action stage, numerous indicators were used, including whether or not a person had ever used mobile money; whether or not a person had ever engaged in different types of financial activities; whether or not anyone had adopted mobile money as a consequence of conversing with the respondent; and monthly income (both as a categorical and as a continuous variable).

Exposure to Shujaaz

Study participants reported information about exposure to any Shujaaz media and the timing of first exposure to any analogue media (e.g., comic and radio) and to any digital media (e.g., Facebook, SMS, Twitter, Instagram, YouTube, WhatsApp). For the analysis, exposure was determined by the timing of exposure to digital and analogue media, defined as nominal variables, with possible categories being: ‘Never exposed,’ ‘exposed for the first time in the last year,’ and ‘exposed for the first time more than a year ago.’

Analytical methods

Univariate statistics were calculated to describe the sample in 2016 and 2017. Bivariate statistics were calculated for associations between exposure to Shujaaz media and each of the principal outcomes. In addition, multivariate linear and logit regression were utilized with each sexual and reproductive health and money management indicator regressed on the variables for exposure to Shujaaz analogue and digital media and a set of socio-demographic control variables. Exposure to Shujaaz was self-reported. As such, it was non-randomly distributed, and therefore the estimation strategy attempted to control for both observable characteristics of respondents and unobservable innate differences. Control variables included: age (continuous), gender, relationship status (single no partner, single with partner, married or living with someone), and education level (none/some primary, primary, secondary, post-secondary). A separate set of regressions included monthly income in place of self-assessed financial status.

Because of the panel nature of the data (i.e., repeated observations for the same individuals) and the need to control for both observable and unobservable differences between those who self-reported exposure to Shujaaz and those who did not, correlated random effects regression models were used to estimate the impact of exposure to analogue and digital Shujaaz media. Repeated observations for individuals introduces serial correlation for each individual attributable to the time invariant unobservables (Wooldridge, 2013). While fixed-effects models are often used to address time invariant unobserved heterogeneity in panel data models, the correlated random-effects model relaxes the assumption of zero correlation between the individual-level time-invariant unobservable and the individual-level explanatory variables. Estimation involves decomposing the time-invariant unobservable into a fixed unobservable within individuals and a set of means of time-varying observable variables. Using this approach, we are able to estimate the effects of individual-level variables unbiased by correlation with the unobservable term at the individual-level, estimate random effects coefficients for the time invariant variables (e.g., gender), and generate an augmented Hausman specification test of the equivalence of fixed and random effects estimates (Allison, 2009; Schunck, 2013; Wooldridge, 2010). In nearly all causes, the robust version of the Hausman test strongly rejected the nulll of exogeneity of exposure, indicating that the use of the correlated random effects model was justified. For the income estimations in which the outcome involved left censoring at zero, we used the same strategy and estimated a correlated random effects tobit model.

Results

Descriptive results

Descriptive statistics of the sample at baseline and endline (Table 2) largely reflect the natural maturation of the sample as they aged. In the 2016 wave, the average age was 18.4 years, increasing to 19.9 in the 2017 wave; more than half the sample was older than 19 years by 2017. The highest level of education attained varied across the sample, with the 59% having primary and 22% having secondary education respectively in 2016. By 2017, over a third of the sample had a secondary level of education. The percentage of the sample reporting that they were single, and dating increased from 34.1% to 46.6% from 2016 to 2017, while sexual activity increased from 57.3% to 62.1%.  Mean income increased from KSH 2,399 in 2016 to KSH 3,934 in 2017. Finally, related to exposure to the intervention, the percentage of the sample who had ever heard of Shujaaz increased from 61.6% in 2016 to 89.1% in 2017.

H1: Awareness of Shujaaz will be built primarily by initial exposure to Shujaaz interventions through analogue media, followed by exposure to digital media.

Table 2. Descriptive statistics at baseline and endline.

	Baseline (2016)		Endline (2017)
Characteristics	n	Proportion	n	Proportion
Gender
Male	394	56.3	–	–
Female	306	43.7	–	–
Age*
15–19 years old	482	68.9	330	47.1
20–24 years old	218	31.1	370	52.9
Education (highest level)
None/Prim	96	13.7	52	7.6
Primary	412	58.9	335	49.2
Secondary	152	21.7	229	33.6
Post-Sec	40	5.7	65	9.5
Relationship status
Single, dating	239	34.1	326	46.6
Single, not dating	406	58.0	304	43.4
Married	48	6.9	60	8.6
Other	7	1.0	10	1.4
Sexually active
No	299	42.7	265	37.9
Yes	401	57.3	435	62.1
Monthly income (mean KES)	2,398.7		3,933.8
Have you ever heard of Shujaaz?
No	269	38.4	76	10.9
Yes	431	61.6	624	89.1

* Total number = 700; mean age at baseline = 18.4 (SE 2.6); mean age at endline = 19.9 (SE 2.6)

From Figure 1, it is clear that the Shujaaz comic serves as the key portal by which youth become familiar with Shujaaz, thereby supporting H1. In 2016, youth were more than eight times as likely to be exposed to analogue alone as they were to be exposed to analogue plus digital (49.6% versus 6.3%), while 44% of the sample had not been exposed to any Shujaaz media. By 2017, 68.9% of the unexposed sample had transitioned to being exposed to analogue, bringing to 71.4% the percentage who had been exposed to Shujaaz by analogue media alone. An additional 8.4% had transitioned to being exposed to analogue plus digital. Among those exposed to analogue alone in 2016, 12.7% transitioned to being exposed to digital as well by 2017. Almost no one was exposed to digital alone.

Figure 1. Exposure to Analogue and Digital Media, by year.

Bivariate and multivariate results

H2: Exposure to both digital and analogue media will be positively associated with normative and behavioural outcomes throughout the stages of change.

H3: The longitudinal effects of exposure to Shujaaz media on intermediate and final outcomes will vary significantly by channel of exposure (analogue vs. digital).

In support of H2 and H3, bivariate modelling indicated that longer exposure (i.e., more than a year prior to the survey) to any Shujaaz digital media channel was associated with better sexual and reproductive health and income-generating outcomes in 12 out of 19 measured outcomes (Table 3). For example, relative to those not exposed to Shujaaz digital media, those exposed to the Shujaaz digital platform for more than one year prior to the survey were 23.9 percentage points more likely to declare having had any conversation related to contraception (82.2% versus 58.3%), 22.8 percentage points more likely to report ever use of a condom (73.3% versus 50.5%), and 25.9 percentage points more likely to ever use modern contraception (80.0% versus 54.1%). For analogue media, longer exposure to Shujaaz was associated with better SRH and income-generating outcomes for 9 out of 17 indicators, although the magnitude of effects tended to be smaller than for digital media.

Table 3. Bivariate analysis of timing of exposure to analogue and digital media, year 2017.

Variable	Analogue			p-value(a) related to 'no exposure'		Digital			p-value(a) related to 'no exposure'
	No exposure	< 1 year	>1 year	<1 year	>1 year	No exposure	< 1 year	>1 year	<1 year	>1 year
Family planning
Family planning is important	82.7	77.8	84.9	0.379	0.570	82.3	92.2*	91.1	0.071	0.129
Family planning smart hustlers	65.4	66.7	74.2*	0.847	0.065	70.5	86.3**	73.3	0.016	0.690
Number of family planning methods known(b)	60.6	56.6	58.6***	0.655	0.001	60.6	56.6	58.5	0.542	0.134
Any family planning conversations with peers	49.0	52.5	56.9	0.619	0.140	53.1	72.5	62.2	0.008	0.239
Any family planning conversations with adults	32.7	35.4	35.2	0.689	0.624	31.3	60.8***	53.3***	0.000	0.002
Ever use of condom	51.0	50.5	53.3	0.948	0.661	50.5	58.8	73.3***	0.253	0.003
Ever use of modern contraceptives	57.7	52.5	56.5	0.459	0.829	54.1	58.8	80.0***	0.519	0.001
Recommend condom	40.4	43.4	45.7	0.660	0.324	42.7	45.1	68.9***	0.741	0.001
Have at least one child	21.2	11.1*	8.9***	0.053	0.000	11.4	7.8	8.9	0.435	0.604
Income generating skills
Money uses (b)	4.5	4.5	5.3***	0.861	0.000	5.1	5.1	5.2	0.323	0.707
Ever talked to adults about money	59.6	68.7	69.6**	0.178	0.047	66.7	68.6	84.4**	0.781	0.014
Ever talked to adults about mobile money	22.1	28.3	48.5***	0.311	0.000	39.4	51.0	62.2***	0.106	0.003
Ever used mobile money	63.5	44.4***	59.8	0.007	0.483	56.5	60.8	77.8***	0.549	0.005
Ever engaged in financial activities	85.6	96.0**	91.1*	0.011	0.083	90.6	96.1	91.1	0.186	0.903
Used money to budget	56.7	59.6	55.1	0.679	0.765	53.8	64.7	75.6***	0.133	0.005
Used money to save	56.7	69.7*	70.4***	0.056	0.006	66.2	76.5	86.7***	0.135	0.005
Income (proportion with no income)	59.6	72.7**	58.1	0.049	0.783	62.1	62.7	35.6***	0.926	0.000
Income (proportion with low income)	9.6	11.1	17.1*	0.727	0.057	14.6	15.7	22.2	0.829	0.167
Income (proportion with high income)	30.8	16.2**	24.7	0.014	0.202	23.3	21.6	42.2***	0.773	0.005
Income (continuous) (b)	4449.0	2055.6	4200.2	0.245	0.695	3540.1	3278.4	9962.2***	0.287	0.000

(a) Chi2 for proportions and t-test for continous variables

(b) Means are presented for continous variables *** p < .01 ** p < .05 * p < .1

The multivariate regression modelling (Table 4), further supported hypothesis H2 that exposure to both digital and analogue media are positively associated with normative and behavioural outcomes related to contraception and income generation skills. With a respect to family planning, exposure to analogue media tended to affect the contemplative variables but not variables further along the Transtheoretical Model. For example, exposure to analogue media was associated with a 12.4 percentage point increase in agreement with the statement that smart hustlers use family planning.’  Digital media, however, tended to influence actions further along the model, such as a 18.0 percentage point increase in ever use of a condom and a 19.0 percentage point increase in recommending a condoms.

Table 4. Multivariate analysis.

	Analogue				Digital				Income
	< 1 year		>1 year		< 1 year		>1 year		Low		High
	B	p-value	B	p-value	B	p-value	B	p-value	B	p-value	B	p-value
Family planning
Family planning is important	-0.012	0.798	0.0666*	0.067	0.066	0.384	-0.050	0.603	0.042	0.258	-0.041	0.349
Family planning smart hustlers	0.091	0.127	0.1243***	0.005	0.2056**	0.028	0.017	0.859	-0.031	0.473	-0.0945*	0.072
Number of family planning methods known	-0.5628*	0.055	-0.067	0.757	-0.585	0.126	0.046	0.916	0.219	0.298	0.6136**	0.013
Any family planning conversations with peers	-0.012	0.849	0.034	0.466	0.025	0.756	-0.005	0.957	0.116***	0.009	0.000	0.993
Any family planning conversations with adults	0.1036*	0.062	0.0803*	0.051	0.2139***	0.002	0.106	0.167	0.0669*	0.093	0.063	0.157
Ever use of condom	0.002	0.975	0.024	0.524	0.031	0.629	0.1801**	0.019	0.037	0.300	0.039	0.348
Ever use of modern contraceptives	0.004	0.943	0.045	0.238	0.002	0.981	0.1377*	0.078	0.052	0.150	0.0888**	0.036
Recommend condom	0.031	0.555	0.054	0.157	0.077	0.253	0.1904**	0.012	0.014	0.703	0.029	0.491
Have at least one child	-0.023	0.428	-0.023	0.256	0.005	0.899	0.010	0.797	0.027	0.225	0.0883***	0.000
Income generating skills
Money uses	0.069	0.785	0.221	0.242	-0.144	0.665	0.123	0.746
Ever talked to adults about money	0.021	0.724	0.008	0.855	-0.120	0.148	0.123	0.206
Ever talked to adults about mobile money	0.045	0.494	0.1129**	0.019	-0.028	0.725	0.1742*	0.071
Ever used mobile money	-0.024	0.682	0.030	0.485	-0.087	0.240	0.046	0.609
Ever engaged in financial activities	0.016	0.720	0.0615**	0.045	-0.071	0.262	-0.061	0.384
Used money to budget	-0.032	0.326	-0.014	0.528	-0.006	0.870	-0.020	0.610
Used money to save	-0.040	0.491	0.052	0.220	0.053	0.460	0.076	0.343
Income (proportion with no income)	0.028	0.608	-0.061	0.131	0.002	0.974	-0.141	0.075
Income (proportion with low income)	-0.009	0.615	0.016	0.145	-0.001	0.974	0.0275***	0.005
Income (proportion with high income)	-0.019	0.605	0.044	0.128	-0.002	0.974	0.114	0.104
Income (continous)	3.304	0.994	502.846	0.122	1208.674*	0.074	2096.269***	0.008

*** p<0.01, ** p<0.05, * p<0.1

For the models examining the steps to developing income generating skills, both analogue and digital media were associated with outcomes at different stages of the Transtheoretical Model. For example, exposure to analogue media was associated with discussions with adults about mobile money (preparation) and whether or not a respondent had ever engaged in financial transactions (action). Similarly, exposure to digital media was associated with an increase in income —both as a categorical variable and as a continuous variable (action).

H3: The longitudinal effects of exposure to Shujaaz media on intermediate and final outcomes will vary significantly by channel of exposure (analogue vs. digital).

For nearly all outcomes related to income generating skills, longer term exposure (> 1 year prior to the survey) had stronger effects than more recent exposure (< 1 year prior to the survey), with the exception being for awareness of the uses of money. For outcomes related to family planning, the effects of exposure to analogue media were statistically significant only for longer term exposure. In no cases did Shujaaz analogue exposure in the year preceding the survey influence outcomes. For digital media, effects were seen in the FP journey for both recent and longer term first exposure.

H4: Shujaaz operates to improve family planning outcomes by increasing income. The results from the multivariate modelling also support H4 that Shujaaz operates to improve family planning outcomes by increasing income. Notably, exposure to digital media at least one year prior to the survey was associated with a KSH 2,096 (US$20.9) increase in monthly income. In turn, being in the highest income tercile, as shown in the right-most panel of Table 4, was associated with a 8.9 percentage point increase in the ever use of modern contraceptives. In short, by influencing income, Shujaaz also works to improve family planning outcomes.

Discussion

This study has sought to evaluate a comprehensive youth-targeted, health and income skills behaviour change strategy involving a combination of a monthly comic, a syndicated radio program and multiple forms of digital media, including social media and SMS. The strategy is not the first to employ comics as a vehicle for social change and health promotion. Comics have been used to address issues related to nutrition and physical activity (Baranowski et al., 2002; Branscum, Sharma et al. 2013), sexual and reproductive health and HIV/AIDS (Gillmore et al., 1997; Phrasayamongkhounh, 2015; Willis, Kachur et al. 2018), lymphatic filariasis (el-Setouhy & Rio, 2003), kidney transplants and cancer risk (Boynton, 2018), and even human trafficking (Benton & Daniela, 2012). Despite the wide appeal of comics, rigorous evaluations that document the impacts of comics on behaviour change have been relatively rare. The majority of evaluations have either been process evaluations (Branscum, Sharma et al. 2013) or descriptive in nature (Benton & Daniela, 2012; Boynton, 2018). In one of the few cases with rigorous study designs, Branscum et al (Branscum, Sharma et al. 2013) found that a theory-based nutrition and physical activity comic for children was associated with increased fruit and vegetable consumption, physical activity, and water and sugar-free beverage consumption.

The evidence that digital media, particularly social networking sites and SMS texting, can lead to behaviour change is still emerging. Authors (Hamm, Shulhan et al. 2014) have noted the potential for social network sites to provide a forum for adolescent peer communication but have also lamented the evidence base demonstrating effects on behaviour change. A few studies have used randomized control trials to examine the effects of social media on diet and exercise (DeBar et al., 2009; Lao, 2011; Rydell et al., 2005) and sexual health (Cox, Scharer, & Clark, 2009). However, while some of these were effective at increasing knowledge (Cox et al., 2009), only one was able to affect behaviour change.

Because few studies have examined the effectiveness of a combination of comic-based messaging with social networking, this study, therefore, represents an important contribution to this literature. Using a robust evaluation design that controls for non-random exposure to Shujaaz, this study demonstrates that a combined approach can work at multiple stages along the behaviour change framework. In the area of family planning, exposure to both analogue (e.g. the comic) and digital media affected key outcomes, including knowledge indicators (e.g. number of family planning methods known), valuation indicators (e.g. the importance of family planning), interpersonal communication about family planning, and ultimately use of family planning and recommending family planning to others. In a similar vein regarding the adoption of income-generating skills accumulation, exposure to digital and analogue media were associated with greater likelihood of discussions about money, and greater engagement in financial activities.

These results are consistent with previous evaluations of the Shujaaz media platform. Between 2010 and 2014, the Carolina Population Center conducted an evaluation of the Tupange Reproductive Health Program in Kenya, a multifaceted approach including the Shujaaz comic. Findings revealed that 15–19-year-old girls exposed to the Shujaaz comic were 1.8 times more likely to have never had sex and 2.5 times more likely not to have been pregnant than girls unexposed to Shujaaz (Speizer et al., 2018). Furthermore, results from Well Told Story’s 2015/2016 Cross-Sectional Surveys evidenced auspicious relationships between exposure to Shujaaz media and assorted contraception outcomes, as well as a negative relationship between exposure to Shujaaz and perceptions of smoking as being cool (Beaudoin, 2016). In addition, and related to income generation activities, survey participants reporting ever use of one or two Shujaaz platforms had higher levels of hustler attitudes, aided money activity recall, and money activity behavior. Ever users of two Shujaaz platforms exhibited higher levels of perceived current money situation (Beaudoin, 2016).

While this work used a rigorous panel design, future work might explore the possibility of using an experimental design, perhaps at the community level. For analogue media, different comics could be distributed in different parts of the country to test the effects of different messaging. Similarly, using geocoding, different messages and content could be shared on Facebook in certain areas of the country, again allowing for different combinations of digital and analogue media to be explored.

Strengths and Limitations

As main strength of our analytical approach, the strategy that was used to address self-determined exposure to Shujaaz was a multivariate control strategy with measurable control variables (e.g. age, education, gender, income, urban/rural) included in the multivariate models. In addition, correlated random-effects regression models were used that compared individuals to their earlier selves. These models looked at changes in variables as they related to changes in outcomes across time. Importantly, the random-effects approach tested and controlled for potential endogeneity of exposure to Shujaaz activities, thereby allowing for unbiased estimates of the effects of Shujaaz that simpler models would not achieve. The study faced several limitations. First, data collection was originally intended as a one-time cross-sectional survey and not a panel. For the cross-sectional survey, Research Plus Africa (the subcontractor in this study) randomly selected and interviewed N = 2,011 youth aged 15–24. While routine contact data were collected, participants were not originally advised of the panel-nature of the study, as that decision was made subsequent to the first round of data collection. Re-contact with participants, who represent a socially and geographically mobile population, was therefore not attempted until the second round of interviews (Shujaaz 360 Kenya, year-end report, 2017, full report).

As discussed in the section on the sample design, the panel study team made a plan for reaching as many youths as possible from the original sample. This involved (1) making phone calls to primary contacts to confirm the respondents’ location and schedule an interview, (2) making phone calls to secondary and tertiary phone contacts to inquire from families and neighbours the location of the respondent, and (3) making physical visits to the respondents’ original locations to find new locations and/or obtain new phone contacts. Successful contacts resulted in immediate interviews; the calls and visits started 2 weeks before the launch of the fieldwork and continued for the duration of the fieldwork (i.e. approximately six weeks). In spite of these attempts, only 700 youth in total could be re-contacted and re-interviewed.

A second key limitation of the study was the absence of an experimental design, which inhibits inferences about causal relationships. By choice or by nature, some people were exposed to Shujaaz materials while others were not, thereby increasing the risk of selection bias; the sample of people who were exposed to Shujaaz may have been different from the sample of people who were not in ways that may confound estimates of the impact of Shujaaz. These differences may be measurable – age, education, gender, socio-economic status – or unmeasurable – how people value the present versus the future, their willingness to take risks, and their access to social support systems. Not controlling for these differences can lead to over-estimates or under-estimates of program effects. This eventuality was addressed in part by the multivariate correlated random-effects models that relaxed the assumption of correlation between control variables, including exposure to Shujaaz, and time invariant unobservables. To the extent that the relationship between exposure to Shujaaz and outcomes is confounded by time varying unobservables, some bias in estimates of the effects of Shujaaz may remain.

A final limitation was that the first cross-sectional survey was not a true baseline. Shujaaz had already been in operation for several years. As a result, some of the effects of exposure to Shujaaz may have already been experienced prior to the evaluation. This could have resulted in a situation in which the full effects of Shujaaz might be underestimated.

Program implications

This study reviews Shujaaz media’s unique BCC approach to engaging diverse groups of youth, initially through analogue channels and subsequently through social platforms, where they join peer networks to benefit from giving and receiving social support, sharing ideas, and shifting norms. As the study results indicate, this sequential engagement, rooted in the Transtheoretical Model of Change, stimulates behaviour change in the areas of health and financial security/independence.

The Shujaaz experience can be used as a case study for BCC programs that aim to shift norms, attitudes and behaviours among youth through a purposeful integration of analogue and digital media, and through addressing topics related to sexual and reproductive health and financial security. This integrated approach reflects the holistic, multi-faceted way that a typical modern adolescent gathers information for decision-making. By closely identifying with youth and utilizing youth experiences as story narratives, Shujaaz creates an authentic, recognizable and audience-friendly community, which stimulates normative and behavioural change. The journey of a Shujaaz user detailed by this study can serve (1) as a blueprint for the design of BCC interventions, and (2) as a guide for linking evaluation activities to capture, correct and reinforce shifts in sexual and financial behaviours. Future data collection will help to determine if the impact can be replicated in other socio-cultural contexts, with other types of audiences, or in other developmental sectors.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This study was part of the impact assessment activities for the 3-year grant (2014-2017) by Bill & Melinda Gates Foundation in support of the Shujaaz media initiatives aimed at advancing SRH, financial security, positive attitudes towards agriculture, and anti-smoking consensus among Kenyan youth aged 15-24