Effects of an e-reader intervention on literacy, numeracy and non-verbal reasoning among adolescent girls in Zambia: evidence from a randomised controlled trial

ABSTRACT

This paper presents results from a three-arm randomized controlled trial in Zambia with a sample of nearly 1200 adolescent girls enrolled in Grade 7. Selected primary schools (N=36) were randomly assigned within each of three districts to one of three arms: 1) e-readers, which girls could take home, provided within a safe space group platform plus community engagement activities; 2) safe space groups plus community engagement activities; and 3) control. The intent-to-treat estimates indicate that girls in the e-reader arm scored significantly better on two basic literacy assessments as well as non-verbal reasoning compared with girls in the control arm.

KEYWORDS:

• Literacy
• e-readers
• girls’ groups
• RCT
• Zambia
• sub-Saharan Africa

Introduction

As access to school has expanded, the experience of schooling has changed in many sub-Saharan African countries. The financial strain of implementing free primary education policies and accommodating higher school enrolments has compromised school quality and learning outcomes (Pritchett 2013; World Bank, & UNICEF 2009); in many settings these stresses have led to overcrowded classrooms, inadequate facilities, and poorly trained teachers (Avenstrup, Liang, and Nellemann 2004). Research has emphasised that school quality may be more important than school quantity for improving labour market participation and fostering economic development, and that failure to consider quality may lead to misinterpretation of the effect of education on health and other outcomes (Behrman and Birdsall 1983; Behrman, Ross, and Sabot 2008; Hanushek and Woessman 2007).

Between 2000 and 2010 the rate of secondary enrolment in Zambia increased 75 percentage points, a ‘historically unprecedented’ expansion (World Bank 2018, 58). Yet at the same time, learning levels remain poor. According to the Demographic and Health Survey (DHS) conducted in Zambia in 2013–2014, nearly one-quarter of women aged 20–24 could not read in any of the country’s seven major language groups compared to 11% of their male counterparts (Central Statistical Office [CSO], 2014).¹ Among the 15 countries that make up the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ), Zambian students in Grade 6 tied with Malawi for the lowest English reading score with over two-thirds of pupils reading below a minimally acceptable level.² One likely factor contributing to low literacy is that only about one-fifth of students in each district of Zambia have access to an English textbook (Musonda and Kaba 2011). Yet English is both the language of instruction starting in Grade 5, and the language of the Grade 7 Composite Exam that determines placement in secondary school.

The primary research question this 3-arm randomised control trial attempted to answer is whether GirlsRead! Zambia – an e-reader intervention embedded in a ‘safe space’ platform targeted to students in Grade 7, the last year of primary school – improved literacy. In addition, we investigated the effect of the intervention on numeracy and non-verbal reasoning. We found that girls in the e-reader arm scored significantly better than girls in the control arm on two basic literacy assessments as well as on the non-verbal reasoning assessment. While the coefficients were all positive for the more advanced literacy assessment in the e-reader arm, they were not significant. Nor were effects of e-reader exposure on numeracy observed. Self-reports further indicate that reading books is the pathway through which literacy improved: on average girls in the e-reader arm spent significantly more time reading than girls in either the control arm or the other intervention arm.

The study, with a sample of nearly 1300 adolescent girls at baseline, was conducted in Ndola and Chingola in Copperbelt province, and in Lusaka. These three districts were prioritised by DREAMS (Determined, Resilient, Empowered, AIDS-free, Mentored and Safe), an initiative to reduce HIV infections among adolescent girls and young women in 10 sub-Saharan African countries, that includes support from a consortium of private donors in addition to the US President’s Emergency Plan for AIDS Relief.³ Given the significant effect of grade attainment on reducing adult HIV infection that has been observed in empirical studies (Behrman 2015; De Neve et al. 2015), one of the DREAMS Innovation Challenge focus areas is girls’ education.

Background

Literacy levels

In many sub-Saharan African countries young women, even those who have been to school, are unable to read (World Bank 2018). An analysis of recent DHS data from 21 sub-Saharan African countries found that over 25% of female primary school graduates in 13 countries who did not attend secondary school and over 50% in 8 countries – one of which was Zambia – lacked basic literacy skills. Female disadvantages were much more likely to be observed than male disadvantages. Comparisons with surveys conducted approximately 10 years earlier revealed that in 13 countries, including Zambia, the proportion of young people with only a primary school education who had basic literacy skills had declined in recent years (Psaki, McCarthy, and Mensch 2018).⁴

Data from the Adolescent Girls’ Empowerment Programme (AGEP) a social, health and economic asset building programme for vulnerable adolescent girls aged 10–19 that took place between 2013 and 2015 in 10 sites in four provinces in Zambia indicate a low level of literacy even among girls who finished primary school; 17% of the AGEP evaluation sample (N = 5235) that completed primary school could not read a full sentence in English in either the first (2013) or second round of data collection 1 year later; 5%, who could read in Round 1, could not read in Round 2. Among those who completed fewer than seven grades, 68% could not read a full sentence in Round 1 and Round 2, and an additional 4%, who could read in Round 1, could not read in Round 2 (Hewett et al. 2017). Analysis of data from the Malawi Schooling and Adolescent survey, a six-year longitudinal study among young people in the Southern region of the country ages 14–17 when first interviewed in 2007 revealed a significant gender difference in skill level after school-leaving for English oral reading and reading comprehension. Whereas 76% of both male and female respondents were able to read English sentences aloud at school-leaving, by the last assessment this proportion had declined significantly to 70% for males and 64% for females. Multivariate analyses indicated that females had approximately 40% lower odds of being literate compared to their male counterparts after controlling for skill level at school leaving and grade attainment (Soler-Hampejsek et al. 2018). One possible reason for the greater skill loss is that the world of young women contracts after they leave school, as a result of a rapid transition to marriage and parenthood and attendant responsibilities, whereas for young men it does not. Indeed, an analysis of the effect of adolescent childbearing on academic skills using longitudinal data from Malawi, Zambia and Bangladesh, which accounted for the potential endogeneity, found that among women with low levels of grade attainment – Grade 7 and below – becoming a mother led to a deterioration of English literacy and numeracy skills gained in school (Psaki et al. 2019b).

Interventions to improve reading skills in primary school in sub-Saharan Africa have been conducted recently, focusing on supplying instructional materials and providing teacher training and support (Davidson and Hobbs 2013; Jukes et al. 2017; Piper, Zuilkowski, and Mugenda 2014). A critical factor, however, contributing to low literacy is the absence of books in homes and schools (Read 2015). A 2016 report funded by the Norwegian Agency for Development Cooperation, the UK Department for International Development and the US Agency for International Development asserted that ‘[r]eading books – which include leveled and decodable readers, story books, information books, or topic books to provide reading instruction practice – [is] particularly important in building the foundational skill of literacy’ (Results for Development 2016). Despite evidence about the importance of books for the acquisition and retention of literacy, children in sub-Saharan Africa lack reading materials.

Links between literacy and numeracy and non-verbal reasoning

While the focus of this analysis is on the effect of facilitated book groups with e-readers on literacy skills, we also included numeracy and non-verbal reasoning because prior research in high-income countries indicates that there is a link between language ability and numeracy, and between reading ability and non-verbal intelligence. A considerable literature documents an association between language and mathematical skills and lays out the potential pathways (see, for example, Purpura et al. 2011; Purpura and Reid 2016; Chow and Jacobs 2016; LeFevre et al. 2010). Not only are certain language skills, particularly mathematics-specific skills, thought to be essential for numeracy development, it is thought that ‘numeracy and language processing … share neural pathways’ (Purpura and Reid 2016, 260). A longitudinal analysis of U.S. preschoolers found that print knowledge and vocabulary, two components of literacy were predictive of early numeracy skills (Purpura et al. 2011). A longitudinal study of children ages 6–9 from Grades 1–4, both native and non-native speakers of English, found that earlier language ability was associated with later mathematical development specifically understanding mathematical concepts and representations but not with arithmetic manipulation (Vukovic and Lesaux 2013). As for the effect of reading on intelligence, a longitudinal analysis of British identical twin pairs at ages 7–16 raised in the same family, thus controlling for both genetic factors and shared household effects, found an association between earlier reading ability and subsequent assessment of both verbal and non-verbal intelligence (Ritchie, Bates, and Plomin 2015). The authors believe that orthographic learning ̶ the process by which an early reader learns to recognise the written representation of spoken language ̶ as well as the acquisition of new vocabulary, improves verbal intelligence. With regard to the effect of reading on non-verbal intelligence, while the authors are less certain about the mechanism, they suggest it has to do with enhancement of abstract thinking that arises ‘via the process of taking on the perspectives of fictional or historical characters, or imagining other worlds, times and scenarios’ (Ritchie, Bates, and Plomin 2015, 32).

Computer-assisted learning and e-readers

To the best of our knowledge, only one other⁵ study has investigated the effect of e-readers on children’s literacy in a low-income setting. However, evidence from the United States suggests that electronic reading devices may encourage students to read (Anderson 2018; Brown 2016) and that comprehension using such devices is not impaired by comparison to reading paper books (Margolin et al. 2013). However, a considerable body of evidence exists from low-income countries about the effect of computer-assisted learning in schools; here the results are mixed. McEwan (2015) conducted a systematic review of randomised experiments evaluating the effects of school-based interventions on learning in primary schools and, controlling for such moderators as study quality and country context, reported the largest positive effects for a category he labelled computers and instructional technologies. On the other hand, another systematic review by Snilstveit et al. (2016, 32) found that computer-assisted learning programmes in schools ‘have not generally had positive effects on language arts and composite test scores’. Moreover, they provided examples of computer-assisted learning where the effects were negative and noted several issues undermining positive outcomes including 1) where such programmes substituted for regular learning sessions, 2) problems with the technology, and 3) lack of integration with the existing curriculum. They indicated, however, that where computer-assisted learning was provided outside of school, benefits were demonstrated.

The GirlsRead! Intervention

GirlsRead! assessed an untested approach to bolstering literacy among adolescent girls; the programme combined ‘safe space’ groups, digital technology – e-readers designed for rural settings in low-income countries – and community engagement. Safe space groups are designed to build girls’ assets and lessen their social isolation (Austrian et al. 2016). The groups met nearly weekly over a period of 6 months, and were led by two trained female mentors, a teacher at the school and a community member. The programme was implemented by the Forum for African Women Educationalists in Zambia (FAWEZA), a locally established organisation part of the indigenous FAWE network spanning 33 African countries that was founded to foster girls’ schooling. There were 19 group sessions held during the school term, after regular school hours.

An empowerment-based curriculum covered human rights, gender equality and norms, sexual harassment, prevention of HIV, pregnancy and contraception, communication skills, self-esteem, decision-making, and financial education. E-reader activities took place in the context of the safe space groups. Each girl received her own e-reader and was taught how to use it; time was spent in the safe space groups reading and discussing books together. Mentors facilitated reading and discussion using a reading curriculum developed for the project, with books chosen that mirrored themes in the empowerment curriculum when possible.

The e-readers provided by Worldreader, an international NGO, were each loaded with approximately 100 books – primarily fiction written by African authors curated to be engaging to young people. The e-readers were charged using locally sourced solar panels installed on school roofs. The books varied in reading level from very basic readers to more sophisticated novels, many of which include strong female protagonists and non-traditional gender roles. They were written in English, the official language of instruction in that grade and, as noted above, the language of the primary school-leaving exam. Girls kept the e-readers for the duration of the project, were permitted to take them home after the second session and were encouraged to read both on their own and with siblings and caregivers.

In addition to engaging family members/caregivers through reading at home, four community engagement trainings were conducted with adults who influence girls’ opportunities and attitudes towards education to attempt to create a supportive environment for gender equality, literacy development, and secondary school attendance and completion.

Theory of change

Our theory of change involves the three components of the intervention:

• Community engagement activities emphasise the important role of stakeholders and beneficiaries in establishing a culture of reading, supporting girls’ schooling and fostering gender equality.

• Safe space groups are female-only venues designed to be free from physical or psychological threat, and to foster the development of social assets. Groups provide an opportunity to build social networks, including relationships with female role models, and increase leadership skills and agency. The GirlsRead! groups placed a special emphasis on raising educational expectations, discussing and addressing perceived barriers to continued schooling and fostering critical thinking about gender and power inequalities. Participants receive information about puberty, sexual and reproductive health, and how to prevent HIV.

• E-readers offer otherwise unavailable reading materials, with the goal of both increasing desire to read and improving literacy, as well as broadening understanding of critical issues affecting girls’ lives. If girls in the e-reader arm read the books on their e-readers, literacy will likely improve, they will remain in school longer and be more likely to progress to secondary school.

Methods

Study design

The intervention was evaluated via a cluster-randomised trial design. Selected schools (N = 36) were randomly assigned within each district to one of three arms.

• Arm 1: community engagement activities, safe space groups and e-readers

• Arm 2: community engagement activities and safe space groups

• Arm 3: control; no activities

In Arm 1, the first hour was devoted to the empowerment curriculum and the second hour was devoted to the e-readers. To ensure equivalent exposure in the intervention arms, in Arm 2, the first hour was devoted to the empowerment curriculum and the second hour was devoted to study and homework. Arm 2 was included in order for us to assess the effect of participating in a facilitated reading group with access to e-books over and above any effect from participation in the safe space groups.

Each intervention school (Arms 1 and 2) had two safe space groups for a total of 48 groups with approximately 16 girls per group for a total sample of approximately 1152. With a sample size of 1152 (36 schools – 12 per arm – and approximately 32 participants per school), power of 0.80, an alpha of 0.05, and intra-cluster correlation of 0.01 the study was designed to detect a difference of 11 percentage points from 27% to 38% for literacy (based on the percentage of pupils in Grade 6 that could read for meaning in the SACMEQ Zambia assessment). Because of expected attrition between baseline and endline of 10% and expected absenteeism of 10%, we planned to randomly sample 38 participants per school in order to achieve an endline sample of 32 participants per school.

Ethical approval for the study was received from the Population Council IRB as well as a Zambian IRB, ERES Converge. Informed consent was obtained from all participating girls after parents gave passive consent – i.e. parents/guardians were informed of the study and given the opportunity to opt out.

Sample

Because of budget constraints, a limited number of schools to choose from in the three districts and a desire to avoid schools that were implementing other DREAMS activities, the sample was selected from a small number of clusters (schools). Public (government) schools were selected in coordination with USAID to minimise overlap with other DREAMS activities; this resulted in a sampling frame of 22 (out of 268) schools in Lusaka, 14 (out of 41) schools in Ndola, and 9 (out of 44) schools in Chingola. For each district, schools were ranked by the proportion of girls aged 15 and older – i.e. a higher proportion of girls below grade for age – in order to reach the schools with more vulnerable girls. The 15 schools with the highest proportion of such girls in Lusaka and the 12 schools with the highest proportion of such girls in Ndola were selected. In Chingola all 9 schools were selected. Five schools were randomly assigned to each of the three arms in Lusaka, four randomly assigned in Ndola and three randomly assigned in Chingola.

In each school, rosters were obtained of incoming Grade 7 girls prior to the beginning of the 2017 school year. If a school’s roster contained 50 girls or more, all girls on the roster were randomly ordered and the first 38 girls were invited to participate. When the fieldwork teams arrived at the school, the list of the 38 selected girls was reviewed to determine who was currently enrolled. Those not enrolled were removed and girls from the randomly ordered list were added in sequence until we reached 38 currently enrolled Grade 7 girls. If the roster contained fewer than 50 girls, all girls in a school were invited to participate (including girls who were not on the original roster but were enrolled in school when the fieldwork team arrived) so as not to exclude a few girls from participation.

Data collection

A baseline survey was conducted in January–March 2017 prior to the launch of the intervention; non-sensitive questions were administered via a face-to-face interview with responses recorded on tablet computers and sensitive questions were administered via audio computer assisted self-interviewing. Questions were included about the respondent’s biological parents and household, schooling history, reading behaviour, work experience, social capital, self-efficacy, educational aspirations, gender attitudes, school violence, domestic violence, knowledge about reproductive health and HIV, and marriage, pregnancy and births. Sensitive questions on sexual behaviour were administered only to respondents age 15 and older. Literacy, numeracy, and non-verbal reasoning – the outcomes examined in this paper – were also assessed.

The safe space sessions began the week of March 20th and ended the week of September 18th, 2017. In total, 19 sessions plus a graduation ceremony were conducted; eight extra optional vacation sessions were held weekly during the April and August school holidays. While attendance was not mandatory and the GirlsRead! curriculum was not used, these vacation sessions were intended to provide a safe space for girls to meet and socialise, to continue reading with girls in the e-reader arm, and to provide time for girls in Arm 2 to prepare for exams or do any homework assigned over their break.

A post-intervention survey was conducted immediately after the intervention was completed in October and November 2017; the comparison of reading behaviour across arms is based on data from this survey.

The endline survey was conducted between March and May 2018, approximately 6–8 months after the completion of the intervention. Similar to the baseline, the endline also included literacy, numeracy and non-verbal reasoning assessments, and collected data on a variety of attitudes and behaviours including gender attitudes, knowledge about reproductive health and HIV, sexual behaviour, marriage, pregnancy and births, and domestic and school-related gender-based violence.

Measurement of skills and non-verbal reasoning

For this analysis, literacy and numeracy were measured via an interviewer-administered assessment developed for 6–16-year-olds by the UWEZO East Africa group (http://www.uwezo.net/), which is part of the People’s Action for Learning (PAL) Network for Anglophone African countries.⁶

The literacy assessment, developed in English and set at the Grade 2 level, is a diagnostic of individual progress in reading that tests letter recognition, reading simple words and comprehension. The assessment consists of five components of increasing difficulty: reading letters out loud, reading simple words out loud, reading a short paragraph out loud, reading a two-paragraph story out loud and answering two questions about the story. To pass the letter recognition section, each girl was asked to choose five out of ten letters listed and successfully identify four out of five. To pass the ‘reading words’ section, each girl was asked to choose five out of ten simple words listed and successfully read four out of five. To pass the paragraph reading section, each girl was asked to choose one of two 20–22-word paragraphs to read and make two or fewer mistakes. To pass the two-paragraph story section, each girl had to read an 81-word story, make four or fewer mistakes and answer two questions about the story. The cut-offs for passing each section and administering subsequent sections were implemented per UWEZO instructions. The total possible score was six, one point for ‘passing’ each of the first four components plus one point for each story question. At baseline, administration of each section was dependent on passing the preceding section.

Because the UWEZO assessment was pitched at a low level, a 32-item multiple-choice literacy test that included questions from a prior Zambian Grade 7 Composite examination (primary school-leaving exam) as well as from Zambian Grade 3 and 5 English textbooks – was administered at baseline to those who could ‘pass’ the UWEZO basic assessment, which was defined as successfully reading the short paragraph. That restriction, which was intended to minimise the frustration of those participants with a low level of literacy, meant that about half of girls were not given this more advanced test at baseline. At endline, we removed the restriction and administered the multiple choice to all girls regardless of their performance on the UWEZO assessment. The UWEZO and multiple-choice assessments measure different aspects of literacy. The UWEZO assessment reflects reading fluency and comprehension whereas the multiple choice focuses on grammar and specific meaning of words, elements that were not covered in the GirlsRead! intervention. We included analysis of the more advanced literacy assessment as well as the numeracy and non-verbal reasoning tests because we hypothesised that there might be an effect of increased exposure to reading on these other outcomes if basic literacy skills form an important underpinning for other cognitive and more advanced academic skills.

The numeracy assessment, also set at a Grade 2 level, measured students’ foundational skills including number identification, counting, word problem-solving, addition, subtraction, multiplication and division. To pass the number identification and count sections, the girl had to select five of eight options and correctly answer four of the five. To pass the addition, subtraction, multiplication and division sections, the girl had to select three of six questions for each and correctly answer two of the three. There was also one word problem, labelled ‘ethno’ or everyday maths, which the interviewer administered in the language chosen by the girl – English, Nyanja or Bemba. The total possible score was seven, one point for ‘passing’ each of the seven components. For the maths operation sections, the girl could either provide the answer orally or write the answer down and show the interviewer. At baseline, administration of each section except the ethno maths question was dependent on passing the preceding section. At endline that restriction was removed and all sections were administered regardless of whether the girl passed the prior section.

As with literacy, because the UWEZO assessment was pitched at a Grade 2 level we also included a multiple-choice test with 23 questions culled from Grade 3 and 5 mathematics textbooks as well as a prior Grade 7 Composite exam. Questions assessed the ability to do 3-digit addition, more complex multiplication problems, addition and subtraction of simple fractions, word problems, areas and perimeters of squares, basic graphs and percentages.

To measure non-verbal reasoning, we administered a subset of Raven’s Coloured Progressive Matrices.⁷ The Coloured Progressive Matrices consist of three sets (12 each) of coloured shapes and patterns (matrices) each of which is missing a piece. It is designed for children ages 5–11 and is considered easier than the five-set Standard Progressive Matrices. Each girl was told that the exercise was a game or puzzle and in English, Bemba or Nyanja – language selection was up to the respondent – she was instructed to select the missing piece that fit the pattern from a set of six options. Each girl was shown 18 matrices of increasingly complex geometric designs. The first two were done as practice to make sure she had understood the instructions and were not scored.

There has been discussion in the literature about the validity of Raven’s Progressive Matrices as a measure of innate intelligence for sub-Saharan Africa especially given the confounding that has been observed between such measures and the level of development (Wicherts et al. 2010; Raven 2000; Raven and Court 1998). A systematic review of published data on Raven’s tests in sub-Saharan African populations found that reliability is generally high (above 0.80) and that predictive validity was observed in several studies at levels comparable to western samples, but that convergent validity – the degree to which the scores correlate with other assessments of cognitive ability – is considerably lower by comparison to Western samples. The review concluded that Raven’s tests are ‘relatively weak indicators of general intelligence’ in this setting. While there is some justification in criticism of Raven’s testing as culturally determined because of the lack of familiarity with multiple-choice tests and coloured geometric shapes among those with low levels of schooling, these issues are less of a concern with the girls in our sample especially given that a similar assessment of non-verbal reasoning, ‘Special Paper 2’ is one of the eight mandatory exams that make up the Grade 7 Composite Examination. In our analyses we refer to the Ravens outcome as a measure of reasoning rather than innate ability to reflect the possibility that the score can change over time (Behrman et al. 2014).

Statistical analysis

To assess the effect of the e-reader and safe space interventions we used analysis of covariance (ANCOVA) to estimate differences between arms in the post-intervention outcomes of interest controlling for the values at baseline for the following outcomes⁸:

1. UWEZO literacy – passed paragraph: respondent read letters, read simple words, read paragraph aloud containing 3 simple sentences and 20–22 words and made 2 or fewer mistakes.

2. UWEZO literacy – passed story: respondent could do all components listed in ‘#1’ plus could read a two paragraph 12-sentence story out loud and made 4 or fewer mistakes.

3. UWEZO literacy – passed all sections (scored 6): respondent could do all components listed in ‘#2’ plus could correctly answer 2 questions about the story.

4. UWEZO literacy – score 0–6

5. Multiple-choice literacy – score 0–32; two variants:

   1. baseline score imputed if multiple choice not administered

   2. limited to participants for whom the multiple choice was administered at baseline

6. UWEZO numeracy – passed all sections (scored 7); respondent passed all sections (counting, number recognition, addition, subtraction, multiplication, division and ‘ethno’ maths.)

7. UWEZO numeracy – score (0–7)

8. Multiple-choice numeracy – score 0–23; two variants:

   1. baseline score imputed if multiple choice not administered

   2. limited to participants for whom the multiple choice was administered at baseline

9. Non-verbal reasoning (Raven’s) – score 0–16⁹

Note that the UWEZO literacy outcomes were separated into more components than the UWEZO numeracy both because English literacy was our primary outcome and because the level of English literacy was lower than the level of numeracy. The procedure for administering the more advanced sections of the UWEZO literacy and numeracy tests and the multiple-choice assessments differed for the baseline and the endline. However, for the analysis we computed the UWEZO endline scores as if the administration was conducted the same as the baseline. That is, if the girl did not pass the prior section, she was given a score of 0 for the subsequent section. For girls at the baseline who did not pass the UWEZO literacy and/or numeracy, we imputed multiple-choice scores. For literacy, the imputed baseline score was based on the mean multiple-choice scores at endline for girls who scored 0–2 in the UWEZO literacy at endline.¹⁰ For numeracy, the imputed baseline score was based on the median multiple-choice scores at endline for girls who scored 0–5 in the UWEZO numeracy at endline.¹¹ As a robustness check we also estimated models limited to the sample of girls who passed the relevant UWEZO sections and for whom the multiple choice was actually administered at baseline.

We compared each arm to the control arm, following an intent-to-treat (ITT) approach. The ITT analysis includes all girls enrolled in the research study who were attending a school randomly assigned to a specific arm, regardless of their actual exposure to the intervention. All models were estimated with robust standard errors adjusted for clustering at the school level. Statistical analysis was conducted using Stata 15.1.

Formally, the following linear regression model was estimated for each outcome of interest at endline¹²:

$Y_{ijt}={\alpha }_{\mathbf{0}}+{\alpha }_{\mathbf{1}}{S^1}_{ij(t=0)}+{\alpha }_{\mathbf{2}}{S^2}_{ij(t=0)}+{\alpha }_{\mathbf{3}}{Y}_{ij(t=0)}+{\alpha }_{\mathbf{4}}{X}_{ij(t=0)}+{\mathbf{e}}_{ijt}$

where Y_ijt is the outcome of interest for girl i in school j at time t (t = 0 is baseline), S¹ is a dichotomous variable for a girl enrolled in a school assigned to the e-reader intervention (Arm 1), S² is a dichotomous variable for a girl enrolled in a school assigned to the safe spaces intervention (Arm 2), X is a vector of control variables that include district and other covariates measured at baseline to account for any observed differences pre-intervention between the treatment and control arms. The following covariates measured at baseline were included: age at last birthday, ever repeated a grade, English spoken at home, mother’s education, father’s education, mother alive/co-resident status, father alive/co-resident status, count of household items, books in the home, ever use of a cell phone, and student–teacher ratio. The coefficients related to Arm 1 and Arm 2, α₁ and α₂, provide the estimates of the treatment effects. To determine whether there were differences between Arm 1 and Arm 2, tests of equality of coefficients α₁ and α₂ were conducted.

We also investigated whether the level of attendance was correlated with outcomes with attendance measured as a categorical variable. Formally the following linear regression model was estimated for each outcome of interest:

${\mathbf{Y}}_{\mathbf{t}}={\beta }_{\mathbf{0}}+{\beta }_{\mathbf{1}}{\mathbf{A}}^{\mathbf{L}}+{\beta }_{\mathbf{2}}{\mathbf{A}}^{\mathbf{M}}+{\beta }_{\mathbf{3}}{\mathbf{A}}^{\mathbf{H}}+{\beta }_{\mathbf{4}}{\mathbf{Y}}_{\mathbf{0}}+{\beta }_{\mathbf{5}}{\mathbf{X}}_{\mathbf{0}}+{\mathbf{e}}_{\mathbf{t}}$

where Y_t = outcome at time t (endline), A^L = low attendance (13–15 sessions), A^M = mid attendance (16–17 sessions), A^H = high attendance (18–19 sessions),Y₀ = outcome at t = 0 (baseline), and X₀ = other covariates at baseline.

Despite randomisation, the arms were slightly unbalanced; as a robustness check we used entropy balancing to generate weights so that the safe spaces and control arms matched the e-reader arms on specified covariates and outcomes measured at baseline (Hainmueller 2012; Hainmueller and Xu 2013). Entropy balancing differs from propensity score matching, a more commonly used method to address unbalanced samples, in that a functional form is not assumed since treatment is not being modelled. In entropy balancing, “the user starts by specifying a desired level of covariate balance using a set of balance conditions. Entropy balancing then finds a set of weights that satisfies the balance conditions and remains as close as possible (in an entropy sense) to uniform base weights to prevent loss of information and retain efficiency for the subsequent analysis” (Hainmueller and Xu 2013: 2). In other words, each girl is weighted in order to minimise the distance between the covariate distributions for the safe spaces and control arms compared to the e-reader arm. Not only can the matching be done on covariate means it can also include variance and skewness. For each analysis, weights were created with the default tolerance level set at .015. Covariates used to generate weights were: passed UWEZO paragraph at baseline, age at last birthday, Ravens score at baseline, ever repeated a grade, English spoken at home, mother’s education, father’s education, mother alive/co-resident status, father alive/co-resident status, count of household items, books in the home and ever use of a cell phone.

Results

Baseline sample and attrition

Out of 1364 eligible girls, 1299 (or 95.2%) were interviewed for the baseline. Of the 64 girls we were unable to interview, 58 refused participation and six were not in attendance. There were no differences in completion by arm. The reason for the larger sample than was targeted is that more schools than expected had enrolments greater than 38 but less than 50. As noted above, if a school contained more than 38 girls but fewer than 50, all girls were interviewed at baseline so as not to exclude a few girls from participating. Note that based on the roster provided prior to fieldwork, one Arm 2 school appeared to have more than 38 girls but had fewer than 50 and, thus, all were designated to be included based on the sampling design. However, when the field team arrived, they discovered there were 56 girls at that school, all of whom were thus invited to participate. Note, also, that the smallest school in Arm 2 was considerably larger (33 girls interviewed) than the smallest schools in Arm 1 (23 girls interviewed) and Arm 3 (20 girls interviewed). On average, more girls per school were included in the Arm 2 sample than in the other arms.

Of the 1299 girls who were interviewed at baseline, 1178, 90.7%, completed the endline (89.5% in Arm 1, 92.4% in Arm 2 and 90.0% in Arm 3); differences between arms in retention between the two surveys were not statistically significant. The main reason for non-interview at endline was that the girl was not located or had moved (N = 82). See Table A1 for a sample flow chart that includes details on the sampling design and attrition.

Given the larger sample, the safe space groups were also slightly bigger than anticipated. A total of 895 girls were assigned to the 48 safe space groups, with an average of 18.6 girls per group (range 12–28). Ten of the groups contained more than 20 girls and three contained fewer than 16.

Attendance and e-reader loss

Attendance at the safe spaces sessions was high with over three-quarters of girls on average attending the 19 sessions; attendance did not vary significantly between the two intervention arms. Of the 425 e-readers distributed to 421 girls (4 e-readers were replacements), only 10 were lost or stolen and three were broken over the course of the intervention period.¹³

Descriptive results

Table 1 provides background characteristics as well as the results of the literacy, numeracy and non-verbal reasoning assessments at baseline separately by study arm. We have computed significance tests comparing background characteristics across arms using linear regressions accounting for clustering at the school level. The final column provides p-values from a joint test and indicates that there are no significant differences despite the fact that the arms were unbalanced. (Table A2 provides summary statistics for all background characteristics for the full baseline sample as well as p-values for differences across arms.) Grade repetition is common and scores on the basic literacy assessment were low. Although the UWEZO assessments were pitched at a Grade 2 level, only 39% of Grade 7 girls successfully completed all the literacy components. As noted above, despite randomisation, study arms were not balanced. Girls attending schools in the e-reader arm performed better at baseline on the basic literacy and numeracy and non-verbal reasoning assessments than girls in the safe spaces-only or control arms and were of higher socio-economic status, although they did score slightly lower than the control arm on the multiple-choice literacy assessment.

Table 1. Selected background characteristics, literacy and numeracy assessments, and non-verbal reasoning scores at baseline by arm

Baseline characteristic/assessment	Total (N = 1178)	Arm 1 safe spaces + e-reader (N = 374)	Arm 2 safe spaces (N = 425)	Arm 3 control (N = 379)	P-value joint test
Age at last birthday	12.8 (SD 1.3)	12.8 (SD 1.3)	12.8 (SD 1.3)	12.9 (SD 1.3)	0.952
English spoken at home	25%	26%	24%	25%	0.956
Ever repeated a grade	34%	33%	33%	36%	0.644
Count of household items (0–15)	8.1 (SD 2.8)	8.5 (SD 2.5)	8.0 (SD 2.9)	7.7 (SD 3.0)	0.289
Mother’s education < junior secondary^a	36%	33%	36%	37%	0.844
Mother’s education: don’t know	19%	20%	16%	21%	0.262
Father’s education < junior secondary^a	14%	14%	14%	14%	0.993
Father’s education: don’t know	29%	27%	27%	32%	0.451
Passed UWEZO paragraph^b	53%	58%	48%	53%	0.421
Passed UWEZO story^c	44%	48%	40%	46%	0.542
Passed all UWEZO literacy (scored 6)	39%	41%	35%	40%	0.664
UWEZO literacy score (0–6)	3.4 (SD 2.2)	3.7 (SD 2.2)	3.2 (SD 2.3)	3.5 (SD 2.3)	0.499
Literacy multiple choice score^d(0–32)	13.7 (SD 6.7)	13.8 (SD 6.4)	13.2 (SD 6.5)	14.2 (SD 7.1)	0.782
Passed all UWEZO numeracy (scored 7)	63%	69%^†	60%	62%	0.111
UWEZO numeracy score (0–7)	6.2 (SD 1.4)	6.3 (SD 1.3)^††	6.0 (SD 1.5)	6.2 (SD 1.3)	0.234
Numeracy multiple choice score^d (0–23)	8.4 (SD 4.7)	8.9 (SD 4.6)	7.8 (SD 4.7)	8.7 (SD 4.8)	0.384
Non-verbal reasoning score^e (0–16)	8.9 (SD 3.3)	9.1 (SD 3.2)	8.6 (SD 3.4)	9.0 (SD 3.2)	0.684

P-values from F tests for differences across arms accounting for clustering at the school level to test for baseline balance for the sample interviewed at endline.

^†p < .1 difference Arm 1 vs Arm 3; ^††p < .1 difference Arm 1 vs Arm 2

^a<Junior secondary is equivalent to ≤ 8 years of schooling

^bPassing the paragraph indicates the respondent could read letters, read simple words and read a paragraph out loud containing 3 simple sentences and 20–22 words and make 2 or fewer mistakes.

^cPassing the story indicates the respondent could do all components listed in ‘a’ plus could read a two paragraph 12 sentence story out loud and make 4 or fewer mistakes.

^dScores were imputed for girls who did not pass the UWEZO

^eNon-verbal reasoning score is the total number of correct answers to 16 items from Raven’s Coloured Progressive Matrices.

We also examined whether the outcomes differed by level of attendance. The proportion passing UWEZO (scoring 6) at baseline and endline and the differences between the two by arm and level of attendance for the intervention arms are shown in Table 2. The scores on the basic literacy assessment increased for all arms but more so for the arm that included the e-reader and facilitated book group: 0.233 (56.3%) for the e-reader arm compared with 0.155 (44.8%) for the safe spaces only arm and 0.140 (34.9%) for the control arm. While girls in the e-reader arm who attended all or nearly all (18–19) sessions had greater improvement in their UWEZO literacy scores by comparison to those who attended fewer sessions that was not the case for those in the safe spaces only arm even though those in the safe spaces only arm had a slightly higher baseline score; indeed, among girls who attended 18–19 sessions, the increase for the e-reader arm girls was 0.290 (57.4%) compared to 0.148 (28.8%) for the safe spaces only arm girls. Note, however, that variability in outcomes by attendance levels could reflect differences in unobserved student characteristics rather than a causal effect of greater exposure to the intervention. For example, students with higher levels of motivation might attend more sessions and have larger gains in literacy not because they attended more sessions. Similarly, the associations may be in the opposite direction for students with greater domestic responsibilities that drive both low attendance and no or smaller gains in literacy. (Table 3 provides the same descriptive analysis but accounts for the unbalanced arms via entropy weighting and indicates that whatever underlying differences exist between arms does not affect the overall finding regarding greater improvement in literacy among those who had access to e-readers and facilitated book groups; the percentage increases are reduced slightly to 39.7% for the safe spaces only arm and 33.6% for the control arm compared to 56.3% for the e-reader arm).

Table 2. Proportion passing UWEZO literacy (scoring 6) at baseline and endline by number of sessions attended

Arm 1 – E-reader
# of sessions attended	N	Baseline	Endline	Difference
<13	84	0.274	0.476	0.202
13–15	83	0.398	0.602	0.205
16–17	100	0.450	0.670	0.220
18–19	107	0.505	0.794	0.290
Total	374	0.414	0.647	0.233
Arm 2 – Safe spaces only
<13	107	0.224	0.402	0.178
13–15	95	0.253	0.432	0.179
16–17	108	0.370	0.491	0.120
18–19	115	0.513	0.661	0.148
Total	425	0.346	0.501	0.155
Arm 3 – Control
Total	379	0.401	0.541	0.140

Table 3. Proportion passing UWEZO literacy (scoring 6) at baseline and endline by number of sessions attended with entropy weighting

Arm 1 – E-reader
# of sessions attended	N	Baseline	Endline	Difference
<13	84	0.274	0.476	0.202
13–15	83	0.398	0.602	0.205
16–17	100	0.450	0.670	0.220
18–19	107	0.505	0.794	0.290
Total	374	0.414	0.647	0.233
Arm 2- Safe spaces only
<13	107	0.290	0.503	0.213
13–15	95	0.331	0.564	0.233
16–17	108	0.443	0.556	0.113
18–19	115	0.590	0.729	0.138
Total	425	0.426	0.595	0.169
Arm 3 – Control
Total	379	0.450	0.601	0.151

Regression results: effect on literacy

The descriptive results are suggestive of a positive effect of the e-reader. The models shown in Table 4 address the primary question of this analysis, namely did the e-reader arm improve literacy compared to the control arm. Note that all regression tables also include the entropy weighted analyses; those results are discussed below in the section on robustness checks. With the exception of the most basic component, ‘passed paragraph’, all coefficients for the e-reader arm are positive and significant at the p < 0.10 level or better for ‘passed story’ and ‘passed story questions’ (that is, passed UWEZO – scored 6) indicating that the performance of girls in the e-reader arm at endline exceeded that of girls in the control arm. This was not the case for girls in schools assigned to safe spaces-only (Arm 2), which is not unexpected given their lack of exposure to the e-books; indeed, for all outcomes except ‘passed paragraph’ differences between the e-reader and safe spaces-only arms are significant.

Table 4 Summary of literacy results at endline

	(1)	(2)	(3)	(4)
	Control Arm 3 Endline mean	Safe spaces+ e-reader Arm 1 Coefficient (Robust SE)	Safe spaces only Arm 2 Coefficient (Robust SE)	Arm1 vs Arm 2 Coefficient (Robust SE)
UWEZO literacy
Passed paragraph (=1), unadjusted	0.694	0.036 (0.069)	-0.082 (0.093)	0.118 (0.077)
Adjusted, baseline outcome only		0.002 (0.038)	-0.047 (0.045)	0.049 (0.032)
Adjusted, baseline outcome & covariates		-0.014 (0.025)	-0.043† (0.025)	0.029 (0.020)
Adjusted with entropy weighting		-0.006 (0.025)	-0.031 (0.027)	0.024 (0.020)
Passed story (=1),unadjusted	0.599	0.080 (0.065)	-0.065 (0.097)	0.145† (0.082)
Adjusted, baseline outcome only		0.062† (0.032)	-0.025 (0.045)	0.088* (0.040)
Adjusted, baseline outcome & covariates		0.044† (0.026)	-0.020 (0.026)	0.064* (0.028)
Adjusted with entropy weighting		0.046† (0.025)	-0.001 (0.028)	0.047 (0.028)
Passed all – scored 6(=1), unadjusted	0.541	0.106 (0.077)	-0.040 (0.108)	0.146† (0.085)
Adjusted, baseline outcome only		0.097* (0.036)	-0.004 (0.053)	0.101* (0.044)
Adjusted, baseline outcome & covariates		0.074** (0.026)	0.005 (0.023)	0.069* (0.027)
Adjusted with entropy weighting		0.064* (0.025)	0.011 (0.029)	0.052† (0.029)
Score (0-6), unadjusted	4.235	0.361 (0.323)	-0.289 (0.464)	0.650 (0.384)
Adjusted, baseline outcome only		0.234† (0.137)	-0.067 (0.168)	0.301* (0.139)
Adjusted, baseline outcome & covariates		0.177 (0.114)	-0.049 (0.112)	0.226* (0.106)
Adjusted with entropy weighting		0.196† (0.112)	0.007 (0.122)	0.189† (0.104)
Multiple choice literacy
Score (0-32),unadjusted	17.858	0.423 (1.652)	-1.709 (2.022)	2.133 (1.664)
Adjusted, baseline outcome only^1		0.494 (0.464)	-0.590 (0.551)	1.084* (0.408)
Adjusted, baseline outcome & covariates^1		0.195 (0.332)	-0.591† (0.323)	0.786** (0.249)
Adjusted with entropy weighting^1		0.282 (0.345)	-0.439 (0.341)	0.721* (0.267)
Score (0-32) restricted^2,unadjusted	23.806	-0.916 (1.297)	-1.024 (1.384)	0.108 (0.983)
Adjusted, baseline outcome only		0.354 (0.505)	-0.227 (0.540)	0.580 (0.439)
Adjusted, baseline outcome & covariates		0.265 (0.457)	-0.419 (0.429)	0.684 (0.407)
Adjusted with entropy weighting		0.314 (0.439)	-0.299 (0.427)	0.613 (0.398)

Notes: The table reports endline means for the control (Arm 3) in column 1 and the estimated ITT effect for each intervention arm (Arm 1 and Arm 2) relative to the control (Arm 3) in columns 2–3. Column 4 compares the estimates for Arm 1 and Arm 2. For example, the unadjusted estimate in column 4 for ‘Passed paragraph’ (0.118) is the difference between the estimate for Arm 1 (0.036) in column 2 and the estimate for Arm 2 (-0.082) in column 3. Minor differences are due to rounding. Numbers in parenthesis indicate robust standard errors. N = 1,178 unless otherwise noted. Regressions were estimated with standard errors adjusted for 36 clusters in schools. Adjusted regressions included controls for the outcome measured at baseline and the following baseline covariates: non-verbal reasoning score, district, age, speaks English at home, ever repeated a grade, mother alive/co-resident status, father alive/co-resident status, mother’s education, father’s education, books in the household (other than textbooks), count of household items, ever used a cell phone, and student-teacher ratio. The adjusted regressions with entropy weighting reweight observations using the weights generated with entropy balancing.

¹Baseline score imputed if multiple choice was not administered, regressions include a binary control variable for imputed score.

²Among participants for whom the multiple choice was administered at baseline, N = 621.

*** p<0.001, ** p<0.01, * p<0.05, † p<0.1

In addition to estimating models assessing the effect of the intervention on the UWEZO literacy scores, we also estimated models for the multiple-choice literacy assessment. As noted above, the multiple-choice tests examined familiarity with concepts that were not explicitly covered in the e-reader sessions. The coefficients for the e-reader arm are in the expected (positive) direction for both the sample with imputed baseline scores assigned to girls who were not given the multiple choice at baseline as well as for the sample restricted to participants to whom the multiple choice was administered at baseline (N = 621). But the coefficients comparing the e-reader arm to the control are not significant. Differences, however, between the e-reader and safe-spaces only arm are significant, suggesting that reading can improve literacy beyond aspects directly covered by the intervention.

Reading behaviour

In order to assess whether behaviour changed with the introduction of e-readers, we also examined self-reports of reading in the prior month two times: 1) immediately after the safe spaces sessions ended and 2) at endline, which as indicated above, took place approximately 6 months after the intervention concluded. We asked about time spent reading on average per week that was not schoolwork or during safe spaces. Immediately post intervention, girls in the e-reader arm were more likely to have been reading, and to have been spending more time reading than girls in the other arms. In the e-reader arm immediately post-intervention, 76.7% of girls reported reading outside of safe space meetings that was not schoolwork compared to 63.3% in the safe spaces arm and 65.0% in the control arm. (Differences between the e-reader arm and the safe spaces arm, and between the e-reader arm and the control are significant at p < .001 whereas the difference between the safe spaces arm and the control arm is not significant). The percentage of girls who spent an hour or more on average each week reading also was greater immediately post-intervention for the e-reader arm: 47.1% versus 37.1% in the safe spaces arm and 33.0% in the control arm. (The difference between the e-reader arm and the safe spaces arm is significant at p=.023 and the difference between the e-reader arm and the control arm is significant at p < .001; the difference between the safe spaces arm and the control arm is not significant).

At endline, however, the average time spent reading any printed materials that were not for schoolwork in the last day did not differ much by arm: 10.5 minutes for the e-reader arm, 11.1 minutes for the safe spaces arm and 11.9 minutes for the control arm. When asked at endline to estimate the time in the last 7 days spent reading books not for schoolwork, girls in the e-reader arm reported 111 minutes versus 99 minutes for girls in the safe spaces arm and 135 minutes for girls in the control arm. This analysis indicates that when e-reader arm girls no longer had access to the device, their reading behaviour did not differ much from girls in the other arms. Note that time spent reading during the last school day (but not for school) declined from baseline to endline for girls in the e-reader arm (11.5 minutes versus 10.5 minutes) compared to girls in the safe spaces (6.0 minutes versus 11.1 minutes) and control (8.8 minutes versus 11.9 minutes) arms. This finding also indicates that the self-reports of reading behaviour are probably a reasonable reflection of reality. Were social desirability to be an issue, one would expect e-reader girls to inflate reports at endline given the emphasis placed on reading during the intervention for that arm.

Regression results: effect on numeracy and non-verbal reasoning

As noted earlier, in addition to assessing the effect on literacy we also examined whether the intervention had an effect on numeracy and non-verbal reasoning. The numeracy results are shown in Table 5. While the coefficients for the UWEZO component are positive for the e-reader arm, they are not significant. Moreover, the coefficients for the multiple-choice assessment are negative for both intervention arms. However, the differences between the e-reader arm and the safe spaces only arm are significant.

Table 5 Summary of numeracy and non-verbal reasoning results at endline

	(1)	(2)	(3)	(4)
	Control Arm 3 Endline mean	Safe spaces+ e-reader Arm 1 Coefficient (Robust SE)	Safe spaces only Arm 2 Coefficient (Robust SE)	Arm1 vs Arm 2 Coefficient (Robust SE)
UWEZO numeracy
Passed all – scored 7 (=1),unadjusted	0.728	0.055 (0.042)	-0.048 (0.054)	0.103† (0.052)
Adjusted, baseline outcome only		0.024 (0.035)	-0.041 (0.035)	0.065† (0.037)
Adjusted, baseline outcome & covariates		0.033 (0.030)	-0.022 (0.029)	0.055† (0.027)
Adjusted with entropy weighting		0.040 (0.031)	-0.005 (0.031)	0.045† (0.026)
Score (0-7), unadjusted	6.425	0.123 (0.106)	-0.098 (0.136)	0.221† (0.124)
Adjusted, baseline outcome only		0.065 (0.073)	0.005 (0.068)	0.059 (0.063)
Adjusted, baseline outcome & covariates		0.066 (0.056)	0.021 (0.057)	0.045 (0.044)
Adjusted with entropy weighting		0.092 (0.062)	0.069 (0.062)	0.023 (0.045)
Multiple choice numeracy
Score (0-23), unadjusted	11.148	0.189 (0.771)	-1.141 (1.010)	1.330 (0.852)
Adjusted, baseline outcome only^1		-0.007 (0.404)	-0.426 (0.398)	0.419 (0.356)
Adjusted, baseline outcome & covariates^1		-0.219 (0.217)	-0.465 (0.282)	0.246 (0.205)
Adjusted with entropy weighting^1		-0.175 (0.239)	-0.267 (0.322)	0.092 (0.239)
Score (0-23) restricted^2, unadjusted	12.415	-0.027 (0.664)	-0.805 (0.772)	0.778 (0.596)
Adjusted, baseline outcome only		-0.065 (0.371)	-0.560 (0.389)	0.495 (0.339)
Adjusted, baseline outcome & covariates		-0.244 (0.208)	-0.587* (0.277)	0.343 (0.214)
Adjusted with entropy weighting		-0.197 (0.221)	-0.423 (0.307)	0.226 (0.236)
Non-verbal reasoning
Score (0-16), unadjusted	10.443	0.546 (0.468)	-0.286 (0.572)	0.832† (0.450)
Adjusted, baseline outcome only		0.468*(0.190)	-0.042 (0.229)	0.510** (0.173)
Adjusted, baseline outcome & covariates		0.363* (0.163)	-0.023 (0.172)	0.386* (0.160)
Adjusted with entropy weighting		0.331† (0.174)	0.051 (0.193)	0.281 (0.167)

Notes: The table reports endline means for the control (Arm 3) in column 1 and the estimated ITT effect for each intervention arm (Arm 1 and Arm 2) relative to the control (Arm 3) in columns 2–3. Column 4 compares the estimates for Arm 1 and Arm 2. Minor differences are due to rounding. Numbers in parenthesis indicate robust standard errors. N = 1,178 unless otherwise noted. Regressions were estimated with standard errors adjusted for 36 clusters in schools. Adjusted regressions included controls for the outcome measured at baseline and the following baseline covariates: passed UWEZO paragraph, non-verbal reasoning score, district, age, speaks English at home, ever repeated a grade, mother alive/co-resident status, father alive/co-resident status, mother’s education, father’s education, books in the household (other than textbooks), count of household items, ever used a cell phone, and student-teacher ratio. The adjusted regressions with entropy weighting reweight observations using the weights generated with entropy balancing.

¹Baseline score imputed if multiple choice was not administered, regressions include a binary control variable for imputed score.

²Among participants for whom the multiple choice was administered at baseline, N = 959.

*** p<0.001, ** p<0.01, * p<0.05, † p<0.1

We also examined whether exposure to the e-reader improved non-verbal reasoning as measured by performance on the subset of Raven’s Progressive Coloured Matrices. Results are provided in the lower section of Table 5. The coefficient for the e-reader arm is positive and significant whereas the coefficient for the safe spaces-only arm is negative. Girls in the e-reader arm scored .363 points (2.3%) higher than girls in the control arm.

Regression results: Was the intervention more effective for low-performing students?

Another question we considered was whether the e-reader intervention was more effective for low-performing students than for high-performing students. We ran separate models for the subsamples who passed and did not pass the UWEZO paragraph at baseline. To test for differences in the two subsamples we estimated a model with the full sample including interactions of all covariates with ‘passed UWEZO paragraph at baseline’. These results are shown in Table 6 and indicate that the effect of the intervention on endline literacy did not vary by baseline literacy but that the effect on non-verbal reasoning is driven by the subsample that did not pass the UWEZO paragraph at baseline. In other words, the intervention had an effect on non-verbal reasoning only among those with lower literacy at baseline.

Table 6 Summary of results at endline by passed UWEZO paragraph at baseline

	(1)	(2)	(3)	(4)
	Control Arm 3 Endline mean	Safe spaces+ e-reader Arm 1 Coefficient (Robust SE)	Safe spaces only Arm 2 Coefficient (Robust SE)	Arm1 vs Arm 2 Coefficient (Robust SE)
UWEZO literacy
Passed all – scored 6 (=1)
Did not pass UWEZO paragraph at baseline (N=557)	0.180	0.027 (0.037)	-0.038 (0.034)	0.064 (0.039)
Passed UWEZO paragraph at baseline (N=621)	0.861	0.058 (0.042)	0.035 (0.037)	0.023 (0.039)
Difference		0.031 (0.053)	0.073 (0.056)	-0.041 (0.054)
UWEZO numeracy
Passed all – scored 7 (=1)
Did not pass UWEZO paragraph at baseline (N=557)	0.528	0.022 (0.046)	-0.045 (0.045)	0.067 (0.049)
Passed UWEZO paragraph at baseline (N=621)	0.905	0.013 (0.027)	-0.025 (0.033)	0.038 (0.026)
Difference		-0.009 (0.048)	0.020 (0.054)	-0.029 (0.056)
Non-verbal reasoning
Score (0-16)
Did not pass UWEZO paragraph at baseline (N=557)	8.713	0.767** (0.234)	0.040 (0.204)	0.727*** (0.185)
Passed UWEZO paragraph at baseline (N=621)	11.975	-0.111 (0.179)	-0.118 (0.200)	0.007 (0.198)
Difference		-0.878** (0.289)	-0.158 (0.270)	-0.720** (0.226)

Notes: The table reports endline means for the control (Arm 3) in column 1 and the estimated ITT effect for each intervention arm (Arm 1 and Arm 2) relative to the control (Arm 3) in columns 2–3. Column 4 compares the estimates for Arm 1 and Arm 2. Minor differences are due to rounding. Numbers in parenthesis indicate robust standard errors. Regressions included controls for the following baseline covariates: non-verbal reasoning score, district, age, speaks English at home, ever repeated a grade, mother alive/co-resident status, father alive/co-resident status, mother’s education, father’s education, books in the household (other than textbooks), count of household items, ever used a cell phone, and student-teacher ratio. Regressions for UWEZO numeracy also controlled for the outcome measured at baseline. Difference refers to the interaction term between arm and passed UWEZO paragraph at baseline from regressions using the full sample that included interactions between all covariates and passed UWEZO paragraph at baseline. Regressions were estimated with standard errors adjusted for 36 clusters in schools.

*** p<0.001, ** p<0.01, * p<0.05, † p<0.1

Descriptive results including attendance

Although attendance was high with over half of girls attending 16 or more of the 19 sessions, given the descriptive findings indicating an association of better attendance with the UWEZO score for girls in the e-reader arm, which was not observed in the safe spaces arm, we also estimated regression models that investigated whether the level of attendance was correlated with the UWEZO literacy and non-verbal reasoning, the two outcomes significantly affected by the intervention. We examined whether within each intervention arm attendance is associated with the endline outcome, controlling for the baseline outcome and other covariates. We also estimated a model with interactions that examined whether the association of attendance with the endline outcome differed between the two intervention arms, controlling for the baseline outcome and other covariates. The results of these descriptive models, with attendance measured as a categorical variable, are shown in Table 7 and Table 8. For Arm 1, the e-reader arm, compared to attendance of 0–12 sessions, attendance at 18–19 sessions is associated with a 0.160 higher probability of passing the UWEZO (p = 0.010). Attending 13–15 or 16–17 sessions is not statistically significantly different. For Arm 2, the safe spaces only arm, attendance is not significant. For non-verbal reasoning, in contrast, attendance is significant for girls in Arm 2 but not for girls in Arm 1. In Arm 2, attending 18–19 sessions is associated with a 0.772 higher Ravens score (p = 0.013). As for the models with the interactions, none are individually significant for the literacy model, but the joint test shows that the seven terms are jointly marginally significant. For non-verbal reasoning five of the seven interaction terms are individually significant, and the coefficients for 16–17 sessions and 18–19 are negative indicating that the association of the Ravens score with attendance is lower for the e-reader arm relative to the safe spaces only arm.

Table 7. The effect of attendance on UWEZO literacy and non-verbal reasoning at endline: ANCOVA models for each intervention arm separately^a

		UWEZO Passed all (scored 6)		Non-verbal reasoning score (0–16)
		Coefficient (Robust SE)		Coefficient (Robust SE)
Arm 1 – e-reader
	≤12 sessions (ref)
	13–15 sessions	−0.007 (0.067)		0.191 (0.361)
	16–17 sessions	0.069 (0.064)		−0.380 (0.381)
	18–19 sessions	0.160* (0.061)		−0.348 (0.367)
	Joint test: F	3.53*		1.28
	Number of observations	374		374
Arm 2 – safe spaces only
	≤12 sessions (ref)
	13–15 sessions	−0.013 (0.047)		0.548 (0.337)
	16–17 sessions	−0.047 (0.047)		0.701* (0.331)
	18–19 sessions	0.029 (0.048)		0.772* (0.309)
	Joint test: F	0.94		2.40†
	Number of observations	425		425

**p <.01; *p <.05; †p <.1

^aLinear regressions with robust standard errors adjusted for 36 clusters in schools. Models control for the outcome at baseline and the following baseline covariates: district, age, cognitive score, speaks English at home, ever repeated a grade, mother alive/co-resident status, father alive/co-resident status, mother’s education, father’s education, books in the household (other than textbooks), count of household items, ever used a cell phone, and student–teacher ratio.

Table 8. Interaction of attendance with UWEZO literacy and non-verbal reasoning at endline ANCOVA models^a

	UWEZO Passed all (scored 6)		Non-verbal reasoning score (0–16)
	Coefficient (Robust SE)		Coefficient (Robust SE)
Arms 1 & 2 only
Arm1	0.019 (0.062)		1.070** (0.362)
13–15 sessions	−0.016 (0.045)		0.484* (0.173)
16–17 sessions	−0.046 (0.065)		0.617 (0.366)
18–19 sessions	0.046 (0.045)		0.763* (0.335)
13–15 sessions*arm1	−0.005 (0.072)		−0.313 (0.463)
16–17 sessions*arm1	0.089 (0.084)		−1.033† (0.504)
18–19 sessions*arm1	0.095 (0.086)		−1.151* (0.409)
Joint test: F	2.3†		4.11**
Number of Observations	799		799

**p <.01; *p <.05; †p <.1. ^aLinear regressions with robust standard errors adjusted for 36 clusters in schools. Models control for the outcome at baseline and the following baseline covariates: district, age, cognitive score, speaks English at home, ever repeated a grade, mother alive/co-resident status, father alive/co-resident status, mother’s education, father’s education, books in the household (other than textbooks), count of household items, ever used a cell phone, and student–teacher ratio.

While it is plausible that attending more e-reader sessions increases girls’ ability to read out loud and comprehend what they are reading, although as noted above this may simply be due to a selection effect, we cannot readily explain the attendance results for non-verbal reasoning. While the Ravens score for girls in the safe spaces only arm was not significantly different from the score for girls in the control arm (see Table 7), those who attended 18–19 sessions did improve more than girls in the e-reader arm. The girls in the safe spaces arm were supposed to be studying and doing homework during sessions; perhaps those who attended 18–19 sessions spent more time practicing questions similar to the Ravens, since questions like those on the Ravens are included in the Grade 7 Composite exam. Nonetheless, if reading ability is related to non-verbal reasoning, we would expect that improving reading ability would improve non-verbal reasoning; given that reading ability improvement was greater among those with higher attendance, we would expect to observe the same finding for non-verbal reasoning.

Robustness checks

We also estimated regression models with entropy weighting to determine if adjusting the sample so that the safe spaces and control arms were comparable to the e-reader arm on observed baseline characteristics and skills altered our conclusions. These results are displayed in Tables 4 and 5 for our main models, that is for all models except those investigating the effect of attendance. The estimated effects are very similar for the weighted and unweighted models. For UWEZO literacy, coefficients in the weighted analyses for the e-reader arm are slightly larger and standard errors slightly smaller such that effects for models estimating the overall score (0–6) that were not significant for the unweighted sample are marginally significant, and effects for models estimating ‘passed story’ that were marginally significant in the unweighted analyses are significant at p < 0.05 in the weighted analyses. On the other hand, for non-verbal reasoning, coefficients in the unweighted analyses for the e-reader arm that were significant at p < 0.05 are only marginally significant in the weighted analyses. However, given that the differences in the coefficients and standard errors are small our overall conclusions remain unchanged.

Limitations

As we noted in the discussion of the sample, the choice of districts was dictated by the donor. Schools likely differ systematically across districts. For example, the student–teacher ratio for the full baseline sample (N = 1299) was 37.6 (SD = 22.4) for Lusaka, 27.5 (SD = 9.8) for Ndola and 33.6 (SD = 14.2) for Chingola. While we control for district and student teacher ratio at the school level in all adjusted models, undoubtedly there were other differences across districts that may affect student outcomes.

The main limitation of this analysis is that, despite randomisation, the arms were not balanced at baseline, which is primarily a function of the relatively small number of clusters (schools). While the entropy weighting attempts to address this imbalance, it cannot account for any unobserved heterogeneity between arms. A second drawback is that our more advanced assessments – a 32-item multiple-choice literacy test and a 23-item multiple-choice numeracy test – were only given to those who could ‘pass’ the UWEZO basic assessment at baseline. That restriction, which was intended to minimise the frustration of those participants with a low level of literacy and/or numeracy, meant that at baseline nearly half of girls were not administered the more advanced literacy and almost 20% were not administered the more advanced numeracy assessment. Such a small sample for literacy meant we were not really able to determine whether GirlsRead! had an effect on the more advanced skills needed for secondary school, although as indicated above, the intervention did not focus on these advanced skills given the very basic levels of literacy observed at baseline. A third limitation relates to selectivity that may affect attendance models; while we control for observed factors that potentially affect participation, we cannot account for unobserved factors. A fourth shortcoming is that we did not examine whether the intervention affected transition to secondary school, one of our initial objectives, as sample schools apparently have a process of social promotion from Grade 7 to Grade 8. Perhaps this is not surprising given that 35 of the 36 schools extended beyond primary with 34 of 35 containing the 2 years of junior secondary and one containing the three additional years of higher secondary. At endline while the percentage of girls still enrolled in school was high, it did not vary by arm: 80.8% in Arm 1, 80.5% in Arm 2 and 79.7% in the control arm. Those enrolled in school were overwhelmingly attending Grade 8; only between five and eight girls per arm were repeating Grade 7. Finally, in the absence of additional data collection, we were unable to investigate whether the effects we observed would be sustained beyond 6 months and whether the boost in basic literacy and non-verbal reasoning among girls in the e-reader arm would make a difference in girls’ performance in secondary school where considerably more advanced literacy skills are required.

Discussion and conclusions

The rapid pace of technological change in sub-Saharan Africa indicates that innovations, including e-readers and mobile access to books, are potentially a more realistic and cost-effective long-term solution to the chronic shortage of reading materials than expanding access to paper books. While rigorous studies of textbook provision have found no effect on learning outcomes, when learning materials are provided in the context of structured pedagogy programmes with a tailored curricula such interventions have been found effective¹⁴ (Snilstveit et al. 2016). The e-books selected for inclusion in GirlsRead were written to be of interest to adolescents and while girls could read as many e-books as they wanted independently, mentors facilitated reading and discussion of book content and themes in weekly groups following a structured curriculum. E-readers, with their capacity to hold hundreds of books, and their easily up-datable libraries, can provide a range of content for diverse reading levels and interests. Prior research has indicated that while the fixed costs of providing digital content are higher than for paper textbooks, the marginal costs are lower (Bando et al. 2017). As prices fall, e-readers or other digital devices can bridge the literacy and digital divides by inexpensively providing immediate access to thousands of informational resources. In addition, e-readers are likely to be more cost-effective than hard copy books because the expense of procuring and shipping paper books is substantial. Based on e-reader and e-book costs for the GirlsRead! project, assuming an e-reader lasts for 4 years, the cost of 100 books on each device per student per year was approximately 35 cents per book (including the cost of the device, accessories, shipping, and e-books). The World Bank has estimated, for nine sub-Saharan African countries, that in Grade 6, the median annual cost per student was 5.38 USD for four textbooks with a median book life of 3.5 years (Fredriksen, 2015). Even if we assume that textbooks are about twice the cost of non-textbooks, e.g. 67 cents/paper book, the e-books cost less. Moreover, after initial shipment, new e-books can be easily added. Further, compared to tablets or computers, e-readers are single-function devices, which can only be used for reading and not for playing games or searching the internet.

That said, gains documented in this study are a result of an intervention package, not simply the delivery of e-readers. Ongoing costs such as safe spaces mentors – teachers and women from the community – who were trained, paid for their time, and received supportive supervision, as well as the overall coordination costs of relationships with the schools, outreach to the community, monitoring implementation, and problem-solving (such as replacing chargers that were burnt out), were all necessary for the smooth functioning of the programme. While there are ways that the intervention components might be integrated into school systems at lower cost than this pilot – for example, many schools already run afterschool programmes that would not entail a separate, additional management and supervisory system apart from initial training – we are sceptical that it could be implemented cost-free, as renumerating mentors, whether teachers or community mentors, is essential. This is an important implementation question for future research, as is testing whether simply providing e-readers to students in the context of the formal school experience, or instead providing e-books via students’ own phones, can demonstrate the same improvements in literacy that GirlsRead! achieved.

This analysis of the effect of a relatively short-term 19 session pilot intervention among Zambian school girls shows that, in a setting where books are scarce, provision of e-readers with engaging content primarily written by African authors and lesson plans for mentors to read selected books with students has the capacity to improve English literacy and non-verbal reasoning. Moreover, despite the fact that girls kept their e-readers with them, taking them home, carrying them to and from school, and keeping them over school breaks, a very small number of e-readers were lost, stolen or broken by the end of the intervention.

The results of the evaluation indicate that girls in the e-reader arm scored significantly better on two of three basic literacy assessments than girls in the control arm; this was not the case for girls in the safe spaces-only arm. Girls in the e-reader arm also scored significantly better on the non-verbal reasoning assessment than girls in the control arm. While the Ravens Matrices is a non-verbal test, it is likely that exposure to books and facilitated discussion of books was sufficiently stimulating to improve girls’ reasoning ability. Although the e-reader arm coefficients were all positive for the more advanced multiple-choice exam they were not significant, which was not unexpected given that this tested familiarity with grammar and vocabulary beyond what was covered in the e-reader sessions. In addition, no effects of e-reader exposure were observed for numeracy, which tested basic arithmetic skills. While word problems were included in the UWEZO maths assessment, unlike the literacy assessment, girls had the option of taking this component in a local language. (The multiple-choice test for numeracy was only administered in English but included very few words).

One question is whether increasing the number of sessions or expanding the amount of time spent reading during each safe space session would have led to a larger effect on literacy in the e-reader arm than we observed. As noted, the intervention included only 19 two-hour sessions over a six-month period with half the session devoted to the empowerment curriculum and half to literacy activities. In addition, if literacy were the only goal, it could be that intervening in an earlier grade might ultimately make a bigger difference in educational outcomes. Early reading skills are fundamental to understanding more advanced concepts as Chall (1983) made clear in her widely cited model of reading stages from initial reading and decoding to greater fluency and then reading for learning. Without a solid foundation early on, children may end up essentially excluded from the schooling experience. Another benefit to intervening at an earlier grade is that there would be less competition with test prep for the primary school-leaving exam as well as other activities in which adolescents engage.

An inability to read is undoubtedly both a cause and a consequence of school dropout. The implications of low levels of educational attainment and elevated levels of illiteracy for the health and well-being of young women and their offspring are profound. Researchers have theorised about the ways in which literacy contributes to better health among women via, for example, the provision of targeted knowledge and the tools to understand health information and engage with health bureaucracies (Rowe et al. 2005; LeVine et al. 2012; Basu and Stephenson 2005; Glewwe 1999; Jejeebhoy 1995). While few studies have investigated the effects of improvements in literacy on health, a recent systematic review of the literature assessing the evidence for a causal effect of education in low-income countries found that increased grade attainment for women has led to lower fertility, lower HIV prevalence, and lower child mortality in a number of settings (Psaki et al. 2019a; Mensch et al. 2019b). The findings reported here summarising the results of an impact evaluation of a 19-session intervention demonstrate that access to e-readers with engaging books along with facilitated book groups have the potential to enhance basic literacy and non-verbal reasoning ability in the short term with possible longer-term consequences for the health and wellbeing of young women in a low-income setting.

Highlights

• Girls in the e-reader arm scored significantly better than girls in the control arm on two of three basic literacy assessments.

• While the coefficients were all positive for the more advanced literacy assessment in the e-reader arm, they were not significant.

• No effects of e-reader exposure on numeracy were observed.

• Girls in the e-reader arm scored significantly better than girls in the control arm on the non-verbal reasoning assessment.

• Exposure to a facilitated book group and access to an e-reader can enhance basic literacy skills and reasoning ability in a low-income setting.

Acknowledgments

GirlsRead! Zambia was funded in part by a grant from the United States Department of State as part of the DREAMS Innovation Challenge, managed by JSI Research & Training Institute, Inc. (JSI). The opinions, findings, and conclusions stated herein are those of the authors and do not necessarily reflect those of the United States Department of State or JSI. An earlier version of this paper was presented at the Research on Improving Systems of Education (RISE) Annual Conference, Center for Global Development, Washington DC, 20 June 2019.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This project was funded by a grant from the United States Department of State as part of the DREAMS Innovation Challenge, managed by JSI Research & Training Institute, Inc. (JSI). The opinions, findings, and conclusions stated herein are those of the authors and do not necessarily reflect those of the United States Department of State or JSI.

Notes on contributors

Barbara S. Mensch

Barbara Mensch, who retired from the Population Council in 2020, has conducted research on a broad array of public health, demographic and development issues in low income countries including analyses of:  1) the linkages between education and health; 2) adolescent schooling experiences and the effect on educational attainment;  3) the reliability of self-reports on sexual behavior in demographic surveys and clinical studies; 4) behavioral issues in HIV prevention trials focusing on product acceptability and adherence; and 5) the quality of family planning services and the effect on contraceptive use. She has an MA in social and political sciences from Cambridge University and a PhD in sociology and demography from Princeton University.

Nicole Haberland

Nicole Haberland is senior associate in the Population Council’s Poverty, Gender, and Youth Programme. She conducts research on a range of topics related to gender, education, HIV prevention, and sexual and reproductive health. Haberland leads the Evidence for Gender and Education Resource (EGER) project, and the Council’s work on empowerment-based sexuality education. She co-edited It’s All One Curriculum, a user-friendly resource which provides a rationale, content, and sample activities for placing gender and rights at the center of life skills, sexuality, and HIV education curricula; and is co-author of the recently released Girls’ Education Roadmap, which synthesizes current data on gender and education needs, practice, and evidence.  She has an MPH from Columbia University.

Erica Soler-Hampejsek

Erica Soler-Hampejsek, formerly a Bixby Fellow and associate at the Population Council, is an independent consultant based in Barcelona, Spain. She specializes in quantitative analysis and has conducted research and impact evaluations in diverse low- and middle-income countries. Her research interests include the interrelations between education and health among adolescents and young people. She holds a PhD in demography from the University of Pennsylvania.

Jean Digitale

Jean Digitale is a PhD candidate in Epidemiology and Translational Science at the University of California, San Francisco. Her current research focuses on using electronic health record data to improve the quality of care of pediatric patients in hospital settings, building on her background in clinical care as a pediatric nurse. She worked at Population Council from 2014-2018, where she managed fieldwork and data collection for randomized controlled trials of interventions to improve the health and literacy of adolescent girls in Zambia. She has a Master of Public Health from Columbia University and a Bachelor of Science in Nursing from the University of Pennsylvania.

Natalie Jackson-Hachonda

Natalie Jackson Hachonda has 19 years of experience working in international development in sub-Saharan Africa, designing, testing, and implementing innovative public health and education programs. For the last 12 years, Natalie has managed the day-to-day operations of adolescent girls’ and women’s empowerment programs in Zambia including GirlsRead! and AGEP at Population Council. Currently, Natalie is working with the World Bank in Zambia to integrate empowerment programming into the Keeping Girls in School (KGS) Initiative and to increase awareness, prevention, and response to GBV in schools and communities. Natalie has an MPH in Executive Public Health Policy and Administration from the University of Minnesota-Twin Cities.

Nachela Chelwa

Nachela Chelwa is a public health advocate with over 10 years’ experience in research design and management, monitoring, and evaluation of multi-year health and development programs. She works for the Population Council as Manager for Monitoring, Evaluation and Implementation Research providing Technical Assistance for Sexual and Gender-based Violence programmes in refugee settings in Zambia as well as managing a randomized control trial for HIV prevention care and treatment programmes for Adolescents in Zambia which seeks to test an integrated wellness clinic layered with a community de-stigmatization model for adolescent girls. Prior to joining the Council, she worked with Palladium (formerly Futures Group) as MonitoringandEvaluationTechnicalAdvisorforamulti-millionUSAID-fundedOVCprogramme (2012-2015) in Zambia.  Nachela holds an MPH specializing in Population Studies from the University of Zambia. She is also a 2011 MILEAD fellow.

Pamela Nyirenda

Pamela Nyirenda who currently works for Akros Research as Programme Manager has implemented sexual and reproductive health programmes aimed at improving access to services, implemented an adolescent empowerment program aimed at building social, economic, and health assets to help girls avoid early marriage, sexually transmitted infections,  including HIV/AIDS and unintended pregnancy, also implemented an education and empowerment programme aimed at keeping girls in school by developing their social assets and confidence to progress to secondary school and prevent HIV/AIDS. She has an MA in Community Development from the Hebrew University of Jerusalem.

Erica Chuang

Erica Chuang is a PhD candidate in Economics at the University of California, San Diego. She has co-authored several studies on topics covering development, survey methods, education and the environment, most notably on consistency tests for indirect response methods, the interactions between urban settings and the environment, and a systematic review of the causal links between education and health. She holds a Masters in Pacific International Affairs from the School of Global Policy and Strategy at UCSD.

Lisa Polen

Lisa Polen is the project manager for the Indigenous Girls Empowerment Network at the Population Council. She has over 15 years of experience developing, implementing, and evaluating adolescent, maternal and reproductive health programming both internationally and domestically. She received an MPH from Columbia University.

Stephanie R. Psaki

Barbara Mensch, who retired from the Population Council in 2020, has conducted research on a broad array of public health, demographic and development issues in low income countries including analyses of:  1) the linkages between education and health; 2) adolescent schooling experiences and the effect on educational attainment;  3) the reliability of self-reports on sexual behavior in demographic surveys and clinical studies; 4) behavioral issues in HIV prevention trials focusing on product acceptability and adherence; and 5) the quality of family planning services and the effect on contraceptive use. She has an MA in social and political sciences from Cambridge University and a PhD in sociology and demography from Princeton University.

Stephanie Psaki is a social demographer with expertise in the intersections of gender, health, and education, and she has led numerous studies on these topics. Her research focuses on what works to improve the lives of girls and women globally, with an emphasis on translating evidence for policy and programme audiences. She is currently Senior Advisor on Human Rights and Gender Equity in the Office of Global Affairs at the U.S. Department of Health and Human Services. Previously, Dr. Psaki was Director of the Population Council's Girl Innovation Research and Learning Center, which provides guidance to governments, international organizations, and philanthropies on investing resources to expand opportunities for women and girls globally. She holds a PhD in Public Health from the Johns Hopkins Bloomberg School of Public Health, and an MS in Population and International Health from the Harvard T.H. Chan School of Public Health.

Nkomba Kayeyi

Nicole Haberland is senior associate in the Population Council’s Poverty, Gender, and Youth Programme. She conducts research on a range of topics related to gender, education, HIV prevention, and sexual and reproductive health. Haberland leads the Evidence for Gender and Education Resource (EGER) project, and the Council’s work on empowerment-based sexuality education. She co-edited It’s All One Curriculum, a user-friendly resource which provides a rationale, content, and sample activities for placing gender and rights at the center of life skills, sexuality, and HIV education curricula; and is co-author of the recently released Girls’ Education Roadmap, which synthesizes current data on gender and education needs, practice, and evidence.  She has an MPH from Columbia University.

Nkomba Kayeyi currently holds a position of Senior Scientific Manager at Population Council – Zambia. He has conducted research and analysis in public health and demographic issues in Zambia. The broad topics that he has focused on include factors affecting HIV/AIDS infection, risky sexual behaviors, orphans and vulnerable children and HIV/AIDS, adolescent’s health and HIV/AIDS, and prevention of maternal deaths from unwanted pregnancies. Nkomba has an MPhil in Health Promotion (International Health Track) and a PhD in Epidemiology and International Health from the University of Bergen.

Michael T. Mbizvo

Mike Mbizvo is a Senior Associate, Senior Country Adviser and Country Director for Population Council, in Zambia. This entails providing research, scientific, and technical leadership in the Council’s mission areas. He has constant dialogue with Government Ministries, Country Missions, and multiple research and training stakeholders in the country, region and globally, to facilitate and support policy and programmes towards advancing sexual and reproductive health and rights (SRHR), and HIV prevention and care, through generation and synthesis of research evidence. Previously he was Director for Reproductive Health and Research, including the co-sponsored Human Reproduction Program (HRP) at WHO/HQ; and Professor at University of Zimbabwe. He holds a DPhil in human reproduction and an MPhil in medical sciences from the University of Zimbabwe and an MSc in reproductive sciences from Monash University, Australia. He has published peer reviewed publications widely in the areas of SRHR, HIV, adolescence, and male reproductive health, and authored two scientific books

Notes

1. In the 2013–14 Zambian DHS the literacy assessment – ability to read all or part of a sentence – was administered only to those who had not attended secondary school as it was assumed that those who had gone beyond primary were able to read (Psaki, McCarthy, and Mensch 2018).

2. http://www.sacmeq.org/?q=sacmeq-projects/sacmeq-iii/readingmathscores.

3. www.pepfar.gov/partnerships/ppp/dreams.

4. It could be, however, that declines in literacy over time among those who completed primary school but did not continue to secondary are due to changes in the composition of those completing primary only rather than to deterioration in school quality (Psaki, McCarthy, and Mensch 2018).

5. Worldreader conducted its own randomised evaluation of its iREAD2 intervention among students in primary grades 1–3 in Ghana, which, in contrast to GirlsRead!, did not permit the e-readers to be taken home. Significant effects were observed in the treatment arm compared to the control in 1) both local language and English reading comprehension, 2) local language literacy in lower performing students, and 3) number of books read per day (Jaffe, Lowe and Mahesri 2014).

6. http://palnetwork.org/what-we-do/.

7. We used the same subset of the Raven’s Coloured Progressive Matrices as was used in the Mexican Family Life Survey (Rubalcava and Teruel 2004) .

8. As a robustness check we also estimated difference in differences (DID) models for the UWEZO literacy outcomes; although the direction of effects is generally the same as in post-difference ANCOVA models, we did not have enough power to detect differences between arms.

9. While raw scores from the complete set of Raven’s Coloured Progressive Matrices can be converted to a normed percentile and an IQ score, comparisons between the GirlsRead! sample and other populations was not a component of this study.

10. If the UWEZO literacy was equal to 0, the girl was assigned a multiple choice score of 8; if UWEZO equalled 1, the assigned score was 9 and if the UWEZO was equal to 3, the assigned score was 10.

11. For numeracy, the medians were used rather than the means because the means were inconsistent as many fewer girls had low scores by comparison to the UWEZO literacy. If the UWEZO was ≤3, the assigned multiple choice score was 1; if the UWEZO equalled 4, the assigned score was 2. Given that some girls who scored 5 in the UWEZO did not take the multiple choice test, If the UWEZO was equal to 5 and the girl was missing the multiple choice, she was also assigned a score of 2 on the multiple choice.

12. For all models, even those with dichotomous outcomes, we estimated linear models. Given that the probabilities we are modelling are not close to either 0 or 1 – they are between .35 and .69 – linear and logistic models are said to fit about equally well https://statisticalhorizons.com/linear-vs-logistic.

13. Seventeen girls in Arm 1 who were not interviewed at both baseline and endline received e-readers.

14. While textbooks alone have not been observed to improve learning outcomes, if the provision of learning materials is part of a structured pedagogy programme with a tailored curricula such interventions have been found effective (Snilstveit et al. 2016)

Appendix

Table A1. GirlsRead! sample flow chart

Enrolment
		Public Schools: 353 schools
		- Lusaka: 268 schools
		- Ndola: 41 schools
		- Chingola: 44 schools
		Purposively selected^a: 45 schools
		- Lusaka^b: 22 schools
		- Ndola: 14 schools
		- Chingola: 9 schools
		Purposively selected^c: 36 schools
		- Lusaka: 15 schools
		- Ndola: 12 schools
		- Chingola: 9 schools
		Rosters of incoming Grade 7 students
		- Lusaka
		15 schools with 50+ girls (38 girls randomly selected)
		0 schools with fewer than 50 girls (all girls selected)
		- Ndola
		10 schools with 50+ girls (38 girls randomly selected)
		2 schools with fewer than 50 girls (all girls selected)
		- Chingola
		3 schools with 50+ girls (38 girls randomly selected)
		6 schools with fewer than 50 girls (all girls selected)^d
		Selected girls: 1364 girls
		- Lusaka: 570 girls
		- Ndola: 452 girls
		- Chingola: 342 girls
Randomised allocation at school level		Arm 1 safe spaces + e-reader (12 schools)	Arm 2 safe spaces^b (12 schools)	Arm 3 control (12 schools)	Total (36 schools)
Baseline survey (January-March 2017)
	Target sample: N	439	483	442	1364
	Non-response: %	4.8%	4.8%	4.8%	4.8%
	Completed: N	418	460	421	1299
	Mean number of girls per school	34.8	38.3	35.1	36.1
	[min,max]	[23,41]	[33,56]	[20,43]	[20,56]
Post-intervention survey (October-November 2017)
	Target sample^b N	418	460	421	1299
	Loss to follow-up: %	3.3%	3.9%	2.4%	3.2%
	Reasons for loss to follow-up
	Refusal: N	6	5	2	13
	Deceased: N	0	1	0	1
	Not located/moved: N	4	7	5	16
	Other/no information: N	4	5	3	12
	Completed^f N	404	442	411	1257
	Mean number of girls per school	33.7	36.8	34.3	34.9
	[min,max]	[22,39]	[30,54]	[19,42]	[19,54]
Endline survey (March–May 2018)
	Target sample^g N	418	459	421	1298
	Loss to follow-up: %	10.5%	7.0%	10.0%	9.1%
	Reasons for loss to follow-up
	Refusal: N	4	4	2	10
	Incapable physically or mentally: N	0	0	1	1
	Not located/moved: N	31	20	31	82
	Other/no information: N	9	8	8	25
	Completed^h N	374	425	379	1178
	Mean number of girls per school	31.2	35.4	31.6	32.7
	[min,max]	[16,38]	[28,51]	[17,37]	[16,51]

^aWe purposively selected schools with Grade 7 in Lusaka, Chingola, and Ndola, the 3 districts specified for Zambia by the DREAMS Innovation Challenge. In coordination with USAID, we aimed to minimise overlap with other DREAMS activities: we excluded DREAMS core schools and Zambia Centre for Communication Programmes (ZCCP) schools in Ndola and Chingola (in which Girls Leading Our World (GLOW) camps were being conducted).

^bWe attempted to get data on 23 schools in Lusaka, but were unable to get data on one, resulting in 22 schools in the sampling frame.

^cWe then visited each school and gathered data on the age of Grade 7 girls. Ndola and Chingola schools listed the ages of girls from the school census (conducted mid-year in April). Lusaka schools listed the ages of girls from the school census or exam entry lists (Grade 7 exams are conducted in October). For each district, we ranked schools by the proportion of girls aged 15 and older. We then selected the 15 schools with the highest proportion of such girls in Lusaka and the 12 schools with the highest proportion of such girls in Ndola.

^dPer roster provided prior to fieldwork, one school had fewer than 50 girls so we planned to invite all of them to participate. When the field team arrived they discovered there were 56 girls, all of whom were thus invited to participate.

^eTarget sample for immediate post-intervention survey was all girls interviewed at baseline.

^f55 girls not interviewed at baseline were also interviewed at the post-intervention survey (e.g. additional programme participants), but were not included in this analysis.

^gTarget sample for endline was all alive girls interviewed at baseline.

^hData from 2 girls in arm 2 who were interviewed at baseline were not included in the analysis. 48 girls not interviewed at baseline were also interviewed at endline (e.g. additional programme participants), but were not included in this analysis.

Table A2. Baseline means for the full baseline sample, by arm

Baseline characteristic/assessment	Total (N = 1299)	Arm 1 safe spaces + e-reader (N = 418)	Arm 2 safe spaces (N = 460)	Arm 3 control (N = 421)	P-value joint test
Age at last birthday	12.9 (SD 1.3)	12.8 (SD 1.3)	12.9 (SD 1.3)	12.9 (SD 1.3)	0.923
English spoken at home	25%	27%	23%	24%	0.830
Ever repeated a grade	35%	34%	35%	37%	0.799
Count of household items (0–15)	8.0 (SD 2.8)	8.5 (SD 2.5)	7.9 (SD 2.9)	7.7 (SD 3.0)	0.306
Mother’s education < junior secondary^a	36%	34%	36%	37%	0.914
Mother’s education: don’t know	19%	19%	16%	21%	0.362
Father’s education < junior secondary^a	14%	15%	14%	14%	0.980
Father’s education: don’t know	29%	28%	27%	31%	0.554
Mother not alive	10%	9%	11%	10%	0.261
Mother not coresident	26%	28%	22%	30%	0.131
Father not alive	22%	21%	18%	29%	0.001
Father not coresident	30%	32%	30%	28%	0.407
No books at household	31%	27%	34%	34%	0.365
1–5 books at household	34%	35%	34%	34%	0.950
6+ books at household
Ever used a cell phone	51%	54%	48%	51%	0.801
Passed UWEZO paragraph^b	53%	59%	47%	53%	0.343
Passed UWEZO story^c	44%	48%	39%	45%	0.519
Passed all UWEZO literacy (scored 6)	38%	41%	34%	40%	0.629
UWEZO literacy score (0–6)	3.4 (SD 2.2)	3.7 (SD 2.2)	3.2 (SD 2.2)	3.5 (SD 2.2)	0.472
Literacy multiple choice score^d (0–32)	13.7 (SD 6.6)	13.8 (SD 6.3)	13.1 (SD 6.5)	14.1 (SD 7.0)	0.777
Passed all UWEZO numeracy (scored 7)	63%	69%^†	60%	61%	0.084
UWEZO numeracy score (0–7)	6.2 (SD 1.4)	6.3 (SD 1.3)	6.0 (SD 1.5)	6.2 (SD 1.3)	0.293
Numeracy multiple choice score^d (0–23)	8.5 (SD 4.7)	8.9 (SD 4.6)	7.9 (SD 4.7)	8.7 (SD 4.7)	0.436
Non-verbal reasoning score^e (0–16)	8.8 (SD 3.3)	9.1 (SD 3.2)	8.5 (SD 3.4)	8.9 (SD 3.2)	0.656
Student-teacher ratio	33 (SD 17)	29 (SD 14)	35 (SD 15)	35 (SD 21)	0.489
District: Lusaka
District: Ndola	34%	35%	32%	35%	0.982
District: Chingola	26%	23%	30%	23%	0.913

P-values from F tests for differences across arms accounting for clustering at the school level to test for baseline balance.

^†p < .1 difference Arm 1 vs Arm 3

^a<Junior secondary is equivalent to ≤ 8 years of schooling

^bPassing the paragraph indicates the respondent could read letters, read simple words and read a paragraph out loud containing 3 simple sentences and 20–22 words and make 2 or fewer mistakes.

^cPassing the story indicates the respondent could do all components listed in ‘a’ plus could read a two paragraph 12 sentence story out loud and make 4 or fewer mistakes.

^dScores were imputed for girls who did not pass the UWEZO

^eNon-verbal reasoning score is the total number of correct answers to 16 items from Raven’s Coloured Progressive Matrices.