The Cost-Effectiveness of an Accelerated Learning Program on the Literacy, Numeracy and Social-Emotional Learning Outcomes of Out-of-School Children in Northeast Nigeria: Evidence from a Mixed Methods Randomized Controlled Trial

Abstract

Nigeria is home to more out-of-school (OOS) children than any other country on Earth. More than 10.2 million children match that description throughout Nigeria, a conflict-affected country that grapples with long-standing challenges of entrenched poverty and weak governance. We used a randomized control trial (RCT) with 1,723 out of school children, ages 9–14 (850 treatment and 873 control), to determine the impact of an accelerated learning program (ALP) on the literacy, numeracy and social-emotional learning (SEL) outcomes of OOS children in Northeast Nigeria and gathered qualitative and costing data to determine stakeholder’s experiences with the program and the cost of implementation. Results show that at an average cost of £66 (2018 GBP) per child for 7 months of service, the program had positive, small to medium, and statistically significant effects on children’s literacy and numeracy skills, but no effects on SEL outcomes. We include subgroup analysis to identify baseline equity gaps and the differential impact of the intervention by gender, displacement status and mother tongue language. We discuss limitations and policy and practices implications of these findings.

Keywords:

• Experimental design; Accelerated Learning Program; cost-effectiveness; academic outcomes; social emotional learning

Introduction

Despite global commitments to ensuring universal access to education made over two decades ago, more than 258 million children and youth are still not in school (UNESCO, 2019). Challenges around accessing education are often more acute in crisis and conflict settings. As of 2018, nearly 31 million children had been displaced from their homes due to violence and conflict (UNICEF, 2020), and an estimated 24 million of them are out of school (Global Partnership for Education, 2019). The COVID-19 pandemic further complicated the difficulties of ensuring children in fragile and conflict-affected settings can access quality education. These trends underscore the need to identify cost-effective solutions to provide access to quality education opportunities for children in emergency settings. In this paper, we present the results of a mixed methods randomized control trial that we conducted in northeast Nigeria the to examine (a) the experiences of out of school children with home, school and the community, and their experiences with a nine-month ALP, (b) the effects of the ALP on OOS children’ literacy, numeracy and social-emotional skill development, (b) variations in the program’s effects among subgroups of children defined by gender, displacement status, and mother-tongue language (MTL), and (c) the program’s cost. We hope the evidence will assist policymakers in Nigeria and other conflict affected countries in supporting OOS children’s education through impactful alternative learning opportunities when the formal education system is hard to reach.

Background

The Context of Northeast Nigeria

The Federal Republic of Nigeria gained independence in 1960 following six decades of British colonial rule. Nigeria’s population is estimated at 202 million people in 2019, with about half living in rural areas. The country is a multi-ethnic and linguistically diverse federation, which has experienced long-standing political tensions over the massive gaps between the country’s oil wealth and deeply rooted poverty and corruption, linked with government neglect, insecurity, and violence. The northeast region of Nigeria has long suffered conflict, natural disasters, poverty, exclusion and weak state legitimacy and capacity. Since 2009, the militant group Boko Haram has attacked and terrorized the 24.5 million people living in northeast Nigeria, creating one of the worst humanitarian crises in the country’s history (FMoE, 2014). Boko Haram attacks—which have included burning of homes, destruction of villages and schools and the abduction and killing of school children—have killed more than 35,000 people and led to the displacement of 1.7 million people in Borno, Yobe and Adamawa states of Nigeria.

Continuous conflict and crisis in Nigeria have created many barriers to quality education. UNICEF (2019) estimated that there are over 10.2 million OOS children throughout Nigeria, more than any other country on Earth. In northeast states such as Borno and Yobe, Boko Haram and other non-state armed groups have deliberately and systemically targeted local education systems with devastating results (Isokpan & Durojaye, 2016) . More than 600 teachers have been killed and 19,000 displaced. About 2,500 schools have been damaged, destroyed or forced to close. An estimated 731,000 school-aged children have been displaced to refugee camps. These attacks have resulted in an estimated 900,000 children losing access to learning opportunities (OCHA, 2018). Even before Covid-related school closures, nearly 75% of children in northeast Nigeria remained OOS (WB, 2016).

Besides the violence associated with Boko Haram, poverty and other social and health issues such as parental death and child labor remain critical barriers to overall children’s education in Nigeria (FMoE, 2014; Kainuwa & Yusuf, 2013b; Kazeem & Musalia, 2018; OCHA, 2018). Specific barriers reported by research investigations vary. They include barriers at different levels from macro to micro. At a macro level, educational barriers include a lack of school facilities and infrastructure, a lack of certified teaching personnel, a lack of teaching and learning resources, a lack of recovery funds and planning in an education sector and the prevalence of child labor. Additionally, at a macro level, education barriers also include social norms that create resistance to Western education or preference for religious institutions, as well as unequal gender norms and expectations toward child education between boys and girls (Kainuwa & Yusuf, 2013b; OCHA, 2018; Uscategui & Andrea, 2017). At a meso level, common barriers include long distance to school and hazardous conditions on the way to school, for instance poor roads and kidnapping (Antoninis, 2014; Kazeem et al., 2010). When girls have access to learning, many of them are sent to non-formal, religious education institutions such as Islamiyya religious institutions where children are typically taught religious principles rather than an official government curriculum (Antoninis, 2014). Lastly, at a micro level—individuals’ behavior and interactions in everyday lives—barriers include household poverty and a consequent lack of financial capacity to pay school fees and other education-related costs such as uniforms, learning materials and unofficial levies, a lack of child interest in education, and lack of parents’ interest or support in child education, parental death and orphan-hood (Antoninis, 2014; Coinco & Morris, 2017; Kainuwa et al., 2017; Kazeem et al., 2010; Kazeem & Jensen, 2017; Kazeem & Musalia, 2018; OCHA, 2018).

Educational barriers expand to other challenges in home and school that often lead to disrupted attendance and drop out after the first few months or years of school. Frequent drop-out causes reported by research studies include long distance to school, unequal gender norms and expectation toward girls’ education and household labor, early marriage or unwanted pregnancy, resistance to secular education, and fear of security, harsh discipline, unfriendly peers and teachers at school, poor assimilation in class and school cultures, lack of money to pay for school fees, lack of child’s interest in education, poor learning gains and performance at school, lack of parental interest or support in child education, parental death and orphan-hood, and illness (Abdullahi & Terhemba, 2014; Antoninis, 2014; Duze, 2012; Kainuwa et al., 2017; Kainuwa & Yusuf, 2013a, 2013b; Kazeem et al., 2010; Isokpan & Durojaye, 2016; Olufunke & Oluwadamilola, 2014; Shehu, 2018).

Governmental and non-governmental actors have tried to address these challenges and provide education opportunities to children as part of a humanitarian response in the Northeast, but many challenges persist. The formal education system is struggling to absorb the rapid increase in the region’s refugee population due to limited capacity (Coinco & Morris, 2017). To increase access, in 2019 Nigeria’s Ministry of Education allocated funding for rehabilitating or rebuilding schools and providing learning materials and teacher support in the states most impacted by Boko Haram. As a target strategy, the Ministry has sought to form partnerships with development actors for “the provision of special means of access to schooling through the use of temporary structures where appropriate structures are not readily available” and “building the confidence of teachers and their ability to handle children who suffer from traumatic experiences of violence and war” (FMoE, 2019, p. 11). But despite their necessity and good intentions, these initiatives have been hampered by a shortage of qualified teachers, schools and learning materials to meet the great demand for education within the context.

Accelerated Learning Programs (ALPs)

Alternative education approaches such as ALPs are a widely used solution to meet the education needs of OOS children around the world (e.g., Adams et al., 2009; Charlick, 2003; Menendez et al., 2016). ALPs are flexible, age-appropriate programs that run in a short timeframe to provide unrestricted access to education opportunities for children whose schooling have been disrupted as a result of poverty, social upheaval or natural disasters (Boisvert et al., 2017). In ALPs the pace of learning is quickened to complete a curriculum faster than in formal schools to help over-age or OOS children fast-track their (re-)entry into mainstream education or obtain equivalent certified competencies as in the formal system (Boisvert et al., 2017; GCPEA, 2018). ALPs are often operationalized in non-formal learning centers (NFLCs) with small classes and learner-centered, interactive pedagogy. Classes are scheduled with flexibility for the convenience of learners who often bear responsibilities of household chores or income generating work outside home. Curriculum varies from a condensed version of the national curriculum to other context-relevant subjects such as literacy, numeracy, peace building, human rights issues, gender equality, and health education (Charlick, 2003; Menendez et al., 2016; Shah, 2015). In conflict and crisis affected settings such as northeast Nigeria, ALPs can also include SEL programming, as a strategy to promote children’s wellbeing to mitigate the negative effects that toxic stress can have on their learning and development, as a result of difficult life experiences such as exposure to poverty, violence, displacement and uncertainty (Dodge, 2006; Perkns & Graham-Bermann, 2012; Shonkoff & Garner, 2012; Zuilkowski & Betancourt, 2014)

Evidence is rich in both high-income (e.g., Adams et al., 2009; Capizzano et al., 2007) and low-income but stable countries (Akyeampong et al., 2018; Charlick, 2003; Jere, 2012) that ALPs can mitigate at-risk students’ learning loss and promote academic achievement in various subjects, including reading and math. In these contexts, ALPs have also been shown to help children transition to the formal system and continue with their education. However, despite a growing interest in ALPs, there is dearth of rigorous studies identifying the impact of ALP in conflict affected settings (Menendez et al., 2016). Existing evidence documents the achievements of ALPs primarily in terms of enrollment, completion or transition to the formal system (e.g., Longden, 2014; Manda, 2011; Sempere, 2009; Shah, 2015), but not in terms of impacts on learning outcomes such as literacy, numeracy or SEL. For instance, in 1999, the Liberian Ministry of Education introduced a nationwide ALP for overaged children. The program compressed six years of the primary school cycle into a three-year curriculum. Since its implementation, Liberia’s ALP has enrolled approximately 250,000 children. Completion rates across different cohorts ranged between 67% and 79%. The program contributed to a rapid increase in student enrollment in regular primary schools. Between 2005 and 2006, the gross primary enrollment rate without ALP learners was just about 45% while the rate reached 103% when ALP learners were counted in (Manda, 2011). In a different study, the Norwegian Refugee Council collaborated with the Uganda Ministry of Education to implement an ALP to help OOS children ages 9–14 rejoin primary education in Uganda in the year 2010. During its implementation, this ALP enrolled about 7,000 learners, 55% of which were girls. The program reported poor completion rates, as only 51.7% of learners completed at least one cycle. Of this group, only 55% transitioned into formal school (Shah, 2015). In Nigeria, an INGO implemented an ALP program for 6–17 year old OOS children in northern Nigeria, as part of the Education Crisis Response (ECR) program. A pre-post study showed that 75–90% of enrolled learners gained basic literacy skills, and 29–69% of enrolled learners mainstreamed to formal schooling after the completion of the ECR program (ECR, 2017). Crucially, neither of these studies identified the changes in students’ education outcomes that were caused by the ALPs because without a control group, researchers are unable to determine what would have happened to children in the absence of the intervention.

A growing body of research from high income and stable settings suggests that supporting social emotional skills can lead to improved academic outcomes (Durlak et al., 2011; Heckman & Rubinstein, 2001; Jones et al., 2011; Yeager et al., 2013). SEL may be especially critical in conflict and crisis-affected settings where children have been exposed to poverty, violence, displacement and uncertainty (Dodge, 2006; Perkns & Graham-Bermann, 2012; Reed et al., 2012; Shonkoff & Garner, 2012; Zuilkowski & Betancourt, 2014). This is because severe adversities can cause children to experience toxic stress, which will interfere with their academic and socioemotional development (Kim et al., 2020). However, most evidence about the impact of SEL programming comes from high income countries and stable societies, and little rigorous evidence exists in low-income countries and conflict affected settings. The few studies that exist in conflict and crisis settings show limited impact, for reasons that are yet to be determined (Aber et al, 2017; Brown et al., under review; Tubbs Dolan et al., 2021).

An ALP for OOS Children in Conflict-Affected Northeast Nigeria

With support from [a donor], [an International Nonprofit Organization] has been working to implement a nine-month comprehensive ALP for OOS children in northeast Nigeria. Since its inception in 2017, the program has created 400 Non-Formal Learning Centers (NFLC) in Yobe and Borno states, which have served 33,883 OOS children ages 9–14 years old. The project conducted a needs assessment to identify communities with a large number of out-of-school children and created NFLC where they were more needed. Children who had dropped out of school for more than two years or who had never been in school are eligible to enroll in the program. Once enrolled in the ALP, children are supported to learn the foundational literacy and numeracy skills they need to transition to the formal education system, as the SEL skills they need to thrive in life, and to mitigate the negative effects of the trauma and difficult life circumstances they have experienced.

The great majority of NFLC were hosted in established primary government schools, but the INGOs built 40 semi-permanent NFLC in two local government areas (Gwoza and Monguno) because they didn’t have enough functioning primary schools. While both types of NFLC had doors, windows, blackboards and handwashing facilities, the NFLC hosted by government primary schools had desks and the NFLC built by the INGO used floor mats.

The ALP is provided to children free of cost, in community-led NFLCs. Children come to the program 5 days per week, and spend 3 hours per day learning literacy, numeracy and SEL, with some free time to also play games. During the first cohort of the program which is the focus of this study, the ALP was conducted over a period of 7 months. In subsequent cohorts children spent 9 months learning. Each day has three 45-minute learning sessions. Literacy lessons focus on the five components of foundational reading: phonological awareness, alphabetic knowledge, vocabulary, fluency and comprehension. Numeracy lessons include counting, number sense of quantity, and operations of addition and subtraction. SEL lessons provide opportunities to practice empathy, perseverance, stress and impulse management, emotional recognition, communication and conflict resolution. The SEL component also helps children practice executive functioning skills required for learning, such as focus and memory. To support instruction, the program provided literacy, numeracy and SEL guides, which included a scope and sequence of lessons for the three subject matters. All teaching and learning materials were jointly developed by the INGO with partners and various stakeholders from the government ministries, and were validated and accredited by the host government at the state and federal levels. Unlike mainstream school, the schedule for the classes at non-formal learning centers were decided in discussion with members from the community, providing children with the opportunity to choose the most convenient time to learn.

The ALP classes are led by learning facilitators (LFs), who are literate community members trusted by the local community, with a high school degree and in some cases a teaching certificate. LFs receive a monthly stipend for their work. Each LF teaches about 45 learners in their classroom. To ensure that LFs acquire the skills they need to teach literacy, numeracy and SEL, and to build safe and caring learning environments, an INGO provides them with a six-day face-to-face training. In these trainings, LFs learn subject matter knowledge for literacy, numeracy and SEL and about lesson planning, classroom management and positive discipline strategies. These strategies include how to use classroom rules to set behavioral expectations and develop trusted relationships with learners. LFs also attend monthly teaching learning circles (TLCs), which are sustained opportunities to learn collaboratively through modules focused on inclusive education, creating local learning materials, SEL and classroom management. During TLCs, LFs discuss the challenges they are experiencing in the classroom, share experiences and best practices, working with their fellow LFs to reflect and build each other’s knowledge and practice. LFs meet monthly over the 9-month course of the ALP, with each meeting lasting 2–3 hours. Finally, LFs also receive visits from coaches, who are government employees who traditionally worked as supervisors for local ministries of education in Borno and Yobe. As part of this project, coaches attended a 2-day face to face training, focused on helping them develop coaching skills. During on-site visits, coaches observe how LFs teach and interact with learners and provide feedback informed by their classroom observations and focused on improvement strategies.

At the community level, the ALP mobilizes parent and community support for program implementation and monitoring activities. Together with project staff, community members developed early warning systems to address specific risks and concerns within communities and enhance the sense of safety and security, especially for women and girls. Community members also carried out visits to the learning centers to monitor learning facilitators’ and children’s attendance and check the center’ resource usage. Furthermore, community members conducted safety audits to ensure that children in the ALP centers, to and from the centers are safe.

Finally, the ALP project works with the government at all levels to increase support for formal and non-formal education and strengthen coordination mechanisms to achieve better management, more robust technical support, and more effective monitoring practices. The project improves coordination between the State of Universal Education Board (SUBEB), the State Agency for Mass Education (SAME), the MoE and non-governmental actors, to support more effective mainstreaming of learners from NFLCs to formal schools and respond to communities’ extensive education needs more effectively.

The Present Study

Given limited evidence about the impact of ALP programs on the education outcomes of OOS children in emergency settings, we conducted a rigorous study to build the evidence base about what works, for whom and at what cost to improve the learning and SEL outcomes of OOS children in northern Nigeria. Specifically, we ask the following research questions:

1. What are children’s experiences at home, school and the community before the ALP and with the ALP?

2. What are the effects of a nine-month ALP on OOS children’ literacy, numeracy and social-emotional skills?

3. How do the effects of the ALP vary for different subgroups of children, by gender, displacement status, and MTL?

4. What is the cost of the program?

We hope that this study will generate rigorous evidence that will assist policymakers and practitioners in northern Nigeria and beyond to support OOS children’s education through impactful learning opportunities when the formal education system is hard to reach.

Method

The present study used a mixed-methods randomized controlled trial with treatment and control groups to identify the impact of an ALP program on the literacy, numeracy and SEL skills of OOS children. Children assigned to the treatment were provided with the opportunity to attend the ALP and children in the control group were placed in a wait list so they could attend the ALP in later cohorts of programming. In addition to quantitative surveys and assessments used to evaluate the impact of the program on students’ literacy, numeracy and SEL skills, we also collected qualitative data to examine the experiences of LF and students with the ALP and examined financial data to identify the cost of the program. Table 1 presents the treatment contrast between treatment and control groups.

Table 1. Treatment contrast.

Intervention activities	Treatment	Control
Three-day face-to-face trainer of trainers training	✓	–
Four-day face-to-face facilitator training and two-day refresher	✓	–
Teaching learning circles, once a month for two hours	✓	–
One-day community coalition training and one-day refresher	✓	–
Civic society organization face-to-face training	✓	–
Learning materials—notebooks, bags, pens, erasers	✓	–
Instructional materials for learning facilitator—teaching guides and lesson plans for literacy, numeracy and SEL	✓	–
Classroom equipment—teaching aids (charts), blackboard, chalks, workbooks, paper, scissors, cups, trash bin, cleaning tools, and attendance book	✓	–
Classes with 45 children and one learning facilitator—three-hour sessions per day, three times per week for seven months (45 minutes literacy, 45 minutes numeracy, 45 minutes SEL, 45 minutes free-games)	✓	–
Stipend for facilitators	✓	–

Participants

The target population for the program are 9–14 year-old OOS children in northeast Nigeria, who had never been in school or had been OOS for more than two years. The ALP supported a total of 33,883 OOS children in 400 communities in Borno and Yobe from 2017 through 2020. For the quantitative sample of the research study, we first randomly selected 80 NFLC from the 400 NFLC supported by the INGO. In the communities where the ALP was implemented, the demand for the ALP program was greater than the program’s capacity. To deal with oversubscription, community lotteries were conducted within each of the NFLC by learning facilitators with assistance from community coalitions to randomly place OOS children eligible for the program in the treatment or in a wait control list.

At baseline, 1,723 children participated in the study, including 850 children in the treatment group and 873 children in the control group. At endline, we were unable to find 91 children (5.3% attrition), so the final sample included 1,632 children with 824 in the treatment group and 808 in the control group. This sample size was sufficient to detect a minimum effect size of 0.24 standard deviations (SD) with 80% power at the 5% significance level. Table 2 tabulates the number of children and NFLC involved in this study. The average age of participants was 11.4 years, and 53% were girls. About 72% spoke Hausa at home—the language of instruction. 72% were displaced children, while 28% were from host communities (See Table 3).

Table 2. The number of children and learning centers involved in the present study.

	Baseline (n = 1,723)		Endline (n = 1,632)
	Treatment	Control	Treatment	Control
Students	850	873	824	808
Learning centers	40	80	40	80

Table 3. Descriptive for baseline background characteristics and test of balance by group.

Variable	Sample mean (SD) (n = 1,632)	Treatment (n = 824)	Control (n = 808)	Treat-Control
Individual characteristics
Age	11.37 (1.54)	11.55	11.18	0.37**
Female	0.53 (0.50)	0.54	0.51	0.03
Disabled	0.03 (0.17)	0.03	0.03	0.00
Displacement status	0.72 (0.45)	0.73	0.71	0.02
Hausa language spoken at home	0.72 (0.45)	0.76	0.68	0.08*
Highest grade completed	0.86 (1.52)	0.98	0.73	0.25*
School repetition experience	0.03 (0.18)	0.04	0.02	0.02*
School transfer frequency	0.11 (0.40)	0.12	0.09	0.03
Family backgrounds
Living in a refugee camp	0.11 (0.31)	0.08	0.13	−0.05
Living in a rental house	0.68 (0.46)	0.72	0.65	0.07*
Low socioeconomic status	0.63 (0.48)	0.64	0.62	0.02
Low home learning environment	0.80 (0.40)	0.79	0.81	−0.02
Involved in household chore	0.85 (0.36)	0.85	0.85	0.00
Involve in labor outside home	0.12 (0.32)	0.12	0.12	0.00

*p < .05.**p < .01.

To ensure the treatment and control groups were balanced in a set of exogenous and endogenous characteristics we conducted t-tests that accounted for clustering at the community level. Exogenous characteristics are those that cannot be changed by the intervention, such as gender, age, MTL. Endogenous characteristics are those that are susceptible to change by the treatment, such as literacy, numeracy and SEL skills. This pre-intervention balance test was meant to determine whether randomization of child assignments yielded balanced samples between treatment and control groups, and whether a covariate adjustment was needed (Murnane & Willett, 2011).

Overall, we found the two groups differed in 5 of 14 exogenous characteristics, but were balanced in all baseline literacy, numeracy and SEL outcomes. The differences in exogenous characteristics appeared in average student age, home language, school repetition, and highest grade completed before enrolling in the ALP. It is worth noting that the observed differences do not consistently favor children in the treatment group. Specifically, compared to their counterparts, children in the treatment group were, on average, older by 0.4 years (about 5 months), had completed an additional quarter of a grade of education and were more likely to have repeated school, all of which gives them advantages over the control group. However, they were 8% more likely to speak Kanury at home, and their families were 7% more likely to live in a rental than to owning, which puts them at disadvantage over the treatment control group. The magnitudes of the background differences were small but statistically significant. To control for potential biases, we included covariates in regression models to estimate treatment effects. Table 3 shows the balance statistics of baseline exogenous characteristics, which will not change as result of the intervention.

Table 4 shows baseline balances for endogenous variables -which can change at endline as a result of the intervention- such as literacy, numeracy and SEL skills.

Table 4. Baseline descriptives for learning outcomes and test of balance by group.

Variable	Sample mean (SD) (n = 1,632)	Treatment (n = 824)	Control (n = 808)	Treat-Control
Literacy
Letter identification	11.67 (22.33)	11.78	11.55	0.23
Non-words reading	4.97 (12.03)	4.67	5.27	−0.60
Oral reading fluency	7.41 (17.19)	7.27	7.55	−0.28
Listening comprehension	59.95 (38.68)	60.66	59.23	1.43
Reading comprehension	7.79 (19.40)	8.42	7.15	1.27
Numeracy
Number identification	10.83 (13.83)	10.12	11.54	−1.42
Number discrimination	38.19 (34.99)	38.09	38.28	−0.19
Missing number	18.93 (23.02)	17.85	20.02	−2.17
Addition L1	6.77 (10.25)	6.61	6.94	−0.33
Subtraction L1	5.99 (9.41)	5.82	6.15	−0.33
Addition L2	18.46 (28.15)	17.99	18.94	−0.95
Subtraction L2	26.78 (35.10)	27.11	26.44	0.67
Word problems	30.32 (29.57)	29.96	30.69	−0.73
SEL
Hostile attribution bias	0.76 (0.56)	0.75	0.76	−0.01
Sadness orientation	1.02 (1.03)	1.03	1.00	0.03
Anger orientation	0.77 (1.07)	0.77	0.77	0.00
Disengagement	0.05 (0.17)	0.05	0.05	0.00
Problem-solving	0.58 (0.40)	0.59	0.58	0.01
Aggression	0.47 (0.60)	0.48	0.47	0.01
Emotional accuracy	0.70 (0.15)	0.71	0.70	0.01
Depression	3.48 (4.07)	3.60	3.36	0.24

The qualitative research sample included 48 children, 15 learning facilitators and 8 coaches, from 20 ALPs in Borno and Yobe states. The sample of children included 24 girls and 24 boys, 31 IDPs and 17 members from the host community. The qualitative sample is not representative of the population of children in Yobe and Borno but was selected in a purposeful manner to capture diverse experiences and viewpoints from different geographic locations and demographic groups of respondents.

Instruments

To assess children’s learning outcomes, we adapted several instruments to the local context, measuring children’s literacy, numeracy and SEL skills—the Early Grade Reading Assessment (EGRA), the Early Grade Mathematics Assessment (EGMA), Children’s Stories, the Assessment of Children’s Emotion Skills (ACES), and the Moods and Feelings Questionnaire (MFQ). The EGRA is a performance-based reading assessment, designed by RTI to measure foundational reading skills including letter identification, phonemic awareness (non-words), oral reading fluency, and listening and reading comprehension (Dubeck & Gove, 2015). Cronbach’s alphas averaged 0.79, and ranged from 0.75 (listening comprehension) to 0.91 (reading comprehension), reflecting good levels of reliability. The EGMA is a performance-based math assessment designed by RTI, that includes measures of number identification, quantity discrimination, missing number, addition at level one (L1), subtraction at level one (L1), addition at level two (L2), subtraction at level two (L2), and word problems (Gove et al., 2013). Cronbach’s alpha averaged 0.77 and ranged from 0.71 (word problems) to 0.81 (addition L2), reflecting good levels of reliability.

Children’s Stories (Dodge et al., 2015) is a scenario-based assessment, designed to measure children’s social behaviors and emotional orientation. It includes measures of hostile attribution bias, emotional orientation and conflict solution strategy. Six scenarios (e.g., “Imagine that you are standing on the playground, playing catch with friends. You throw the ball to another child and the child catches it. Then s/he has thrown the ball and it hits you hard in the back), are followed by questions such as “Why did the child throw the ball to you?”; “How would you feel?” and “What would you do next to solve the problem?.” Answers to the first question are coded for hostile vs. non-hostile intent, answers to the second question are coded for orientation toward calm, sadness and anger. Answers to the third question are coded for problem solving, disengagement and aggression. The reliabilities averaged 0.78, and ranged from 0.67 (hostile attribution bias) to 0.85 (disengagement). The ACES (Schultz et al., 2004) is another scenario-based assessment, and measures children’ ability to accurately recognize others’ emotions (e.g., “Sara took care of her kitten, which she loved very much. One day the kitten disappeared and never came back. What do you think Sarah would feel?”). The reliabilities averaged 0.65 across assessment times. The MFQ (Angold et al., 1995) is a screening tool for depression in children and adolescents aged 6–19, and consists of a series of descriptive phrases regarding how the child has been feeling or acting recently (e.g., “In the past two weeks, I felt like talking less than usual.”). The reliabilities averaged 0.90 across assessment times. All assessments were administered in Hausa and responses were recorded in tablets. Table 5 presents the detailed descriptions of the instruments used in our study.

Table 5. Instrument descriptions for literacy, numeracy and SEL (Reliability is estimated for untimed tasks only).

Instrument	Description (un/timed, score range)	Num. of items	Cronbach alpha	Measures
Literacy
Letter identification	Reading letters and letter names (timed)	100	N/A	Correct letters per minute
Non-words reading	Decoding nonsense words (timed)	50	N/A	Correct words per minute
Oral reading fluency	Reading familiar words (timed)	55	N/A	Correct words per minute
Listening comprehension	Recalling facts or infer information after listening (untimed)	5	0.77	Percentage correct
Reading comprehension	Recalling facts or infer information after reading (untimed)	5	0.80	Percentage correct
Numeracy
Number identification	Identifying numbers (timed)	20	N/A	Correct answers per minute
Number discrimination	Identifying which of two numbers is larger (untimed)	10	0.77	Percentage correct
Missing number	Identifying the missing number from a pattern (untimed)	10	0.78	Percentage correct
Addition L1	Adding two single-digit numbers (timed)	20	N/A	Correct answers per minute
Subtraction L1	Subtracting two single-digit numbers (timed)	20	N/A	Correct answers per minute
Addition L2	Adding two double-digit numbers (untimed)	5	0.79	Percentage correct
Subtraction L2	Subtracting two double-digit numbers (untimed)	5	0.79	Percentage correct
Word problems	Operating numbers to solve problems (untimed)	6	0.78	Percentage correct
SEL
Hostile attribution bias	Interpreting others’ behavior as hostile, when it is vague (1/0)	5	0.69	Avg. of items reflecting bias
Sadness orientation	The intensity of sadness expressed in a social situation (0–4)	5	0.75	Avg. of items reflecting sadness
Anger orientation	The intensity of anger expressed in a social situation (0–4)	5	0.77	Avg. of items reflecting anger
Disengagement	Being apt to ignore or avoid in a conflict situation (1/0)	5	0.84	Avg. of items reflecting disengagement
Problem-solving	Being apt to find a solution in a conflict situation (1/0)	5	0.84	Avg. of items reflecting problem solving
Aggression	Being apt to react with aggression in a conflict situation (1/0)	5	0.78	Avg. of items reflecting aggression
Emotional accuracy	Recognizing other people’s emotions accurately (1/0)	12	0.65	Avg. of accurate responses
Depression	Recent depressive symptoms experienced (0–2)	12	0.89	Avg. of items reflecting depression

Additionally, we used protocols to conduct semi-structured interviews with children and learning facilitators. All protocols, which were developed in collaboration with local researchers from an NGO, included questions for key thematic areas (e.g. background, life before the ALP, perception and experience with the ALP, life after the ALP, and future aspirations) across respondents to allow for comparison, quantification and triangulation of findings.

Procedures

Quantitative student assessments were completed by a team of trained local enumerators who had worked with the INGO in various education projects and had ample experience in child assessments. All enumerators received five days of training before baseline data collection commenced. In the training, enumerators learned about the logistics of the project, the ethical principles they should follow, and reviewed and practiced how to administer the assessment tools utilized in our study as well as strategies for tablet-based data collection. Baseline data was collected in May 2018 and endline data was collected at the end of November 2018. The numerators attended a five-day refresher training prior to the field work.

Qualitative interviews were conducted in local languages (Hausa and Kanuri) by local data collectors who are adolescent girls aged 16–24 that are trained to conduct interview research with mobile devices in their own communities. Using this approach unlocks open and honest conversations that might otherwise be lost or not included when collecting data in traditional ways. All audio and video data were translated and transcribed before local researchers began the process of analyzing the data. On the whole, both children and adult respondents were open and forthcoming with answers. The local data collectors built a good rapport with participants, and provided feedback to the researchers about the data collection to generate additional insights based on their experience of conducting the interviews. Local researchers from northeast Nigeria, who brought a rich contextual understanding of the data, analyzed all the interviews.

The study received support from a local research steering committee, that included representatives from the MoE, the chairman of SUBEB, the executive secretaries of SAME and local researchers from northeast Nigeria and donors in the region, who provided their input from the beginning and throughout the research process, helping the research team refine research questions, interpret findings and their implications for policymakers and practitioners in the region. The study was reviewed and received IRB approval by INGO’s Research Ethics Committee and the National Health Research Ethics Committee of Nigeria (NHREC).

Data Analysis

To answer RQ1 we analyzed the qualitative data by reviewing all audio content by respondent to understand their individual stories and exporting the data to CSV for deeper analysis via Excel and cross-reference of qualitative findings with closed question/quantitative answers. The qualitative data was analyzed using thematic analysis, through an etic and emic approach where all data was coded using preexisting categories of analysis and also allowing new categories to emerge from the data. Researchers wrote memos throughout the process by recording reflective notes about the data.

To answer RQ2 and estimate the impact of the ALP on students’ learning outcomes, we estimated the average treatment effects of providing the ALP lessons using a difference-in-differences (DID) model. We chose DID to account for existing baseline differences that we encountered between the treatment and control groups after conducting the RCT. The DID estimator compares changes in a program’s outcomes at two assessment points for treatment and control groups after accounting for the change observed in the control group over time. This sequenced comparison helps separate treatment effect from a counterfactual change that might have occurred in the absence of treatment (Murnane & Willett, 2011). The DID model fitted in our analysis is: (1) $$y_{\mathit{ijt}}={{\beta }_{1}ALP}_i+{\beta }_2{\mathit{Post}}_t+{\beta }_3{\mathit{Post}}_t.{\mathit{ALP}}_i+X_{it}\delta +u_j+v_{\mathit{ijt}}$$(1) where $y_{\mathit{ijt}}$ denotes the EGRA, EGMA, or SEL subtask score of student $i,$ in school $j,$ in time period $t.$ $\mathit{Pos}{t}_t$ indicates the post-treatment period, and takes on a value of one if student $i$ is observed in post-treatment or endline and zero otherwise. ${\mathit{ALP}}_i$ denotes treatment assignment, and takes on a value of one if student $i$ is assigned to the treatment group, and zero if assigned to the control group. $X$ is a vector of student covariates presented in Table 3, such as age, gender, physical disability, displacement status, MTL, HLE, and socioeconomic status (SES).¹ The model also includes $u_j,$ denoting school-specific random effects, to account for student clustering within schools. Lastly, $v_{\mathit{ijt}}$ is the individual error term. The parameter of interest is ${\beta }_3,$ and it shows the average effect of ALP lessons by comparing pre-post changes in the outcome variables for the treatment group relative to the control group.

To answer RQ3, we estimated differential treatment effects for subgroups of children by gender (male and female), displacement status (displaced and host community member) and MTL (Hausa and Kanuri). Differential treatments effects for each subgroup were estimated by interacting each subgroup category with treatment status on student learning outcomes. The DID model we fitted was: (2) $$y_{\mathit{ijt}}={{\beta }_{1}ALP}_i+{\beta }_2{\mathit{Post}}_t+\mathrm{ }{\beta }_{3}S\mathit{ubgrou}{p}_i+{\beta }_4{\mathit{Post}}_{\mathrm{t}}.{\mathit{ALP}}_i+\mathrm{ }{\beta }_5{\mathit{Post}}_t.{\mathit{ALP}}_i.{\mathit{Subgroup}}_i+X_{it}\delta +u_j+v_{\mathit{ijt}}$$(2) where $\mathit{Subgrou}{p}_i$ is a dummy variable, that takes on a value of one if student $i$ is female, displaced or a Kanuri speaker, and zero otherwise. The parameter of interest is ${\beta }_5,$ and it shows if treatment effects differ for different subgroups of children.

Lastly, to answer RQ4, we estimated the cost of program activities from October 1, 2017 through September 30, 2018 (12 months). Costs were estimated by center and by child, using an ingredients-based cost-effectiveness methodology that followed these steps: (1) estimate the total cost of each core program activity, pair time and effort allocations to financial spending records for all inputs; (2) pair activity cost estimates with enrollment data to determine cost-efficiency estimates per student and NFLC, and (3) pair cost-efficiency estimates with program effect analysis.

Findings

What Are Children’s Experiences at Home, the Community and School, before the ALP and with the ALP?

Children’s Family Background

Children come from a mix of different types of households, including nuclear and extended family settings. Some children have lost one or both parents and live with guardians. There are also orphans who are left to fend for themselves. Most children were engaged in supporting their parent or guardian’s livelihood.

“We live with our mother in our house, our father is seriously sick. We are the only ones in the house. It’s one of our elder brothers that is taking our responsibility” (Boy, 12 years old, IDP, Borno)

“I live with my mum and dad, and with my two stepmothers. We are sixteen in number with my grandparents, and three other family members. We are nineteen in all” (Girl, 10 years old, returnee, Yobe)

Boys often worked in the farms with their father, while girls were mostly engaged in helping with household chores, hawking and knitting caps. The primary livelihood opportunities available for both IDP and host community households across locations included subsistence farming, petty trading, selling water, cobblers, porters, knitting caps, laundry services, domestic work and security guards. Many households face acute food insecurity and for this reason, some children are heavily involved in coping strategies such as hawking and begging.

“I used to hawk soup ingredients. Things like onions, onions leaves, Baobab-leaves-powder, after that I go to the market to buy what we will eat.” (Girl, 11 years old, Host community, Yobe)

“If it is the day my mom cooks I wash dishes. If it is not her day to cook or we are fasting, I go out to hawk for her” (Girl, 10 years old, IDP, Yobe)

Children’s Community Background

Displaced children recounted experiences of life in their former communities before displacement as unstable and full of violence and fear, which led them to make the decision to move away from their villages in search of safety. Most displaced families left after an attack by Boko Haram. In some cases, families had been displaced multiple times before settling in their current locations. Many of the IDP children described experiences of instability and violence at a very young age and struggle with that trauma.

“When Boko Haram attacked the place, my father decided that we couldn’t live there anymore. So he sought a transfer of location for his job and we moved there. We had only been there for a little more than a week… we were playing football and then Boko Haram attacked us. They nearly killed us. They were trying to stop us and we started running. One of my friends broke his leg” (Boy, 14 years old, IDP Yobe)

Children reported wanting normal lives devoid of violence. They enjoy supporting household economic activities, playing with friends and attending school. They also enjoy being treated kindly and indicated that they want to forget the bad experiences some have had as a result of violence and instability. IDP children who reside in the communities are settling into normal lives and identify the peace they experience as the main reason they like their host community.

Children’s Schooling Background

Most of the children interviewed had never attended school. Only seven out of the 24 respondents in Borno and 11 out of the 24 respondents in Yobe had previously attended any formal school. Children attribute their lack of schooling to insecurity and a shortage or absence of schools in their former community before displacement. Most children who dropped out of formal schools were girls. They explained that they left school due to caregivers’ perceptions of poor quality of education in formal schools and low teacher attendance.

“I was in in school before I stopped going because the school does not impact knowledge, the teachers do not like coming to school, so I was asked to stop going because I am wasting my time going there” (Girl, 13 years old, Returnee, Yobe)

“When we asked to enrol in school, my father would say “not now.” At that time we did not have money, even if we had it will only help us to buy food or to buy clothes, it’s not that he had no intention to enroll us in school, it’s just because he does not have money. We were old enough to be in school” (Boy, 12 years old, IDP, Borno)

Girls also indicated that they had fallen behind in class performance as a result of irregular school attendance; poor economic prospects of parents and the prioritization of boys’ education and a greater emphasis on moral education for girls, who were more likely to be enrolled in religion-focused Koranic, Tsangaya and Islamiyya schools.

Children’s Motivations to Attend the ALP

All children expressed great excitement to be enrolled in the program and have access to education opportunities. As an internally displaced girl from Yobe put it:

“When I came home I was told they had enrolled me. I was very happy, I was happy like never before! The happiness was just too much, it was more than the happiness I felt when I was younger and my mother used to put me on her back. In the past we didn’t have the opportunity to attend school, so when we were enrolled, we were very happy because now there is no way our friends will make fun of us because we are not educated” (Girl, 10 years old, IDP, Yobe)

Children provided different responses to their motivation to attend the ALP. First, the program was seen as a flexible alternative which allows them to either start or continue their education without over-burdening parents and guardians with additional costs. The flexibility of the ALP allows them to carry on with their daily activities such as supporting family income and household chores and their established schedules with other education, including attending Islamiyya schools. Children also expressed their hope to grow up to be professionals and educate their own children. These children appear to be aware of the importance of education as an enabler toward a brighter future with “better economic prospects” and “better jobs.” These were closely followed by specific references to “learn to read and write,” “being respected in the community” and not “staying idle.”

“If people don’t learn, when they want employment they find that they only employ the educated. The uneducated one is lucky to be employed… but he’d only be employed as a security guard” (Girl, 10 years old, IDP, Borno)

Third, children indicated that simple registration procedures, word of mouth, peer influence and food motivated them to enroll. Adolescents heard about the ALP from peers and community members and there appeared to be no restrictions or challenges to enroll. The provision of biscuits, sweets and sachet water during school hours was a key motivation for many children to continue attending the ALP.

Children’s Experiences with The Curriculum

Children situate the most significant changes in their lives as a result of attending the ALP with the ability to learn to read, write, count and perform basic arithmetic calculations, all skills which are greatly valued.

Math

We found that 27 out of 48 students considered math their favorite subject because it was easy to understand and perceived to be the most useful given the practical applications for math knowledge in their daily lives:

“I like math because it will be most useful for me in the future. Everything involves calculation. For instance… if you are in a shop and you do not know how to calculate you can get cheated” (Boy, 13 years old, Host, Borno)

LFs generally found numeracy easier to teach because practical instructional aids are easy to access and use. Coaches considered math to be the easiest subject to support learning facilitators and reported that learning facilitators generally feel more comfortable teaching numeracy because of the practical approach involved.

Literacy

We found that 15 out of 48 students considered literacy to be their favorite subject because of its ease of learning and the perceived benefits in the ability to read and write. Students found the ability to write their names as one of the most important milestones in their learning outcomes from literacy instruction. Most LFs found literacy easy to teach thanks to the teaching and learning materials provided by the program, which were considered clear and simple due to the illustrations and storytelling. Some LFs ascribe the challenges in teaching literacy to language differences due to their own low levels of proficiency in Hausa. Coaches also indicated that LF proficiency reading Hausa at the ALP was an issue, and many of them had to rely on coaches to understand some instructions.

Social-Emotional Learning

There were mixed findings with regards to stakeholders’ experience with SEL. Students indicated that SEL helped them focus on their happiness, emotional wellbeing and relationships. LFs indicated that SEL had a positive impact on management of behavioral problems and emotional distress. They also indicated that it helped children get along better with their peers, make responsible choices and better understand the consequences of their actions. However, 17 out 48 children considered SEL difficult to learn:

“SEL is the hardest subject to learn, because it comprises a lot of things and we do not always understand them. We had to study extra carefully to learn it, but now we know it and enjoy it” – (Girl 10 years old, IDP, Yobe).

Additionally, eight out of 16 out of 31 LFs and four out of eight coaches considered SEL the hardest subject to teach and support. Some LFs with previous teaching experience considered SEL to be a way to merely “calm” down children by doing some exercises and that numeracy and literacy were more important. The lack of sufficient training for LFs on SEL, teaching and learning materials and clarity in some activity instructions and other difficulties conducting SEL in their setting were all factors that appeared to negatively impact LF’s ability to teach SEL. Some LFs reported that as a result of these obstacles, they skipped SEL lessons.

“We have challenges in SEL because there are some things we don’t understand from the manual, so need additional explanations” (Male, LF, 28 years old, Yobe)

“We don’t have some of the materials used in teaching SEL so that is why we just have to skip the lessons” (Female, LF, 30 years old, Yobe)

“SEL includes things that are difficult for the teacher about playing games and jumping here and there. Sometimes as a female teacher, you might not find it easy to do some things and then the children might not understand what you are teaching" (Female, LF, 32, Borno).

Four out of eight coaches also indicated that SEL was the most difficult subject to support, suggesting that the reason for this is that SEL is a new concept in the region and the program did not sufficiently support LFs and coaches to understand and adequately implement SEL. As two coaches put it:

“SEL is a new concept, it’s a new thing. Even though we have been teaching for a long time, SEL is still seen as new. One has to think deeply coupled with experience before one understands” (Female, 52 years old, Coach, Borno)

Children’s Experiences with Learning Facilitators

LFs are respected by children and community members alike and seen as an influential, positive role model to the children. Children attribute their educational achievements to the LFs and are appreciative of them. The traditional impression of a teacher, particularly for children who had previously attended school before enrolling in the ALP, is their being saddled with the responsibility of punishing children. This often may involve the use of whips of errant children or those who fall behind in learning. The concept of “not beating” is generally new to children in the ALP and helps in fostering warm teacher-student relationships.

“If we do anything wrong, the learning facilitator talks to us instead of beating us. If we don’t understand something she clarifies it for us” (Girl, 10 years old, IDP, Yobe).

Children acknowledged that positive relationships helped them learn and understand better and improved their social and emotional wellbeing. The lack of corporal punishment at the ALP improved children’s willingness and confidence to seek clarification from the teacher when needed. As some children put it:

“We find it very easy to speak to the learning facilitator because he is easy-going. He told us that if any one of us has an issue, we shouldn’t feel shy or ashamed to talk to him. Even if we don’t understand something, we can ask him. We are always doing that and now we understand him. Because of that we appreciate him” (Boy, 14 years old, IDP, Borno)

Children’s Experiences with Their Peers

We found that 41 out of 48 children reported that making friends and relying on other children was easy and their favorite part of attending the ALP. When asked about what they didn’t like or would like to change about the ALP, most children situated their responses on quarreling and fighting among students, excessive noise making, bullying among students, roaming around the school during class hours and disobedience toward LFs.

Perceptions of Impact

Children situate the most significant changes in their lives as a result of attending the NFLC in their ability to write, read, count and perform basic arithmetic calculations, which are greatly valued by the children interviewed. Children also described skills such as the ability to write one’s name with a great sense of pride.

“Now I can read my father’s name, I can write my name, I can write everything” (Boy, 11 years old, IDP, Borno)

“Calculation, whenever we are taught, I concentrate so I could also do it. Before I started going to school when I was given money and I couldn’t count it. But there was a day I was given N3000 by my dad to count and I was able to count them all” (Boy, 13 years old, IDP, Yobe)

The second most influential change identified by children was an improved social skills and awareness of the importance to behave responsibly and focus on studying:

“I used to roam about before, I regarded anyone who advised me to go to school or any advice as oppression, I felt like they hated me. So when I started attending the NFLC, I understood that roaming about wasn’t good for me” (Boy, 14 years old IDP, Yobe)

“I used to play ball and bully but they taught me that bullying is not good. I consider important that they told me not to fight and I should concentrate on my studies” (Boy, 12 years old, IDP, Yobe)

A third impact identified by children was an improved awareness about their personal hygiene and cleanliness, as they indicated that LFs ensured that they always came to the centers looking clean, a habit which many children reported developing in the NFLC. Finally, children indicated that attending the ALP reinforced their belief in the value of education to improve future livelihoods.

What Are the Effects of the ALP on Children’s Literacy, Numeracy and SEL Outcomes?

Table 6 presents scores for the treatment and control groups at each assessment stage, the differences in the group means, and the average effects of ALP with corresponding Cohen’s d effect sizes. Overall, the ALP shows significant positive effects on various literacy and numeracy outcomes.

Table 6. DID estimates of ALP effects on student learning outcomes and effect sizes.

	Baseline			Endline			DID	Effect size
	Treatment	Control	Difference	Treatment	Control	Difference
Literacy
Letter identification	12.38	12.34	0.04	23.77	17.34	6.43	6.39** (2.18)	0.29
Non-words reading	5.12	5.54	−0.41	10.26	7.59	2.66	3.08** (1.14)	0.26
Oral reading fluency	7.97	8.09	−0.12	14.78	10.95	3.82	3.95* (1.64)	0.23
Listening comprehension	59.79	59.92	−0.13	73.69	68.96	4.73	4.86 (4.20)	0.13
Reading comprehension	7.99	7.57	0.41	20.10	17.05	3.05	2.64 (2.02)	0.14
Numeracy
Number identification	9.98	12.02	−2.04	18.84	15.24	3.59	5.63*** (1.21)	0.41
Number discrimination	38.05	38.98	−0.92	67.74	53.72	14.01	14.94*** (3.46)	0.43
Missing number	17.95	20.59	−2.64	39.52	28.84	10.68	13.32*** (2.37)	0.58
Addition L1	6.79	7.15	−0.35	11.78	9.19	2.59	2.94*** (0.89)	0.29
Subtraction L1	5.97	6.37	−0.40	10.76	8.17	2.59	2.99*** (0.80)	0.32
Addition L2	18.58	19.75	−1.17	42.24	30.36	11.88	13.06*** (2.63)	0.46
Subtraction L2	27.51	26.99	0.52	36.31	27.81	8.50	7.98* (3.38)	0.23
Word problems	28.82	31.75	−2.93	67.86	59.71	8.15	11.08*** (2.88)	0.37
SEL
Hostile attribution bias	0.74	0.74	0.00	0.61	0.65	−0.04	–0.04 (0.05)	−0.07
Sadness orientation	1.02	0.99	0.04	0.92	0.99	−0.07	–0.11 (0.08)	−0.11
Anger orientation	0.69	0.75	−0.05	0.60	0.58	0.02	0.07 (0.09)	0.07
Disengagement	0.05	0.06	0.00	0.04	0.07	−0.03	–0.03* (0.01)	−0.18
Problem-solving	0.57	0.57	0.00	0.61	0.61	0.00	0.01 (0.03)	0.02
Aggression	0.51	0.48	0.03	0.53	0.50	0.02	0.00 (0.05)	0.00
Emotional accuracy	0.71	0.70	0.01	0.72	0.72	0.00	–0.01 (0.02)	−0.07
Depression	3.57	3.40	0.18	2.80	2.64	0.17	–0.01 (0.44)	0.00

*p < .05. **p < .01. ***p < .001.

In literacy, the ALP improved three of the five EGRA-measured outcomes. Participating in the ALP led children in the treatment group to correctly identify six more letters per minute, decoding three more nonsense words and correctly reading four more words per minute than children in the control group. Effect sizes ranged from 0.23 SD (oral reading fluency) to 0.29 SD (letter identification). The program did not have any statistically significant effects on listening and reading comprehension skills.

In numeracy, the ALP improved all eight EGMA-measured outcomes. The program led children in the treatment group to correctly answer approximately six more number identification items, three more addition L1 items and three more subtraction L1 items per minute than children in the control group. The treated group also had higher gains than the control group in number discrimination by 15% points, missing number by 13% points, addition L2 by 13% points, level-2 subtraction by 7% points and word problems by 7% points. Effect sizes ranged from 0.23 SD (subtraction level 2) to 0.58 SD (missing number), indicating moderate to large effect sizes.

In SEL, the ALP led to a significant decrease in students’ tendency to deliberately sidestep or ignore conflict by 3% points but did not show any significant effects on other SEL skills. Given that the effect size of 0.18 SD is too small to be considered meaningful, and the number of tests that we conducted, it is very likely that this finding is spurious.

Do the ALP Effects Differ by Child Gender, Displacement Status and MTL?

Table 7 shows the differences that existed at baseline between different subgroups of children, as well as the differential treatment effects that the ALP had on different subgroups of children.

Table 7. Baseline differences in learning outcomes and differential treatment effects by subgroup.

	Baseline group differences			Differential treatment effects
	Female (vs. male)	IDP (vs. host)	Non-Hausa language (vs. Hausa)	Female (vs. male)	IDP (vs. host)	Non-Hausa language (vs. Hausa)
Literacy
Letter identification	2.02	–1.29	–0.80	0.80	–6.72	–4.55
	(1.27)	(2.76)	(1.80)	(3.49)	(4.74)	(4.04)
Non-words reading	0.86	–1.07	0.13	1.40	–0.86	–0.72
	(0.73)	(1.37)	(1.02)	(1.98)	(2.55)	(1.99)
Oral reading fluency	0.68	–1.42	0.40	–0.98	–1.66	–4.41
	(0.92)	(1.80)	(1.28)	(2.39)	(3.28)	(2.95)
Listening comprehension	1.78	6.93	–3.76	5.27	–14.86	–6.20
	(1.82)	(6.18)	(4.85)	(4.62)	(8.32)	(7.85)
Reading comprehension	0.93	0.09	–0.09	2.28	–2.97	–0.24
	(1.03)	(1.44)	(1.38)	(3.62)	(3.68)	(4.02)
Numeracy
Number identification	0.59	0.35	0.80	0.92	–7.28**	–5.70*
	(0.69)	(1.48)	(1.37)	(2.19)	(2.53)	(2.50)
Number discrimination	–0.69	5.58	–1.09	–3.06	–8.72*	–5.56
	(1.81)	(3.99)	(2.95)	(3.50)	(4.16)	(3.78)
Missing number	–0.66	2.33	–3.21	–3.06	–8.72*	–5.56
	(1.12)	(2.45)	(1.83)	(3.50)	(4.16)	(3.78)
Addition L1	0.68	–0.32	1.12	0.93	–4.14*	–3.55**
	(0.63)	(1.03)	(0.99)	(1.74)	(1.68)	(1.35)
Subtraction L1	0.20	0.16	0.96	1.18	–4.65***	–3.67**
	(0.55)	(0.86)	(0.88)	(1.67)	(1.36)	(1.28)
Addition L2	1.54	1.22	–1.62	6.54	–12.04*	–6.13
	(1.45)	(2.58)	(2.30)	(4.24)	(5.12)	(4.50)
Subtraction L2	–0.76	4.58	–1.71	8.56	–12.88*	–5.74
	(1.75)	(3.56)	(4.21)	(4.57)	(5.48)	(5.33)
Word problems	1.74	1.42	2.11	–2.43	–14.51*	–6.83
	(1.51)	(3.51)	(2.66)	(4.34)	(5.98)	(4.96)
SEL
Hostile attribution bias	–0.02	0.05	0.18***	0.14*	0.12	0.04
	(0.03)	(0.05)	(0.05)	(0.07)	(0.08)	(0.11)
Sadness orientation	–0.07	0.08	0.23*	0.27	0.15	–0.18
	(0.05)	(0.12)	(0.10)	(0.15)	(0.16)	(0.18)
Anger orientation	–0.00	–0.08	0.11	–0.00	0.06	0.16
	(0.05)	(0.08)	(0.13)	(0.17)	(0.18)	(0.23)
Disengagement	–0.00	0.02*	0.00	0.03	–0.05	0.04
	(0.01)	(0.01)	(0.01)	(0.03)	(0.03)	(0.03)
Problem-solving	–0.02	0.02	–0.03	–0.03	0.06	–0.01
	(0.02)	(0.04)	(0.04)	(0.05)	(0.06)	(0.06)
Aggression	0.01	–0.08	–0.02	–0.01	0.04	–0.07
	(0.03)	(0.07)	(0.07)	(0.09)	(0.08)	(0.09)
Emotional accuracy	0.00	–0.04**	–0.11***	–0.01	–0.03	0.01
	(0.01)	(0.02)	(0.03)	(0.02)	(0.03)	(0.04)
Depression	–0.23	1.34**	1.72*	–0.21	0.97	0.94
	(0.22)	(0.43)	(0.68)	(0.57)	(0.81)	(1.02)

*p < .05. **p < .01. ***p < .001.

By Gender

Results show that there were not any observable differences in the literacy, numeracy or SEL of children at baseline. Results from the moderation analysis showed that at endline, the intervention had similar effects on boys and girls in literacy, numeracy and the great majority of SEL outcomes. However, we found that gender moderates treatment effects in hostile attribution bias, such that the treatment helped decrease male learner’s orientation to interpret the intent of others as hostile by 14 more percentage points than it did for girls.

By Migration Status

We observed that at baseline, displaced children and members from the host community exhibited similar levels of literacy and numeracy, but displaced children were more disadvantaged than their counterparts with regards to their social emotional skills and wellbeing. Specifically, displaced children exhibited higher orientation toward disengaging from conflict and lower ability to identify emotions accurately than members from the host community. Displaced children also experienced higher levels of depression. Results from the moderation analysis show that at endline, the intervention had the same effect on the literacy and SEL skills of displaced children and members from the host community. However, displacement status moderated the effect of all eight EGMA-measured numeracy skills, such that the intervention was significantly more helpful for members of the host community than for displaced students with regards to improving their numeracy outcomes. Specifically, across different numeracy outcomes the ALP benefited children from host communities more than displaced children by four to 15 percentage points. As such, the intervention reinforced existing math achievement gaps between IDPs and host community children.

By MTL

At baseline, we observe that children whose MTL is Hausa and those who speak other languages at home exhibit similar levels of literacy and numeracy. However, children whose MTL is different from Hausa exhibit baseline SEL disadvantages. Specifically, they exhibit significantly higher levels of hostile attribution bias, sadness dysregulation and depression and lower levels of emotional attribution accuracy than children who speak Hausa at home. Results from the moderation analysis show that at endline, MTL moderates three numeracy skills (number identification, addition L1 and subtraction L1), which indicates that the intervention had a larger impact on the math skills of children whose MTL is Hausa (the medium of instruction in the ALP) than on children whose MTL is different than Hausa. Specifically, children who speak Hausa gained four to six percentage points of advantage in math over children who spoke other languages at home. Interestingly, at endline the intervention had the same effect on the SEL skills of children who spoke Hausa or other languages at home.

What is the Cost of Providing the ALP?

The ALP program cost on average 66 GBP per child, or 85 USD, for seven months of exposure to the program. All costs are expressed in 2018 GBP. Shared and direct program costs were estimated per student using an ingredients-based approach for the population who received the program and not only the quantitative evaluation sample. Because evaluation sample students were randomly selected from the program population, matching program-level average costs with impact estimates based on the research sample should not create any bias (Cost Analysis Standards Project, 2021). Estimating the per-student costs based on the total program population also allows us to capture costs at a more realistic scale than many impact evaluations of education programs, reducing concerns about the external validity of cost results (Evans & Popova, 2016). Overall, the program implementers spent a total of 1.18 million GBP from October 1, 2017 through September 30, 2018 on ALP. This period included a five-month inception phase (hiring, training, and curriculum and material development) from October 2017 through February 2018, and seven months of direct implementation for cohort 1 from March through September 2018. Our estimates include direct program costs (inputs necessary to run ALP and non-formal learning centers) as well as shared costs (country-level overhead such as HR, Finance, IT and management). Spending was calculated into two distinct cost “buckets,” Center Establishment Costs and delivery of the ALP. Center costs include all spending on infrastructure and supplies for learning facilities. Delivery of the ALP included items such as LF training, community-facing training, ongoing LF support and learner materials.

The program incurred significant startup costs: nearly a quarter of total spending, 23%, occurred prior to implementation start in March 2018. Approximately 15 GBP was spent per child during startup compared to 50 GBP per child during program implementation. Separating startup costs from recurrent costs also allows us to project the costs necessary to sustain ALPs in northern Nigeria for additional cohorts or to launch and sustain a similar program in a new context. The most costly ingredients included vehicle purchases required for operational purposes and the time of frontline personnel, salaries for Education Officers and stipends for LFs.

Over half the national staff positions were Education Officers who oversaw LF’s training and provided support to coaches. They also contributed to reporting and monitoring. The 370 LFs in cohort one received an initial five-day face-to-face training and a three-day refresher training on the ALP curriculum. The face-to-face training cost 100 GBP per facilitator, or 12.50 GBP per day of training for each facilitator (not counting the costs of master trainer training).

Approximately 42% of total spending was dedicated to the shared costs for the implementing organizations. Shared costs are costs for management and operations functions which support implementation (e.g. HR hires Education Officers, Supply Chain negotiates materials procurement) but which cannot be solely tied to one project or activity. Table 8 presents cohort one monitoring data and cost-efficiency results.

Table 8. Monitoring and cost-efficiency data during cohort one (October 2017-September 2018).

	Startup Phase	Implementation (Cohort 1)
Total: Direct Costs	£147,503	£541,282
International Staff	£17,897	£39,930
National Staff	£30,017	£82,515
Transportation	£1,097	£20,125
Capital Assets	£54,776	£0
Materials & Activities	£43,717	£310,376
Sub-Grantees	£0	£88,336
Total: Shared Costs	£130,476	£364,660
Total COST (Direct + Shared)		1,183,921
Total Children Reached	18,000
Avg. Cost per child by phase	£15	£50
Avg. Cost per Child Reached	£66 (2018 GBP)

Discussion

ALPs have been widely used in high, middle and low income settings to help over-aged OOS children access education opportunities to catch up with their peers (Adams et al., 2009; Charlick, 2003; Menendez et al., 2016) and some evidence exists from high income settings that ALPs can successfully help OOS children transition into formal schools (Akyeampong et al., 2018; Capizzano et al., 2007; Charlick, 2003; Jere, 2012). However, there is a dearth of evidence that a similar approach could be a cost-effective solution to help OOS children in conflict and crisis affected settings learn and enroll in school. Existing evidence from low income, conflict-affected settings document how ALPs help children with their enrollment, completion and progression outcomes (Longden, 2014; Manda, 2011; Sempere, 2009; Shah, 2015) but they do not document learning achievements and all studies lack a comparison group. The present mixed-methods RCT aimed to build rigorous evidence about whether an ALP can improve the learning and SEL outcomes of OOS children in Nigeria, how, for whom and at what cost. The insights we provide into the education needs of OOS children in northeast Nigeria and the effects of an ALP in the region contribute to the growing yet insufficiently developed literature on the cost-effectiveness of ALPs for out of school children in conflict-affected countries and the differential impact by different subgroups of children (FMoE, 2019; Menendez et al., 2016). To our knowledge, our findings provide the first rigorous evidence of the impact of an ALP program on the learning, SEL and wellbeing outcomes of OOS children in emergency settings.

Additionally, the qualitative data helped us gain insights about how the program works, and specifically, the challenges children experienced before the intervention, their motivation to attend the ALP, their experiences with the curriculum, the LFs and their peers and their perceptions about impact. The analysis showed that the ALP successfully addressed barriers that children in Nigeria experience enrolling (Antoninis, 2014; Coinco & Morris, 2017; Kainuwa et al., 2017; Kazeem et al., 2010; Kazeem & Jensen, 2017; OCHA, 2018; Uscategui & Andrea, 2017) and staying in school (Abdullahi & Terhemba, 2014; Duze, 2012; Kainuwa & Yusuf, 2013a, Kainuwa & Yusuf, 2013b; Kazeem et al., 2010; Olufunke & Oluwadamilola, 2014). Design features of the program that children identified as particularly helpful included the fact the ALP was provided free of charge, provided food, and was scheduled at convenient times that enabled children to attend the program without interfering with Islamiyya schools or with their ability to support their families with household or economic activities. Additionally, we confirmed that the ALP was perceived to be relevant and that children valued learning to read, write and do math because they felt these skills would help them get better jobs in the future.

With regards to literacy, the RCT in conjunction with the difference in difference analysis showed that the ALP was beneficial to OOS children’s acquisition of foundational literacy skills, including their ability to accurately identify Hausa sounds associated with individual letters, decode nonsense words and read with fluency. However, the ALP did not improve listening comprehension and reading comprehension, which suggests the need to provide systematic and continuous professional development supports for LFs so they can improve their instructional proficiency in teaching literacy as a meaning-making enterprise rather than as separate literacy skills (Lee et al., 2020). Data from the qualitative study indicated that coaches, LFs and children found the literacy curriculum and materials interesting and easy to teach and to learn but highlighted that a barrier of implementation was that some LFs experienced difficulties teaching Hausa. For many LFs, Hausa is their mother tongue, but while they are proficient in speaking, they often struggle reading this language, which suggests that a possible improvement that could lead to larger effects would be to improve LFs proficiency reading Hausa. Program designers and policy makers can explore how to address this issue through the development of strategies to identify, and recruit LFs who are proficient in the language of instruction and through continuous professional development opportunities.

In numeracy, the impact evaluation showed that the ALP improved OOS children’s acquisition of various conceptual and operational math skills, including the ability to accurately identify numbers, identify quantities, recognize missing numbers, add single- or double-digit numbers, subtract single- or double-digit numbers and conduct operations with real-world problems. Qualitative data about the experiences of LFs and children with the math curriculum suggest that math was highly valued by LFs and children and the materials provided were considered to be easy to teach and to learn.

With regard to SEL and wellbeing, the impact evaluation showed that the ALP led to a statistically significant decrease in children’s orientation to disengage from conflicts. However, the ALP did not have statistically significant effects on any other SEL outcome or on children’s feelings of depression. These results are consistent with findings from other impact evaluations conducted in other crisis affected settings which also show limited impact of SEL programming on SEL outcomes (Brown et al., under review; Tubbs Dolan et al., 2021). In this regard, data from interviews showed that coaches, LFs and children felt that SEL was useful and valuable, but they consistently reported that SEL was the hardest subject to teach and to learn because SEL is a new concept in northeast Nigeria with which they had little to no previous experience. LFs also reported low implementation of SEL lessons because activities were not clear or they were difficult to implement in their setting. Based on these findings it is very possible that the limited impact of SEL programming may be due to low uptake and low levels of quality and fidelity of implementation, which are known to affect program outcomes (Durlak & DuPre, 2008) and undermine efforts to effectively bring SEL programming to scale (Jones & Bouffard, 2012). Given that SEL is an emergent curricular concept in Nigeria and many other conflict-affected countries (Puerta et al., 2016) and that research shows that novelty in course concepts or teaching methods provokes discomfort and weak commitment to teaching of SEL even among certified teachers (Lee et al., 2019), it is critical to design SEL programming in ways that are closely aligned with local values and needs to increase uptake and acceptance. Additionally, given that SEL interventions that show positive effects in low-income settings commonly report positive implementer attitudes and instructional readiness as vital enablers (Baker-Henningham et al., 2009; Chitiyo et al., 2008), it is important to provide adequate teacher professional development opportunities to ensure that LFs grasp the value of SEL and develop the attitudes and skills they need to implement SEL.

This study used an equity lens to identify existing baseline equity gaps between different subgroups of students by gender, displacement status and MTL and the degree to which the intervention had differential effects on them and contributed to decreased inequalities. Equity analysis are important because rising inequality is a policy concern in Nigeria (FMoE, 2017) and globally (Equity Initiative, 2016), but the great majority of studies do not systematically gather and analyze data on disparities in education outcomes and resources. As a result, there is no systematic and consistent approach to building evidence on what works for building equity (Equity Initiative, 2016). We disaggregated data by gender, as gender equality is a global priority reflected in the Sustainable Development Goal 5, which aims to achieve gender equality and empower all women and girls (UNHCR, 2017). We disaggregated data by displacement status because displaced children are often invisible in education data systems but face multiple challenges accessing quality education opportunities due to discrimination or language and cultural barriers (UNICEF, 2019). We also analyzed data by MTL, as children’s command of the language of instruction is a strong determinant of academic performance (Brock-Utne, 2007).

With regards to gender, we found that girls and boys had similar literacy, numeracy and SEL skills at baseline and that the intervention helped both groups build foundational literacy and numeracy skills but was significantly more helpful for girls in decreasing hostile attribution bias. With regards to displacement status, our analyses show that displaced and host community children had similar baseline literacy and numeracy skills but displaced children exhibited SEL disadvantages compared to children from the host community. When looking at the differential impact of the ALP by displacement status, we observe that children from the host community were able to reap more academic benefits from the program than displaced children. While this finding is only statistically significant for numeracy impacts, we observe a similar trend for literacy impacts. With regards to MTL, we found similar baseline literacy and numeracy skills by MTL. However, non-Hausa speakers appeared to have lower SEL skills than children who spoke Hausa at home, a finding that may be due to differences in how children who do not speak Hausa may have interpreted the SEL questions at baseline. Analysis of the differential impact of the ALP by MTL showed that Hausa-speakers benefited more from the ALP than non-Hausa speakers, which is reflected in significantly larger impacts on their numeracy. A similar trend is also observed for impacts on literacy skills but those differences are not statistically significant.

These subgroup findings are consistent with research that suggests that students are better able to develop cognitive skills in a language known to them than in an unfamiliar language. In multilingual contexts, it is important to allow students to develop cognitive skills in a familiar language, as this facilitates literacy acquisition in additional languages (Alidou et al., 2006; Ball, 2011; Cummins, 2001; Hewlett Foundation, 2014; Ouane & Glanz, 2011; PASEC, 2015; Thomas & Collier, 2002). In Nigeria, Hausa is the official language of instruction for the ALP and for in-school children until grade 3. But for more than 28% of children attending the ALP, Hausa was not their MTL. The differential benefits between children from different language groups suggest that additional instructional support is warranted for children who have to learn reading and math in a second language. These results suggest that additional support is essential for dual language learners to succeed in school (Gibbons, 1993).

Finally, the present study provides relevant information about the cost of the intervention. In recent years, educational research has increasingly focused on doing experimental and quasi-experimental studies to determine the effectiveness of programs. But these studies rarely address the issue of the resources required for effective program implementation (Hollands et al., 2016). Our study shows that at an average cost of 66 GBP per child, the ALP is an impactful approach for promoting OOS children’s foundational literacy and numeracy skills, despite years of learning losses and adverse life circumstances. After the startup phase, the cost-effectiveness would increase because the cost of continuing the implementation of ALP would decrease to 50 GBP per child. These findings expand the positive role of ALPs in facilitating OOS children’s academic achievement from stable, high and low income countries (Adams et al., 2009; Akyeampong, 2014; Akyeampong et al., 2018; Capizzano et al., 2007; Charlick, 2003; Jere, 2012) to conflict-ridden settings. Further research is needed to clarify the impact of the ALP on OOS children’s transition outcomes, a widely used metric to assess cost-effectiveness of education programs, or the degree to which it helped students enroll, attend and learn in school.

Limitations

Concerns about internal validity address the question of whether the intervention caused the changes observed in the outcomes of participants (Shadish et al., 2002). A threat to the internal validity of this study relates to the fact that randomization conducted using public lotteries within each community may not have been successful as we observed some baseline differences between children in the treatment and control groups. We sought to minimize this threat by using DID analysis to account for existing baseline differences and also added controls for variables where we had observed differences between groups. In using DID, we are making a parallel trends assumption, trusting that in the absence of the treatment the differences between the treatment and control groups would be the same over time. While we do not have data points to test the parallel trends assumption before treatment, a possible post-treatment violation would occur if new interventions or services were offered in the region and children in the control group would be prioritized over children in the treatment group given their marginalization status and urgent need for support. The short time between baseline and endline −6 months- reduces this threat, but we do not have data points to test the assumption. A second threat to validity to consider is attrition. At endline we encountered a small attrition rate of 5.3%. Attrition might introduce bias in our analysis if it were differential between the treatment and control groups. To assess this issue, we conducted a balance test between the two groups in the remaining sample. Results showed the attrition was not consequential and that the composition of the study’s participants stayed unaltered. Finally, the third threat to the internal validity of this study is contamination, as children in the treatment and control groups and LFs were from the same communities. Given that children in the control group lived in physical proximity to children in the treatment group it is possible that they might interact outside the ALP classes, which would lead to spillover and a possible underestimation of treatment effects.

Concerns about the external validity of the study refer to the extent to which the causal relationships that we identify between the program and students’ literacy, numeracy and SEL outcomes hold over variations in persons, settings, treatments and outcomes that were in the experiment (Shadish et al., 2002). The findings of the present study are relevant for primary and secondary school-aged out of school children, in low- and middle-income countries affected by conflict and crisis and with low access to qualified teachers, or where an existing formal school system is weak or overwhelmed. Worldwide there are tens of millions of children and youth who meet these criteria and so while these qualifiers may sound narrowing, interventions that positively impact this subset have wide-ranging and global implications. This is particularly true within societies affected by conflict and crisis. The findings of the study are not relevant for older populations of out of school children, or for OOS children in high income countries and stable settings, or for children in conflict and crisis settings where there is access to a qualified teaching workforce and where there is a strong formal school system in place.

Conclusion

Early adversities endured by Nigeria’s war-weary, out-of-school children form a profound risk to their academic and psychosocial well-being. The results of this study can inform policymakers, donors and practitioners in Nigeria interested in providing access to quality learning opportunities for OOS children. In fact, while positive links between various accelerated curricula and OOS children’s learning and development have been established in stable societies, our analysis offers the first rigorous evidence in conflict-ridden countries. The information we offer is rich as it encompasses whether the ALP works to improve the learning and SEL outcomes of OOS children, how, for whom and at what cost. We find that ALPs have the potential to be a cost-effective way to help OOS children in contexts of conflict and crisis acquire foundational literacy and numeracy, but that additional attention needs to be given to development and adequate implementation of SEL programming in these settings. Future research can identify the effects of the program on children’s enrollment and progression in school, and identify ways to improve impact of SEL programming and the provision of targeted school-based mental health support from children who require additional attention. We hope that this information will be instrumental in encouraging more research and policy investments in these areas in the future.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

The work was conducted by the International Rescue Committee. The present research was financially supported by the the Foreign, Commonwealth and Development Office (FCDO), formerly the Department for International Development (DFID).

Notes

1 HLE and SES are indices created through principal components to extract a construct across multiple dimensions. The HLE index contains items indicating various reading materials available to a child at home such as schoolbooks, storybooks or other text materials such as religious books. The SES index is composed of various household assets that a child’s family owns, such as radio, TV, bicycle and land.