Providing academic opportunities to vulnerable adolescents: a randomised evaluation of privately managed tuition-free middle schools in Uruguay

ABSTRACT

We conducted a randomised evaluation of three privately managed middle schools in Uruguay aimed at providing education opportunities to adolescents from low income socioeconomic status. At 3-year follow-up, treatment students fare better in terms of academic promotion and school retention. Students in treatment schools present also better mental health, as represented by lower rates of internalising behaviours and social problems than students in the control group. Our findings suggest that a culture of high expectations, a caring and disciplined school climate, and parental involvement in school could account for some of the observed differences in academic trajectories and mental health.

KEYWORDS:

• Middle school
• impact evaluation
• randomised controlled trial
• education
• vulnerable adolescents
• innovative schools

Funding

Inter-American Development Bank (IADB)

Acknowledgments

We acknowledge funding from the Inter-American Development Bank (IADB). All opinions are our own and do not represent those of the IADB or of Universidad de Montevideo. We are grateful to Joaquin Klot for research assistance and to Sandro Diez-Amigo for useful comments during the early stages of the project.

Ethical approval

All the review was performed following the ethical standards.

Data availability statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to their containing information that could compromise the privacy of research participants.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. In Uruguay, public school teachers can choose every year the number of hours and school(s) in which they will be teaching. Their order of choice is given by the teachers’ degree, defined mostly by their tenure and experience.

2. Although there are efforts aimed at extending the other two practices (providing teachers with feedback and using data to assess school practices), these efforts are not yet widespread across the three schools.

3. Secondary education in Uruguay includes middle school (years 7 to 9), and high school (years 10 to 12).

4. The school year in Uruguay runs from February/March to November/December.

5. Siblings of current or former students are exempted from the lottery and given priority in the enrolment process. In one of the schools, children who attended an after-school programme associated with the organisation are also given priority.

6. To minimise attrition in the control group at baseline, we defined a replacement for each of the students in the control group, in such a way that the observed characteristics of the replacement were similar to those of the original control. The fraction of original controls not found and replaced was 10%.

7. 79% of the students in the control group enrol in public schools (80% of them in general public schools and 20% in vocational schools), 17% attend private schools, and 4% drop-out from school.

8. Results remain unchanged when we only take into consideration the schools with higher compliance. These results can be shown upon request.

9. For one observation, the education of another adult at home was imputed instead, since data on mother´s education was missing.

10. There were 6 students in the sample who were randomly selected into treatment at school B, and were also in the control group at school A. There were 8 individuals selected into treatment at school A, who were also in the control group at school B. 10 additional students were part of the control groups of schools A and B and 6 subjects were part of the treatment groups at schools A and B simultaneously. Overall, 30 observations of the sample shared treatment or control status between schools A and B. To solve this, we duplicated these observations so that each one was uniquely associated either to the treatment or the control group of a single institution. Also, as a robustness check, we excluded these 30 students from all our estimates, and results remain roughly unchanged, confirming the positive impact of the treatment schools. Results are available upon request.

11. These questions seek to capture the processes and tactics employed by the students to improve learning. These tactics include both metacognition – knowledge and control of the cognitive process of learning and strategic regulation of such process (Flavell 1979) – as well as the management of the time assigned to studying. Evidence reveals that better learning strategies favour higher academic achievement (Hattie et al. 1996; Haller et al. 1988; Higgins et al. 2005), particularly when such strategies are simultaneously developed with students’ self-efficacy for the accomplishment of academic goals (Paris et al. 1983; Pintrich and De Groot 1990; Pokay and Blumenfeld 1990).

12. The academic literature has found connections between each of these five academic mindsets and the academic performance of adolescents. For example, a greater sense of school belonging is associated with better student achievement (Goodenow 1993; Battistich et al 1995; Roeser et al. 1996; Osterman 2000; Furrer and Skinner 2003; Flook, Repetti, and Ullman 2005; Cohen and Garcia 2008). Similarly, growth mindsets or positive theories of intelligence prompt better educational performance among young people (Dweck 2006; Blackwell et al. 2007; Yeager and Dweck 2012).

13. The distinction between these two dimensions of motivation is informative, as there is some evidence that when students identify intrinsically with academic goals (rather than extrinsically) they are more likely to commit to education and achieve better academic outcomes (Vansteenkiste et al. 2004; Vansteenkiste et al. 2005; Guthrie et al. 2006).

14. Self-control is a skill that enables conscious decision-making in the short run (Duckworth and Seligman 2006). Grit, on the other hand, also requires an effort to achieve long-term objectives (Duckworth et al. 2007). Empirical evidence indicates that self-control is a very important skill for the educational progress of adolescents (Tangney et al. 2004; Duckworth and Seligman 2005; Duckworth et al. 2010) and that this ability is often primarily developed during childhood (Gottfredson and Hirschi 1990). Grit is also associated with better academic performance (Duckworth et al. 2007; Duckworth and Quinn 2009; Eskreis-Winkler et al. 2014), although there is still inconclusive evidence regarding its malleability.

15. There is evidence linking social skills with better academic performance (Wentzel 1991; Wentzel 1993; Durlak et al. 2011).

16. Non-compliance was particularly large in one of the three schools, where 45% of those with ITT = 1 ended up not attending the school and 10% with ITT = 0 ended up attending.

17. Other problems include being cruel with animals, not eating well, biting nails, overeating, being overweight, showing off, sleeping more than other adolescents, talking too much, and wetting the bed.

Additional information

Notes on contributors

Ana Balsa

Ana Balsa is Professor at the School of Economics and Business of Universidad de Montevideo and Editor of the Journal of Health Economics. She received his PhD in Economics from Boston University. She has several publications in the fields of Economics of Health and Economics of Education. She is interested in Early Childhood/Parenting Programs, Innovative Interventions/Middle Schools, Substance Abuse and Obesity.

Alejandro Cid

Alejandro Cid is Associate Professor at the School of Economics and Business of Universidad de Montevideo. He received his PhD in Economics from Universidad de San Andrés. He has several publications in the fields of Economics of Education and Development. He is interested in Early Childhood/Parenting Programs, and Innovative Interventions/Middle Schools.

Ana Laura Zardo

Ana Laura Zardo is Research Assistant at the School of Economics and Business of Universidad de Montevideo. She has contributed in several projects related to Economics of Education and Applied Microeconomics.