Does grade retention affect students’ achievement? Some evidence from Spain

Abstract

Grade retention practices are at the forefront of the educational debate. In this article, we measure the effect of grade retention on Spanish students’ achievement by using data from Programme for International Student Assessment (PISA). We find that grade retention has a negative impact on educational outcomes, but we confirm the importance of endogenous selection which makes observed differences between repeaters and nonrepeaters appear about 14% lower than they actually are. The effect on scores of repeating is much smaller (–10% of nonrepeaters’ average) than the counterfactual reduction that nonrepeaters would suffer had they been retained as repeaters (–24% of their average). Furthermore, those who repeated a grade during primary education suffered more than those who repeated a grade in secondary school, although the effect of repeating at both times is, as expected, larger.

Keywords:

• grade retention
• educational scores
• PISA

JEL Classification:

• D63
• I28
• J24

View correction statement:

Corrigendum

Notes

¹ The average PISA  2009 test scores of Spanish students in math, reading and science are 480, 484 and 488, respectively, which are 13, 12 and 13 points below the respective OECD means and, obviously, much smaller than the scores in the best-performing countries, the Republic of Korea and Finland, where students score above 530 points in all disciplines.

² See Dearden et al. (2006) for an analysis of policies aimed at reducing dropout rates in the UK.

³ Some studies have found that retention is associated with increased dropout rates (see Jimerson et al. (2002) and Roderick (1994), among others). However, as retention decisions are typically made by the teacher or school principal on the basis of a number of unobservable student characteristics (such as maturity or parental involvement), all of these studies are plagued by serious selection concerns.

⁴ Many studies use this type of model to analyse different aspects of the labour market. See, among others, García-Pérez and Jimeno (2007), Carrasco (2001) and Prescott and Wilton (1992).

⁵ The prevailing educational law in 2009 was the 2006 Organic Educational Law (LOE). For more statistics and details on the Spanish educational system, visit http://www.educacion.gob.es/ievaluacion/publicaciones/indicadores-educativos/Sistema-Estatal.html

⁶ The regions with a representative sample are Andalusia, Aragon, Asturias, Balearic Islands, Canary Islands, Cantabria, Castile Leon, Catalonia, Galicia, La Rioja, Murcia, Madrid, Navarre, Basque Country and Ceuta-Melilla. We refer to the three regions for which no representative sample is available (Extremadura, Castilla-Mancha and Valencia) as ‘the rest of Spain’.

⁷ The PISA programme assesses students’ performance in three disciplines: science, math and reading. PISA 2009 edition focused on reading. Following the OECD’s recommended methodology (see the PISA Technical Report), we use the 5 plausible values and 80 sampling weights to calculate each student’s educational outcome and the SEs of the estimated coefficients.

⁸ This definition is based on questions 1 and 3 of the PISA Student Questionnaire.

⁹ The PISA sample has a stratified two-stage design. First, schools with 15-year-old students are selected, and second, within each school, individual students are selected. See PISA 2009 Technical Report (2011).

¹⁰ Ceuta and Melilla, which participate jointly in PISA, are the cities with the poorest performance (e.g. their average math score is 417). However, because they have small relative dimensions within Spain, we considered them in our econometric analysis, but we do not comment on them when reporting some of our results.

¹¹ A father’s education is ‘high’ if he has a secondary or higher education degree and ‘low’ if he has a primary or lower education degree. The same categories hold for mothers’ education.

¹² Regarding school ownership, we distinguish between public, government-dependent private (i.e. those with a percentage of public funding above 50%) and independent private (i.e. those with a percentage of public funding less than or equal to 50%).

¹³ There is no clear empirical evidence on the impact of class size. Angrist and Lavy (1999) find that reducing class size induces a significant and substantial increase in test scores. However, Hanushek (1998) finds no significant impact of class size reduction on scores. Lazear (2001) argues that the reason why there is no consensus in the literature is because class size is a choice variable: schools adapt class size to students’ type and behaviour.

¹⁴ This definition is based on question 7 of the PISA Student Questionnaire. Note that there is a slight difference between the number of repeaters according to the general definition above (that is, based on questions 1 and 3 of the PISA Student Questionnaire) and the total number of repeaters obtained by adding Rep_ P, Rep_ PS and Rep_ S. We assume this difference to be due to measurement error.

¹⁵ Notice that the percentage of repeaters is the sum of the percentages of repeaters in each subgroup (primary only, primary and secondary, secondary only). Thus, the slope in panel (a) is the sum of the slopes in (b), (c) and (d).

¹⁶ PISA assesses the extent to which students near the end of their compulsory education have acquired some of the knowledge and skills that are essential for full participation in modern societies. PISA seeks not only to assess whether students can reproduce knowledge but also to examine how well they can extrapolate from what they have learned and apply it in unfamiliar settings both in and outside of school (see PISA 2009 Report).

¹⁷ As the error term of each student’s score equation is correlated with the error term of the selection equation, the estimation of the student’s score equations by OLS would be inconsistent. Furthermore, full maximum likelihood is more efficient than the two-step estimation method proposed by Heckman (1979).

¹⁸ For example, if the experience of repeating makes the student’s subsequent effort increase, we may observe a higher PISA score among repeaters compared with the counterfactual of what would have happened had the student not repeated.

¹⁹ Recently Hungerman and Buckles (2010) find, for US censuses data, that season of birth is not really exogenous. In particular, they find that children born in the first quarter are disproportionately likely to be born to women from poor socio-economic status. This might explain the results of those studies that use quarter of birth as an instrument for educational attainment on wage equations (see Angrist and Krueger (1992, 1995) among others). However, we do not find any fertility patterns in Spain (our results using administrative data, Padron de Habitantes, are available upon request). In addition, we find no correlation within our estimation sample between quarter of birth and socio-economic variables. Therefore Hungerman and Buckles’ findings do not undermine validity to our results here.

²⁰ In order to check the robustness of our results, we have estimated the SRM with several alternative specifications for the instrumental variable. In particular, we have considered three dummy variables for three quarters of birth, 11 dummy variables for each month of birth and finally we have introduced the student month of birth in a continuous manner. See Section IV.

²¹ The reference student in both equations is a male from the Canary Islands, a native of Spain, born in the first or second quarter of the year, with low frequency of using a computer for homework and games, whose mother and father are low educated and living at home, with fewer than 26 books at home. Regarding the school variables, the reference student attends a public school with a minority of boys and in a classroom size equal to 22 (the sample mean).

²² Observe that we are cautious about interpreting the coefficients of the school variables here. As PISA does not provide information regarding the length of time the student has spent in the present school, then our school variables might not account for the real educational environment in which retention eventually occurred. In other words, we are implicitly assuming in our analysis that the school variables included in the selection equation are highly correlated with their value in the past. Nevertheless, and to check the robustness of our results, we estimated the model without school variables in the selection equation. The results (available upon request) are basically the same as the ones presented in this section.

²³ Moreover, we test the joint significance of the instruments Q3 and Q4 and estimated a Chi-square statistic (2 dfs) of , reaffirming significance at 1% (p-value < 0.01). The Wald test also concludes that the instruments are also jointly significant at 1% for the rest of the specifications: (three quarters of birth) and (11 months).

²⁴ The remaining results obtained under alternative categorical and continuous specifications of the instrument are all very similar to the ones included here (regarding both size and significance of the coefficients). Complete results are available upon request.

²⁵ In addition, and in order to compare our results with those in which there is no control for endogenous selection, we perform an independent probit estimation for Equation 2. The coefficients of this model and those of the SRM estimation in Table 4 are very similar. The only differences that emerge are that the effect of a majority of girls at school and a private school are significant at 5%, in the probit model, and not at 1%, as in the SRM.

²⁶ The results regarding and are robust to other sets of instrumental variables. For example, we explore the validity of considering two additional instruments to the quarter of birth. These are whether the student’s mother and/or father does not live at home and frequency of playing computer games. We claim here that a student’s father or mother may not live at home because of a previous parental death or divorce, which, according to the existing literature, does not negatively affect teenagers’ cognitive skills (see Sanz de Galdeano and Vuri, 2007), such as the skills measured in PISA scores. However, parental death or divorce may affect a student’s probability of repeating a grade by the time it occurs. Finally, we consider playing computer games as another instrument, as it is not significant in explaining the PISA test scores (see Table 5), but it has a huge impact on the propensity of repeating a grade (see Tables 1 and 4).

²⁷ Among possible explanations, we suggest the early emergence and persistence of gaps in cognitive and noncognitive skills (see among others, Carneiro and Heckman, 2003). This issue also requires special attention as, according to recent evidence, family environments have deteriorated (Heckman and Masterov (2004) find that children raised in these types of families fare worse in many aspects of social and economic life).

²⁸ For example, the average cost of schooling in Spain in 2007, in terms of government and family expenditures, was, at current prices, €4870 and €6508 per student at the primary and secondary level, respectively. These figures amount to a yearly cost of schooling of €811 and €1627, for primary (6 years) and secondary (4 years) education, respectively (see Instituto de Evaluación, 2010).