Precision and stability of schools’ value-added estimates: evidence for Italian primary schools

ABSTRACT

Imprecision of school value-added estimates has some important implications, especially for policies that use school value-added estimates to predict future performance of schools based on their past and current performance. This note investigates how the precision and stability of a school’s value-added estimates relates to student characteristics. Using Italian administrative data provided by INVALSI (National Evaluation Committee for Education), we find that the stability of school value-added estimates across cohorts does not depend on students’ previous achievement and socio-economic background. This suggests that transitory changes, such as changes in teacher effectiveness, might be associated with the instability in school value-added estimates.

KEYWORDS:

• School value-added
• imprecision
• stability
• school effect

JEL CLASSIFICATION:

• I21
• I20

Acknowledgments

We are grateful to INVALSI for having provided the original dataset, and P. Falzetti for the statistical assistance in building the specific database used in this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

¹ Agasisti and Minaya (2018) restricted their analysis to schools with at least 25 students. For this note, we also impose some restrictions on the data in order to accurately identify the parameters of interest. Students missing second grade scores or other covariates used in value-added estimation are excluded.

² For each inter-cohort correlation coefficient, we first fit a mixed model with shrunken school effects using Bayes shrinkage estimators using one, two and three years of data. Our model controls for demographic characteristics such as dummies for student type, gender, immigration status, and student SES and a vector of classroom-level means that include student gender, immigration status, student type, as well as the class size and average second grade scores of the students in the j-th class. Value-added estimates using two and three years of data are also controlled for cohort fixed effects. For instance, the two-year school value-added estimate using the 2013 and 2014 cohorts of data is estimated by pooling these cohorts, fitting our model, and controlling for the 2013 cohort. After fitting these models, we correlate the latest one-year school value- added estimates (2015 and 2016) with the two-year earlier estimates, (2013/14 and 2014/15, respectively).

³ It is worth nothing that by stability we mean the correlation coefficients from cohort to cohort in a school’s value-added estimate.

Additional information

Funding

This work has been funded within FARB—Public Management Research: Health and Education Systems Assessment, sustained by Politecnico di Milano.