Critiques of socio-economic school compositional effects: are they valid?

Abstract

School socio-economic compositional (SEC) effects have been influential in educational research predicting a range of outcomes and influencing public policy. However, some recent studies have challenged the veracity of SEC effects when applying residualised-change and fixed effects models and simulating potential measurement errors in hierarchical regression models. We review the residualised change and fixed effects methods in critical studies and find limitations in their capacity to demonstrate null compositional effects. We show this with an adjusted residualised change model finding significant SEC effects. We show structural equation models can address concerns that measurement errors inflate SEC effects by comparing hierarchical regression models to structural equation models. We find that structural equation models can detect SEC effects free from measurement error. We conclude that the reviewed critiques of SEC effects were due to methods unlikely to detect compositional effects. Future research would benefit from the identification of mediators of SEC effects.

Keywords:

• school segregation
• measurement error
• school composition
• residualised change
• fixed effects
• socioeconomic composition

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1 Sirin cautioned that the effect size for aggregated measures of SES in his meta-analysis may have been biased by the ecological fallacy.

2 We found that a two-level factor analysis of the subindices of ESCS, where holding factors equal across all countries in the full international sample, did not fit the data, having insignificant factor loadings. This is consistent with the OECD’s report (OECD 2017, 340) of differential national item loadings on ESCS. A drawback of allowing item loadings to vary by country is that factors are not internationally comparable.

3 OECD (2016, 225) reports that in OECD countries, 30.1% of variance in academic achievement was at the school level, 69.9% at the student level. OECD (2016, 227) reports that in OECD countries, 62.6% of school-level variance in academic achievement is due to socio-economic status of students and schools while 3.8% of student-level variance in academic achievement is due to socio-economic status of students. Therefore, the proportion of variance in academic achievement due to school-level socio-economic factors = (0.626 × 0.301)/((0.626 × 0.301) + (0.038 × 0.699)) = 0.876.