Meta-analytical insights on school SES effects

ABSTRACT

The present study uses meta-analysis to synthesise findings involving 480 effect sizes from 97 studies (dated 2000-2020) to provide insights on associations between school socioeconomic status (SES) and (a) student learning outcomes; (b) schools’ percentage of ethnic minority students and students’ prior ability; and (c) school processes in K-12 schools. It makes three contributions to the school SES scholarship. First, it elucidates the magnitude (r = .58) and nature of school SES effects (e.g. larger effect sizes for achievement (vis-à-vis attainment) outcomes). Second, it clarifies the conceptual meaning of school SES, namely that school SES is less associated with school processes than it is with schools’ percentage of ethnic minority students or students’ prior ability. Third, the study shows that school SES is more strongly associated with specific school processes (school leadership and climate, teacher capacity, parental involvement benefiting student learning) than others (instructional programmes, educational resources).

KEYWORDS:

• School compositional effect
• school contextual effect
• school SES
• school socioeconomic status
• student achievement

The Coleman Report (Coleman et al., 1966) is one of the most influential educational publications in the last fifty years (Hanushek, 2016). The report involving data from more than 3,000 schools and about 600,000 students in the US found that schools had little influence on student achievement after students’ family and social backgrounds were accounted for. For example, compared to school and classroom instructional processes, the mean level of student SES (i.e. school SES) and ethnic composition in schools explain a greater proportion of the variation in student learning outcomes. Implications of the study are stark; some have argued for school desegregation while others have highlighted the importance of families and irrelevance of schools (Perry & McConney, 2010).

Since the Coleman Report, decades of research have investigated the relationship between school SES and students’ learning outcomes (Early et al., 2020; Kim, 2019; Kim et al., 2019; Sirin, 2005). Students’ SES refers to their relative access to valued economic, cultural, and social resources that contribute to valued life outcomes. It is commonly measured using parental educational attainment, family income, parental occupational status, and home resources. School SES refers to the mean level of student SES in a school. Researchers measure school SES by averaging students’ SES levels within a school or by using other school-level variables such as parental education, family income, home possessions, percentage of students on free/reduced-price lunch or schooling (FRL), Title 1/non-Title 1 school classification (Sirin, 2005; van Ewijk & Sleegers, 2010).

There is evidence that compared to student SES, school SES is more strongly associated with students’ learning outcomes (OECD, 2016; Sirin, 2005). For example, Borman and Dowling’s (2010) reanalysis of Coleman and colleagues’ original data using the same model specification but with a more advanced statistical technique (multilevel regression) found that schools’ social composition (students’ social class, race/ethnicity) was 1.75 times more important than students’ individual social and racial/ethnic background in explaining the variation in students’ learning outcomes. Relatedly, Sirin’s (2005) meta-analysis reported a mean effect size of r = .73 for the relationship between school SES and students’ learning outcomes (e.g. achievement) in the US vis-à-vis r = .30 for the relationship between student SES and their learning outcomes.

Researchers have sought to investigate how and why school SES levels are associated with students’ learning outcomes (i.e. the school SES effect). Studies have generally found a positive association between school SES and students’ learning outcomes (Sirin, 2005; van Ewijk & Sleegers, 2010; Willms, 2010). On the one hand, school SES researchers recognise the importance of school SES and therefore, identify processes underpinning the school SES effect (Akay & Karadag, 2019; Li, 2018; Mitchell et al., 2016; Park et al., 2017; Xuan et al., 2019; Zhu et al., 2019). However, most scholars focus on specific processes instead of examining a comprehensive gamut of processes, thereby precluding the identification of more important influences. On the other hand, school SES research has been criticised on theoretical and methodological grounds (Armor et al., 2018; Zhou & Ma, 2021). For example, researchers sometimes make erroneous individual-level inferences from group aggregated data (Robinson, 1950). Specifically, a positive association between school SES and mean performance level does not mean that student SES is associated with student achievement. Other concerns with school SES research include school-level SES measures suffering from poor reliability (Marks, 2015) and model misspecification arising from omitted student variables (e.g. schools’ percentage of ethnic minority students, students’ prior ability) (Lauen & Gaddis, 2013; Marks, 2015; Mickelson et al., 2013; Snijders & Bosker, 2012).

The present study acknowledges these controversies and employs meta-analysis to clarify associations between school SES and (a) students’ learning outcomes; (b) schools’ percentage of ethnic minority students and students’ prior ability; and (c) school processes. The use of meta-analysis enables us to synthesise findings from the large number of studies examining school SES that have reported different patterns of results for the association between school SES and student outcomes. It also enables us to compare the magnitude of associations between school SES and different school processes. The study is important for two reasons. First, some scholars argue that, compared to school processes, family processes are more powerful predictors of student outcomes, especially when schools become more homogenous due to globalisation (Dumont & Ready, 2020; Heyneman, 2015), so ascertaining the strength of associations between school SES and students’ learning outcomes will contribute to the scholarship on the role of schools in student learning. Second, the present study unravels the “black box” (van Ewijk & Sleegers, 2010, p. 135) of processes associated with high-SES schools, thereby justifying policy interventions for the implementation of these processes in schools to improve student learning.

School SES indicators

Researchers measuring school SES using the average SES level of students within a school reduce measurement errors associated with measures of individual student SES, thereby producing more reliable estimates of school SES. The present study clarifies if school SES effect sizes reported in studies vary with types and the number of school SES indicators used (i.e. parental education, family income, home possessions, percentage of students on free/reduced-price lunch or schooling (FRL), Title 1/non-Title 1 school classification, combination of indicators). van Ewijk and Sleegers' (2010) meta-analysis of the relationship between students’ peer SES (including classmates’ and school SES) and achievement found that studies reported larger effect sizes if peer SES was measured using a composite (vis-à-vis separate indicators).

The percentage of students qualifying for free/reduced-price lunch/schooling is commonly used to measure school SES in US and to a smaller extent, UK studies. Indeed, all except one of the 106 school SES effects measured using this indicator in the present study came from US-based studies; the one non-US-based study was conducted in England and Wales (Walker et al., 2005). In the US, students from families with income not exceeding 130% of the poverty level qualify for free meals while those from families with income between 130% and 185% of the poverty level qualify for reduced-price meals (Sirin, 2005). According to the BBC (2020, October 26), students in the UK who live in households on income-related benefits qualify for free school meals if their annual household income (excluding welfare payments) does not exceed a specified amount after tax (£7,400 in England, Wales, and Scotland; £14,000 in Northern Ireland). In addition to the percentage of students who qualify for free/reduced-price lunch, a US school is classified as having low SES if it is a Title 1 school (Borman et al., 2001); Title 1 federal funds are granted to US public schools with a high proportion of students from low-income families to provide academic support to and learning opportunities for disadvantaged students.

School SES effects for different subjects

School SES effect sizes may vary for different student outcomes such as achievement (i.e. new skills acquired by students that are reflected in assessment grades and scores for different subjects) versus attainment (i.e. students obtaining important learning benchmarks such as being nominated for advanced mathematics programmes, being admitted to college, graduating from college) (Kim et al., 2019). The only published meta-analysis focusing on aggregated SES effects (including but not limited to school SES) by van Ewijk and Sleegers (2010) did not find significant differences in school SES effect sizes for student achievement in languages and mathematics. However, results from other meta-analyses examining individual and aggregated SES effects report mixed results. For example, Sirin’s (2005) meta-analysis found that SES had larger effect sizes for students’ mathematics (vis-a-vis verbal, science, or general) achievement. It is therefore unclear how school SES effects vary with different student outcomes – an area that is investigated in the present study.

Moderators

The study will examine moderating effects of contextual variables (students’ grade levels, school types, countries’ socioeconomic inequality) on the association between school SES and students’ learning outcomes. These contextual variables are examined for the following reasons. First, Sirin’s (2005) meta-analysis showed that SES effects varied among students from different grade levels, with larger effect sizes for students from higher grade levels as compared to kindergarteners. Kim and colleagues’ (2019) meta-analysis of student SES effects in developing countries reported larger effect sizes for the academic achievement of secondary versus elementary school students but no significant effect size differences in attainment outcomes for these two groups of students. Therefore, school SES effects may vary with students’ grade levels. Second, some researchers assert that school SES can explain differences in student performance in different school types (private versus public) (Dronkers & Robert, 2008; Thomson et al., 2013). For example, Thomson and colleagues (2013) found that, compared to government-supported private schools or government schools, independent private schools in 16 countries had higher levels of mathematics performance that was fully explained by school SES. The third set of contextual variables pertains to levels of inequality in different countries/regions. The first variable is the country/region that the study is conducted in (US or other countries). The mainstream SES scholarship is largely based on studies conducted in the US (e.g. Sirin, 2005), so it is important to ascertain if effect sizes vary among different countries/regions. Two other national-level moderators are GINI (degree of inequality in the distribution of family income in a country) and education GINI indices (associations between GDP per capita and individuals’ average years of education) (Ziesemer, 2016). A previous study showed that societal SES gradient in mathematics achievement positively moderated the relationship between parental expectations (a SES-related cultural capital variable comprising families’ familiarity with high-brow cultural elements or evaluative standards of educational institutions) and students’ mathematics achievement (Tan, 2015). Therefore, the two GINI indices, as measures of socioeconomic inequality, may moderate the association between school SES and students’ learning outcomes.

In addition to contextual variables, the present study examines how four methodological variables (number of school SES indicators, measurement level of student outcome data, type of studies, sources of school SES data) may moderate school SES effects for the following reasons. First, van Ewijk and Sleegers' (2010) meta-analysis found that student peer SES effect sizes were inversely related to the number of SES indicators used, so school SES effect sizes may vary with the number of SES indicators used. Second, previous studies suggest that school SES effects may vary with levels of measurement (school, student) of school SES and student outcome variables (Gorard, 2006). The third methodological variable pertains to study type such as published journal articles or unpublished doctoral theses; effect sizes may be larger in published articles (Polanin et al., 2016). Lastly, Sirin’s (2005) meta-analysis showed that school SES effects computed from school SES data obtained from a variety of sources (students, parents, school administrators, state education departments, records on percentage of students who are eligible for free/reduced-price lunch/schooling, or combination of these indicators) may differ.

“Omitted” variables

Some researchers attribute school SES effects to model misspecification (Lauen & Gaddis, 2013; Marks, 2015; Mickelson et al., 2013; Snijders & Bosker, 2012; van Ewijk & Sleegers, 2010); school SES effects are attenuated if relevant variables are included in analyses of school SES effects. One frequently omitted variable is the school’s percentage of ethnic minority students (Mickelson et al., 2013); e.g. percentage of students who are non-White in an US public school. These students may be confronted with schooling experiences (e.g. racial discrimination) that disadvantage their learning (Goddard et al., 2009; King, 2013).

Researchers also argue that school SES effects are inflated when students’ prior ability levels are not accounted for (Marks, 2015; van Ewijk & Sleegers, 2010). For example, Marks' (2015) longitudinal study involving achievement of Year 3, 5, and 7 students in Australia found that school SES effects were smaller when student- and school-level prior ability levels were controlled in the analysis. The present study examines effect sizes for relationships between school SES and (a) schools’ percentage of ethnic minority students and (b) students’ prior ability levels to address the issue of omitted variables.

School processes

Many scholars endeavour to identify processes mediating school SES effects (Reynolds et al., 2014). These processes include school leadership, school climate, teacher capacity, instructional programmes, school educational resources, and parental involvement. First, high-SES schools may be led by more effective school leaders; school leadership refers to “the exercise of influence on organizational members and diverse stakeholders toward the identification and achievement of the organization’s vision and goals” (Leithwood, 2021, p. 2). These school leaders may be more self-efficacious and experienced (Brouwer, 2018; Smith & Kearney, 2013), and they may emphasise student learning (Akay & Karadag, 2019) and establish school structures to enhance teaching and learning (Southern, 2018). Second, high-SES schools may have a more positive climate; school climate encapsulates the social atmosphere of the learning environment in which students learn (Block, 2011). These schools may have high levels of expectations and collective responsibility for academic expectations (Mitchell et al., 2016); positive relationships among school leaders, teachers, parents, and students (Forsyth et al., 2006; Goddard et al., 2009); and safe and conducive learning environments (Muijs et al., 2014). Third, high-SES schools may have higher levels of teacher capacity (i.e. the dispositions, expertise, and resources effective teachers need to achieve the school vision; Muijs et al., 2014). For example, teachers may have relevant pedagogical and content qualifications, teaching experience, and high levels of teaching self-efficacy (Akay & Karadag, 2019; Mitchell et al., 2016) and they benefit from school-supported professional development opportunities (Campoli, 2011; Fancera & Bliss, 2011). Fourth, high-SES schools may have well-designed instructional programmes (comprising coherent curriculum, student-centred pedagogy, and formative assessment; Reynolds et al., 2014) that effectively address students’ learning needs. Fifth, high-SES schools may have more effective educational resources (e.g. better student-teacher ratios, school infrastructure, equipment, curricular and educational materials) that enable quality learning for students (Hein et al., 2015; Mohammadpour & Ghafar, 2012; Tan & Hew, 2017).

The last set of processes is related to parental involvement (i.e. specific parental behaviours to enhance child development) pertaining to the family and school environments (Epstein, 2010; Tan, 2018, 2020). Park and colleagues (2017) made a further distinction between home and school involvement when they argued that proximal, home-based involvement is directed toward helping children learn (i.e. parental involvement yielding “private” benefits) whereas school-based involvement benefits school improvement more generally (i.e. parental involvement as a “public” good). Private aspects of parents’ involvement are exemplified by parental home involvement, trust in schools, and networking to obtain resources for their children (Ahart, 2014; Park et al., 2017). Public aspects of parental involvement include parents’ formal participation in school platforms such as parent-teacher association and parents helping out in school activities (Johnson & Hull, 2014; Park et al., 2017).

Researchers have focused on specific instead of all the six categories of processes, so we do not know which processes are more important than others in explaining school SES effects. In a related study, Holzberger and colleagues’ (2020) meta-analysis found that, compared to schools’ climate and material and personnel resources, processes such as schools’ academic press, classroom climate, instructional practices, and provision of out-of-school student activities had larger effect sizes for secondary school students’ mathematics and science learning. However, the study reported a substantially larger effect size for school SES (r = 0.30) when compared to those for school processes (0.03 ≤ r ≤ .18), so there is a need to identify other processes that are more highly correlated with school SES to better understand compositional effects. The study also did not examine associations between school SES and processes per se; instead, it computed effect sizes on associations between school processes (including school SES) and student learning outcomes. To address this knowledge gap, the present study will compare effect sizes for associations between school SES and different school processes.

Past meta-analyses and the present study

A literature review indicates that there are no meta-analyses examining school SES effects specifically. van Ewijk and Sleegers (2010) examined student peer SES measured at the class, cohort, and school level. Sirin (2005) examined effects of student, school, and neighbourhood SES using studies dated up to 2006, so there is a need to examine school SES effects using more recent studies; Kim and colleagues (Kim, 2019; Kim et al., 2019) only examined student SES effects. Additionally, Holzberger and colleagues’ (2020) meta-analysis compared effect sizes of school processes for students’ mathematics and science learning outcomes, but it is unclear if these processes are related to school SES.

The present study addresses these knowledge gaps and examines four research questions:

• What is the strength of relationships between school SES and student outcomes (achievement, attainment)?

• How are these relationships moderated by contextual and methodological variables?

• What is the strength of relationships between school SES and (a) schools’ percentage of ethnic minority students and (b) students’ prior ability?

• What school processes are related to school SES?

Method

Meta-analysis was employed to summarise empirical findings across studies. It enables effects in individual studies to be converted to a common metric and compared across studies. In conventional meta-analysis, researchers address the requirement of effect size independence by ignoring the dependency and treating effect sizes independently, averaging the dependent effect sizes within a study, selecting only one effect size for each study, or selecting one unit of analysis and averaging effect sizes within the unit. These different approaches result in biased effect sizes. The present study employed a three-level meta-analysis instead of conventional meta-analysis (van den Noortgate et al., 2013, 2014). A three-level meta-analysis incorporates different variance components attributed to three levels of the meta-analytic model: sampling variance of effect sizes at level 1 (participants), variance between effect sizes from the same study at level 2 (e.g. effect sizes corresponding to different school SES indicators for the same student outcome or effect sizes corresponding to the association between a SES indicator and multiple student outcomes reported in the same study), and between-study variance at level 3. All relevant effect size information in each study can be included in a three-level meta-analysis and the correlations among effect sizes appropriately accounted for within and between studies. In the present study, the mean number of effect sizes per primary study was 4.95 (480 effect sizes from 97 primary studies) and 77 of the 97 primary studies had more than one effect size, thereby justifying the use of the three-level meta-analysis.

Study identification

The five members of the project team searched for studies (journal articles, doctoral dissertations, conference proceedings, reports) examining associations between school SES and (a) student outcomes (achievement, attainment) and/or (b) school processes mediating school SES – student outcome relationships in K-12 schools dated 2000–2020 using six computer databases (Academic Search Complete, British Education Index, ERIC, ProQuest Dissertations & Theses, PsychArticles, PsychINFO). Each person focused on 4–5 years in the search process; one research team member set the date of publication of the studies to 2000–2003 in the computer search, another team member focused on 2004-2007, and so on. Search terms in abstracts included combinations of relevant keywords (using Boolean operators OR and AND) pertaining to.

• school SES (namely, school contextual effect OR school compositional effect OR school composite effect OR school aggregated effect OR SES OR socioeconomic status OR socio-economic status OR social class OR social status OR socioeconomic level OR socio-economic level OR social standing OR social position OR socioeconomic factor OR socio-economic factor) AND

• student outcomes (namely, academic achievement OR educational attainment OR school performance OR school success OR student performance OR academic outcome OR academic progress OR academic success OR scholastic achievement OR student achievement OR educational achievement OR amount of schooling OR educational level completed OR level of education completed OR school years completed OR years of school completed OR enrolment OR attendance outcomes OR school attendance OR dropout OR school progression OR grade retention OR graduation OR schooling OR enrol OR absent* OR absence OR drop-out OR grade repetition OR repeat grade OR complete grade OR grade completion).

This search returned 39,469 potential studies.

This search was complemented by two sets of manual searches of relevant studies that might be missed from the computer searches. The first was a search of articles published in generalist education journals (American Educational Research Journal, Asia Pacific Education Review, British Educational Research Journal, Cambridge Journal of Education, Educational Studies, Journal of Educational Research, Oxford Review of Education, Research Papers in Education, Teachers College Record), school effectiveness journals (Journal of Research on Educational Effectiveness, School Effectiveness and School Improvement), and an educational sociology journal (Sociology of Education). This search was performed by reviewing the title and abstract of articles listed in different volumes and issues of the journals’ websites. The second manual search was performed by scouring through reference lists of past reviews (including meta-analyses) of school SES effects to identify potentially relevant primary studies. Studies which may be relevant from an examination of their title and abstract are added to the pool of potential studies derived from the computer searches. The two manual searches returned another 53 potential studies. Therefore, the search returned a total of 39,522 potential studies.

Coding procedure

A formal coding scheme was used to record details from the studies (Table 1); it was developed deductively from a broad review of the extant literature and revised to accommodate categories emerging from the coding of the studies when necessary:

• Study identification details: author(s), publication year, title of study

• School SES indicators: home possessions, parental education, family income, %free/reduced-price lunch/schooling, Title 1/Non-Title 1 schools, other single indicator, mixture (≥ 2 indicators), no information

• Types of student outcomes: achievement, attainment, mixture (achievement, attainment)

• Students’ subject areas: languages, mathematics, science, social sciences, mixture (≥ 2 subjects)

• Processes associated with school SES: school leadership, school climate, teacher capacity, instructional programmes, school educational resources, parental involvement (private goods), parental involvement (public goods)

• School SES effect size data: associations between school SES and (a) student outcomes; (b) schools’ percentage of ethnic minority students; (c) students’ prior ability; and (d) school processes

• Student SES effect size data: association between student-level SES and outcomes

• Student grade levels: early childhood, primary (G1-6), lower secondary (G7-9), upper secondary (G10-12), early childhood-primary (K-G6), early-childhood education-lower secondary (K-G9), early childhood-upper secondary (K-G12), primary-lower secondary (G1-9), primary-upper secondary (G1-12), lower secondary-upper secondary (G7-12)

• School types: public, mixture (private, public)

• Countries: US, non-US, mixture (≥ 2 countries/regions)

• Country and education GINI

• Sources of school SES data: students, parents, school administrators, state education department, records on %free/reduced-price lunch/Title 1/Non-Title 1 classification, other source, mixed (≥ 2 sources)

• Number of school SES indicators used: 1-6, no information

• Measurement level of student outcome data: school, student

• Study type: journal article, thesis

  Table 1. Categories/Range of Key Variables Coded from Studies.

  Variables	Categories/Range	No of independent effect sizes (k)
  School SES indicators	Home possessions	1
  	Parental education	3
  	Family income	4
  	% free/reduced-price lunch/schooling Title 1/Non-Title 1 schools	106
  		15
  	Other single indicator	3
  	Mixed	106
  	No information	3
  Types of student outcomes	Achievement	232
  	Attainment Mixed	7
  		2
  Achievement subject areas	Languages	89
  	Mathematics	97
  	Science	18
  	Social sciences	2
  	Mixed	30
  Student grade levels	Early childhood education	18
  	Primary (G1-6)	73
  	Lower secondary (G7-9)	87
  	Upper secondary (G10-12)	34
  	Early childhood education-primary (K-G6)	5
  	Early childhood education-lower secondary (K-G9)	4
  	Early childhood education-upper secondary (K-G12)	2
  	Primary-lower secondary (G1-9)	11
  	Primary-upper secondary (G1-12)	3
  	Lower secondary-upper secondary (G7-12)	4
  School types	Public	80
  	Mixed (private and public)	161
  GINI	24.2-53.9	237
  	Not available	4
  Education GINI	0.06-0.36	237
  	Not available	4
  Countries	US	157
  	Non-US country/region	82
  	Mixed (≥ 2 countries)	2
  Sources of school SES data	Students	59
  	Parents	8
  	School administrators	5
  	State education department	30
  	Records on % free/reduced-price lunch Title 1/Non-Title 1 school classification	106
  		15
  	Other source	11
  	Mixed	7
  Number of school SES indicators used	1	132
  	2	38
  	3	48
  	5	4
  	6	2
  	No information	17
  Measurement level of student outcome data	School	145
  	Student	96
  Study type	Journal article	128
  	Thesis	113
  Schools’ percentage of ethnic minority students	-	19
  Students’ prior ability	-	24
  School processes	School leadership (effective leadership, enabling school structure/school design, experience, self-efficacy)	34
  	School climate (academic expectations/optimism, school security/safety, positive relationships, teachers’ trust/motivation/commitment/attendance/low mobility, collective responsibility, student discipline/low mobility, supportive learning environment)	74
  	Teacher capacity (teacher certification, experience, efficacy, professional development, collaboration, autonomy)	27
  	Instructional programmes (classroom teaching, programme coherence, instructional practices/feedback/curriculum/assessment/monitoring)	15
  	School educational resources (school resources/facilities, instructional expenditure, teacher-student ratio/class size, use of educational resources)	35
  	Parental involvement – private goods (parental networking, parental trust of schools and principals, family and community involvement)	5
  	Parental involvement – public goods (parent-teacher association membership and attendance, formal school involvement)	6

Coding was performed at the effect size level. More specifically, in studies with more than one effect size, different effect sizes were separately coded. Effect sizes from a study that included samples from different countries were separately coded. Country and education GINI were mean-centred in the meta-regression.

The first author coded substantive and methodological aspects for the 97 studies included in the meta-analysis. Two other team members discussed and jointly performed the same coding for these studies. Results showed high inter-rater reliabilities between the two sets of coding (Cohen, 1960; McHugh, 2012) for different variables coded (k = .99 for school SES indicators; k = .99 for school SES data sources; k = .99 for achievement subject areas; k = .89 for measurement level of student outcomes; k = .94 for school processes; k = .99 for student grade levels). Coding differences were discussed, and a consensus was reached to inform the final coding.

Study selection

The authors collaboratively developed a set of inclusion and exclusion criteria for deciding whether studies should be included in the meta-analysis. Studies were included if they (a) examined effects of school SES on student outcomes (achievement, attainment) and/or processes that mediated relationships between school SES and student outcomes; (b) sampled K-12 students/schools; (c) employed quantitative analytical approaches; (d) reported relevant effect sizes (e.g. correlations, Cohen’s d, means/standard deviations); (e) were dated 2000-2020; and (f) were written in English. Studies were excluded if they (a) examined student, family, classroom, or community SES; (b) examined only high- or low-SES schools (and not providing a comparison of the two); (c) examined student outcomes other than achievement or attainment; (d) sampled college/university students; (e) sampled special groups of students/schools (e.g. special education or religious schools); (f) employed qualitative or meta-analytic methodologies; and (g) were not written in English. Each project team member then reviewed abstracts of the studies for the four to five-year period that they were responsible for to decide if the studies should be included in the meta-analysis. Studies were included if they met all six inclusion criteria. They were excluded if they met any of the seven exclusion criteria. 38,905 studies were excluded in the process.

After excluding another 13 studies without full text and 71 duplicate studies, the first and two other project team members independently reviewed the full text of the remaining 533 studies. They then discussed whether these studies were eligible to be included based on the inclusion and exclusion criteria; differences in opinions were consensually resolved. This collaborative process excluded 436 studies. Therefore, a final pool of 97 studies were included in the present study (Figure 1). These studies examined school SES as an independent (n = 59) or a control variable (n = 38). Information coded from these studies is summarised in Supplementary Material 1. Examples of excluded studies are presented in Supplementary Material 2.

Figure 1. PRISMA Diagram.

GINI and Education GINI indices

Two sets of indices measuring the distribution of family income were included as moderators, namely GINI (https://www.cia.gov/the-world-factbook/field/gini-index-coefficient-distribution-of-family-income/country-comparison) and education GINI indices (Ziesemer, 2016). Both indices ranged from 0 to 1, with smaller values indicating societies that are more equal. GINI, being a broad measure of societal inequality, and education GINI, being a specific measure of educational inequality, are important measures of socioeconomic inequality that may moderate the association between school SES and students’ learning outcomes.

Calculating effect sizes

480 school SES effects were analysed using the metafor package in R (Viechtbauer, 2015). The effects comprised 387 Pearson’s correlation coefficients (r’s), 79 means/standard deviations (comparing high-SES with low-SES schools), and 14 Cohen’s d (comparing high-SES with low-SES schools). The magnitude of these effects was compared with that computed from another 85 effects (comprising 81 r’s and 4 means/standard deviations (comparing high-SES with low-SES students) pertaining to the association between student-level SES and learning outcomes. All effects were converted to Fisher’s z using Borenstein and colleagues’ (2009) formulas. The Fisher’s z’s were weighted by the inverse of their variance in the analysis. The weighting enabled effect sizes from larger-sample studies to be given more weight than those from smaller-sample studies. Effect sizes were then converted back to Pearson’s correlations in the reporting of results because the aim was to examine associations between school SES and different variables.

Four sets of meta-analyses were performed:

• Computing an overall effect size for the association between school SES and student outcomes (achievement, attainment) and comparing it to the effect size measuring the association between student-level SES and these outcomes

• 14 meta-analyses computing effect sizes for different SES school indicators and student outcomes (Table 2)

• Meta-regression involving contextual and methodological variables (Table 3) to ascertain sources of variation among effect sizes

• Nine meta-analyses of relationships between school SES and (a) schools’ percentage of ethnic minority students; (b) students’ prior ability; and (c) different school processes (Table 4).

  Table 2. Associations between School SES and Student Outcomes.

  	No of independent effect sizes (k)	No of studies	Effect sizes (r)				Variance levels
  			Mean	−95%CI	+95%CI	t	2	3
  Overall	241	74	.58	.50	.66	11.13	0.05	0.24
  School SES indicators
  Home possessions	1	1	.78	-	-	-	-	-
  Parental education	3	2	.36	-.79	.95	1.12	0.35	0.00
  Income	4	2	.68	.25	.89	4.59	0.01	0.06
  % FRL/school	106	47	.62	.53	.69	10.81	0.02	0.19
  Title 1/Non-Title 1 school	15	3	.43	-.03	.75	2.00	0.00	0.16
  Other single indicator	3	2	.59	-.40	.95	2.65	0.02	0.12
  Mixed	106	19	.51	.27	.69	3.93	0.04	0.34
  Student outcomes
  Achievement	232	73	.59	.51	.66	11.11	0.05	0.24
  Attainment	7	5	.39	.16	.57	4.14	0.00	0.04
  Mixed	2	2	.57	-.97	1.00	2.98	0.04	0.04
  Achievement subject area
  Languages	89	40	.58	.50	.66	10.83	0.03	0.12
  Mathematics	97	42	.58	.46	.68	7.78	0.05	0.26
  Science	18	10	.55	.41	.66	7.19	0.03	0.05
  Social sciences	2	2	.63	-.85	.99	4.66	0.02	0.02
  Mixed	30	18	.62	.43	.76	5.66	0.09	0.22

  Table 3. Meta-regression (k = 218).

  Variables	β (SE)
  Intercept	0.55(0.47)
  Contextual
  Students’ grade levels (reference category = Early childhood education, k = 4)
  Primary (k = 73)	0.00(0.18)
  Lower secondary (k = 81)	−0.00(0.20)
  Upper secondary (k = 32)	−0.11(0.24)
  Early childhood education-primary (k = 5)	−0.61(0.40)
  Early childhood education-lower secondary (k = 4)	−0.14(0.45)
  Early childhood education-upper secondary (k = 2)	−1.49(0.61)
  Primary-lower secondary (k = 11)	−0.21(0.32)
  Primary-upper secondary (k = 3)	0.35(0.44)
  Lower secondary-upper secondary (k = 3)	−0.53(0.39)
  Type of schools (reference category = Mixed, k = 140)
  Public (k = 78)	0.17(0.16)
  Education GINI (centred; k = 218)	0.03(0.56)
  GINI (centred; k = 218)	0.02(0.01)
  Country/Region (reference category = Non-US country/region, k = 62)
  US (k = 156)	0.19(0.14)
  Methodological
  Number of school SES indicators used (k = 218)	0.03(0.10)
  Measurement level of student outcome data (reference category = Student, k = 78)
  School (k = 140)	−0.20(0.14)
  Type of studies (reference category = Thesis, k = 112)
  Article (k = 106)	−0.22(0.14)
  Sources of school SES data (reference category = Mixed, k = 7)
  Students (k = 53)	0.66(0.39)
  Parents (k = 8)	0.14(0.41)
  School administrators (k = 5)	0.11(0.41)
  State education department (k = 14)	0.13(0.44)
  %FRL records (k = 106)	0.22(0.37)
  Title 1/Non-Title 1 school classification (k = 15)	−0.24(0.42)
  Other single sources (k = 10)	−0.14(0.41)

  Note. k = No of independent effect sizes.

  Table 4. Associations between School SES and (a) Schools’ Percentage of Ethnic Minority Students; (b) Students’ Prior Ability; and (b) School Processes.

  	No of independent effect sizes (k)	No of studies	Effect sizes(r)				Variance levels
  			Mean	−95%CI	+95%CI	t	2	3
  Schools’ percentage of ethnic minority students	19	15	-.43	-.65	-.15	−3.09	0.01	0.32
  Prior ability	24	15	.54	.35	.69	5.22	0.02	.17
  School processes
  School leadership	34	18	.12	.06	.18	3.88	0.00	0.01
  School climate	74	31	.31	.22	.40	6.34	0.02	0.06
  Teacher capacity	27	20	.27	.11	.41	3.42	0.02	0.11
  Instructional programmes	15	6	.20	-.11	.47	1.38	0.00	0.12
  School educational resources	35	16	.02	-.07	.11	0.47	0.00	0.03
  Private parent involvement	5	3	.24	.01	.45	2.87	0.01	0.02
  Public parent involvement	6	3	.53	-.37	.92	1.54	0.00	0.41

Restricted maximum likelihood estimation (default in the metafor package in R) was used in estimating the parameters in the meta-analysis.

Random effects models

The random effects model was employed because (a) the true effect size was expected to vary from study to study, depending on the contextual and methodological characteristics of the studies; (b) it does not require that effect sizes come from the same underlying population; (c) it enables the generalisation of results beyond the studies examined; and (d) results will be identical to those from a fixed effect model even if the observed variance in effect sizes across studies is due to random sampling errors (Borenstein et al., 2009).

Publication bias

A common concern in meta-analyses is the presence of publication bias in studies (Polanin et al., 2016). This means that studies with significant effects are more likely to be published than those with nonsignificant effects. The funnel plot of standard errors by effect sizes shows that effect sizes from individual meta-analyses were distributed on both sides of the mean effect size and that there were effect sizes corresponding to studies with different standard errors (Figure 2). Moderator analysis also indicates that standard errors were not a significant moderator for school SES effects (β = −0.34, p = .60, σ² (level 2) = 0.05, σ² (level 3) = 0.24), F (1, 239) = 0.27, p = .60). Therefore, there is no evidence of publication bias in the data.

Figure 2. Funnel Plot.

Results

School SES effect sizes

Results (Table 2) show that the mean effect size for the random effects model was r = .58. This was large in magnitude according to Kraft’s (2020) rules of thumb (r < .02 for small, .02 ≤ r < .10 for medium, r ≥ .10 for large effect sizes). This effect size means that school SES differences explained 33.64% of the variance in student learning outcomes. Most of the variance in effect sizes occurred within studies (level 2; 16.76%) and between studies (level 3; 83.23%); percentage of sampling variance of effect sizes at level 1 = 0.02%.

Mean effect sizes were large for school SES measured using family income (r = .68, k = 4), percentage of students enrolled in free/reduced-price lunch/schooling (r = .62, k = 106), and a variety of indicators to measure school SES (r = .51, k = 106). The effect size using home possessions was the largest at r = .78 but this finding was based on only one effect size. Mean effects for studies using Title 1/non-Title 1 school classification (r = .43, k = 15), parental education (r = .36, k = 3) or other single school SES indicator (r = .59, k = 3) were each based on a smaller number of effect sizes, so these findings must be interpreted with caution.

Next, mean effect sizes for achievement (r = .59, k = 232) and attainment (r = .39, k = 7) were large. The mean effect for studies using a combined measure of student achievement and attainment was r = .57 but this finding was based on only two effect sizes. The mean effect sizes for most subjects were large (languages, r = .58, k = 89; mathematics, r = .58, k = 97; science, r = .55, k = 18; combination of subjects, r = .62, k = 30). The mean effect for social sciences (r = .63) was based on only two effect sizes, so this finding must be interpreted with caution.

The analysis also computed the mean effect size for the association between student-level SES and learning outcomes. Results, computed from 85 effect sizes, show that r = .35. Therefore, the school SES effect size (r = .58) was 1.66 times that of the student SES effect size.

Meta-regression

Next, meta-regression was performed to examine potential sources of heterogeneity in effect sizes with regards to contextual and methodological variables. Results showed that apart from the comparison of school SES effect size between early childhood education-upper secondary and early childhood education (reference group), the regression coefficients for other variables were all very small (Table 3). In particular, the mean school SES effect size was smaller for studies involving early childhood-upper secondary schools (β = −1.49; k = 2) when compared to studies involving early childhoods (k = 4). It should be noted that findings for early childhood-lower secondary, early childhood-upper secondary, primary-upper secondary, and lower secondary-upper secondary school students were based on two to four effect sizes.

Schools’ percentage of ethnic minority students, students’ prior ability, and school processes

Results (Table 4) show that school SES was negatively associated with schools’ percentage of ethnic minority students (r = -.43, k = 19) and positively associated with students’ prior ability levels (r = .54, k = 24). Effects pertaining to four school processes, namely school leadership (r = .12, k = 34), school climate (r = .31, k = 74), teacher capacity (r = .27, k = 27), and private parental involvement (r = .24, k = 5) were larger than those for schools’ instructional programmes (r = .20, k = 15) or school educational resources (r = .02, k = 35). It should be noted that compared to those for most school processes, the findings for private and public parental involvement were based on a smaller set of effect sizes and hence they should be interpreted with caution.

Discussion

The present study reveals a large variation among the 97 studies that examine school SES. The majority of studies use the percentage of free/reduced-price lunch/schooling as a single indicator to measure school SES, examine school SES effects on student achievement (especially in languages and mathematics) as opposed to student attainment, involve primary and lower secondary students instead of other groups of students, and are contextualised in the US instead of other countries/regions. Results show that the mean effect size of school SES on student learning outcomes was substantial at r = .58. The mean effect size for achievement outcomes (r = .59) was larger than that for attainment outcomes (r = .39), indicating that school SES is more strongly associated with students’ learning quality than their behavioural outcomes. Mean effect sizes for achievement in languages, mathematics, science, and a combination of subjects were large. These results are largely consistent with those reported by van Ewijk and Sleegers (2010) indicating that student peer SES effect sizes did not vary across different subjects. They are also broadly consistent with Kim and colleagues’ (2019) meta-analytic findings that student SES effect sizes did not differ between literacy/reading and mathematics/science achievement in developing countries. However, it should be noted that van Ewijk and Sleegers' (2010) research did not focus on examining school SES while Kim and colleagues (2019) examined student SES. The large effect sizes associated with student achievement in different subject areas (except social sciences) underscore school processes influencing teaching-and-learning in high-SES schools, thereby affirming the need to unravel these processes.

School SES is less strongly associated with school processes than with schools’ percentage of ethnic minority students or students’ prior ability

The present study shows that school SES was less strongly associated with school processes than with schools’ percentage of ethnic minority students or students’ prior ability. School processes which were positively associated with school SES were school leadership, school climate, teacher capacity, and parental involvement benefiting student learning directly. Among these processes, school SES had the largest association with school climate. However, the effect size (r = .31) was much smaller than that for associations between school SES and schools’ percentage of ethnic minority students (r = -.65) or between school SES and students’ prior ability (r = .54). The study also finds that school SES was not associated with other school processes such as instructional programmes, schools’ educational resources, and parental involvement benefiting students indirectly.

The significant association between school SES and schools’ percentage of ethnic minority students, and the association between school SES and various school processes (i.e. school leadership, school climate, teacher capacity) suggests that ethnic minority students may be attending less effective schools (indicative of school segregation) measured in terms of the quality of school leadership, school climate, and teacher capacity (King, 2013). The association between school SES and students’ prior ability means that school SES effect sizes may be smaller if students’ prior ability is controlled. However, there were only two studies that provided effect sizes for students with a specific level of prior ability (i.e. high or low-ability students), so it was not possible to compare school SES effect sizes between high- and low-ability students. The (negative) association between school SES and schools’ percentage of ethnic minority students, and the (positive) association between school SES and students’ prior ability indicate that it may be instructive to use an intersectionality perspective to examine school SES effects (Fahle et al., 2020). The findings also question the validity of examining school SES effects if these student variables are not included in the research design of primary studies. Importantly, it merits reiterating that the findings are associations, so they do not mean that all ethnic minority students or students with lower levels of prior ability attend low-SES schools or that the lower levels of student learning in low-SES schools are due to schools’ percentage of ethnic minority students or students’ prior ability levels. Rather, they indicate that school SES is associated with these student variables and that there are many variables contributing to student learning outcomes.

Measuring school SES using percentage of students on free/reduced-price lunch

Results show that most studies included in the meta-analysis used the percentage of students on free/reduced price lunch as a school SES indicator. Some researchers are concerned with reliability issues using this indicator (Sirin, 2005; van Ewijk & Sleegers, 2010); for example, the process to determine eligibility for such a scheme is prone to errors and that not all financially needy students will apply for this scheme (Sirin, 2005). Additionally, a high percentage of students on free/reduced lunch for a school does not imply that many students in the school are poor (Hill & Jenkins, 2001). Future school SES studies can explore the use of other indicators besides the percentage of free/reduced lunch students to measure school SES.

School processes

Results show that school SES was more associated with school leadership, school climate, teacher capacity, and parental involvement than with school instructional programmes or school educational resources, thereby elucidating the processes that may contribute to the more positive student learning outcomes in higher-SES schools.

Higher-SES schools have better leadership, teachers, and school climate

There is meta-analytic evidence that school leadership (including their role in building a shared vision and setting high expectations for student learning) is associated with organisational, teacher, and student outcomes (Tan et al., 2022), so higher-SES schools may be led by more effective school leaders galvanising the school community to establish a positive climate, and these two processes then contribute to student performance. Effective school leadership strategies need to be implemented via teachers to influence student learning (Hendriks & Scheerens, 2013). The present study underscores two teacher-related qualities associated with high-SES schools, namely school climate (e.g. teacher trust, motivation, commitment, responsibility) and teachers’ demonstrated capacity (e.g. teacher qualifications, experience, self-efficacy, professional development, collaboration, autonomy). The associations of school SES with school leadership and teacher quality suggest that higher-SES students may have better learning outcomes because competent school leaders and teachers cultivate a positive school climate (Berkowitz et al., 2017).

School SES is less associated with school resources or instructional programme

The present study shows that school SES was less associated with school resources. Some researchers argue that school resources have negligible or even negative effects on student learning outcomes (Häkkinen et al., 2003). Therefore, having more school resources are necessary but insufficient for student success. Schools need to have other conditions such as a positive school climate and competent teachers.

Relatedly, the present study also shows that school SES was less associated with instructional programmes than with other school variables such as school leadership, school climate, and teacher capacity. There are three possible reasons for this finding. First, the finding is consistent with the Heyneman-Loxley Effect promulgating that schools are less important in influencing student outcomes when countries become economically developed (Heyneman, 2015). Indeed, it can be argued that international sharing of best practices has converged in most instructional programmes in many schools (including low-SES schools) having elements of student-centeredness in their design and employing different assessment approaches to monitor student learning (Schweisfurth, 2015). Therefore, high-SES school students do not necessarily enjoy better instructional programmes than their peers in low-SES schools. Second, educational effectiveness research (Reynolds et al., 2014) suggests that schools may need other processes to support the implementation of instructional programmes. The third reason why school SES was less associated with instructional programmes is the diversity of data sources for instructional programmes in the studies analysed. Specifically, teachers, students, and school officials provided data for school instructional programmes (seven, five, and three effect sizes respectively); these data were used to measure the association between school SES and instructional programmes in the meta-analysis. However, teachers may overstate the implementation of instructional programmes whereas students may be more critical in their perceptions of instructional programmes, so the overall association between school SES and instructional programmes may be nullified.

The discussion so far suggests that students may have better learning outcomes in higher-SES schools because they benefit from access to quality human resources (competent, motivated school leadership and teachers) instead of more functional resources (instructional offerings, school educational resources). School leaders and teachers then cultivate a positive school climate (an intangible process) supporting student learning. A comparison of effects between school SES and processes in the present study shows that school climate had the largest effect size compared to those for school leadership and teacher capacity. Relatedly, Holzberger and colleagues’ (2020) meta-analysis showed that processes such as school and classroom climate (.08 ≤ r ≤ .13) were more strongly associated with students’ science and mathematics outcomes than processes such as material and personnel resources (.03 ≤ r ≤ .08), thereby suggesting that students in higher-SES schools benefit from a positive school climate (made possible by quality school leadership and teachers).

Ecological fallacy

The present study shows that school SES effect size was 1.66 times that of the student SES effect size. This finding cautions researchers against committing the ecological fallacy when erroneous individual-level inferences are made from group aggregated data (Robinson, 1950). For example, a large positive association between the mean school SES level (i.e. school SES) and performance level of students in a school does not necessarily mean that student SES is strongly associated with student achievement (Marks, 2015; Robinson, 1950; Sirin, 2005; Snijders & Bosker, 2012; van Ewijk & Sleegers, 2010). The study also shows that school SES was associated with schools’ percentage of ethnic minority students, students’ prior ability, and specific processes (school leadership, school climate, teacher capacity). These different findings indicate that school and student SES are different constructs and that school SES effects may be explained by non-student SES factors (e.g. school processes). There is no clear evidence as to whether school SES effects are susceptible to the ecological fallacy.

Limitations

As with all studies, the present meta-analysis should be read with some limitations in mind. First, it covers studies dated 2000-2020, so results reported are more relevant for schools and student learning during this time period. Additionally, it only includes studies written in English. Future meta-analyses can include earlier studies and studies published in other languages and compare results with those reported in this study. Second, the present study ascertains associations between school SES and (a) schools’ composition of ethnic minority students and (b) students’ prior ability but it is unable to test if school SES effects will be attenuated when these student variables are controlled for due to insufficient data. Third, the meta-analysis ascertains associations between school SES and processes but is unable to determine if school processes mediate the relationship between school SES and student outcomes. Future studies can examine mediation effects of these school processes in experimental research. Lastly, the analysis has eschewed the usage of statistical tests of significance, including p values, because the primary studies included are not conducted at the population level, there is no evidence of random sampling in these studies, and information on missing values is incomplete at best. Researchers conducting future studies on school SES should consider more robust means of sampling (e.g. random sampling at the population level) and report the extent to which data are missing on key variables.

Conclusion

The present study is contextualised in the debate on the influence of school composition on student learning outcomes. Being the first meta-analysis to focus on school SES, it examines 480 effect sizes from 97 studies to provide insights on associations between school SES and (a) student learning outcomes; (b) schools’ percentage of ethnic minority students and students’ prior ability; and (c) school processes in K-12 schools. The study also clarifies the moderating influence of various contextual and methodological variables.

The study makes three theoretical contributions. First, it clarifies the magnitude and scope of school SES effects: the mean school SES effect size was large at r = .58 and school SES effects were larger for achievement (especially in language, mathematics, science subjects) than attainment outcomes. School SES effects influenced students in both private and public K-12 schools and were more pronounced in socioeconomically unequal countries/regions. Second, results enhance our understanding of school SES. Specifically, school SES was more strongly associated with schools’ percentage of ethnic minority students and students’ prior ability than with school processes. Results also show that the effect size for school SES was larger than that for student SES and that there was an over-reliance on the use of percentage of students on free/reduced-price lunch to measure school SES. Third, the study elucidates plausible reasons to explain why higher-SES school students outperform their peers from lower-SES schools by identifying processes that are associated with school SES. These processes comprise intangible school aspects (school leadership and climate, teacher capacity) and parental involvement directly benefiting student learning. In contrast, functional school resources (school instructional programmes, educational resources) and parental involvement benefiting school improvement were not associated with school SES. These results suggest that it may be beneficial for school leaders to develop their schools’ “software” (i.e. developing leadership and teacher capacity and building a positive school climate) in addition to “hardware” (i.e. functional resources). Put together, these contributions from the present study provide evidence of a large school SES effect similar to that first documented in the Coleman Report. Additionally, it advances our understanding of school SES effects beyond the Coleman Report by unravelling specific school processes that may explain the influence of school SES on students’ learning outcomes.

The present study has important implications for education policies related to school segregation (Reardon & Owens, 2014) and voucher schools (Epple et al., 2017). The finding from the study that there were more ethnic minority students in low-SES schools means that economic and racial segregation may coexist in schools. This school segregation has sometimes been attributed to education policies such as the school voucher scheme that has supposedly being designed to foster school competition (Epple et al., 2017). The findings that there were some processes that characterised high-SES schools (i.e. more effective school leadership, climate, teacher capacity) provide an opportunity for low-SES schools to prioritise their limited resources to improve these processes and become more competitive in the educational marketplace. The improved school processes may mitigate the inequality in teaching and learning resources arising from school segregation, thereby enriching learning opportunities for students in low-SES schools.

Acknowledgement

The authors acknowledge the research assistance of Kai Hong Lo.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The study is supported by The University of Hong Kong (HKU) Faculty Research Fund. Publication of the study is made possible by support from the 2022 HKU Research Output Prize.