Teachers’ perceptions of students’ school performance: the impact of classroom composition. Evidence from a survey experiment

ABSTRACT

This study uses a randomized vignette experiment to explore the impact of classroom composition on students’ teacher-perceived school performance. It hypothesizes that in classrooms with a high share of disadvantaged students, teachers rate the school performance of both disadvantaged and non-disadvantaged students lower than in classrooms with a low share of disadvantaged students. The experimental design disentangles two mechanisms behind compositional effects: ethnicity and social-status. Findings indicate the more disadvantaged perceptions of Roma and non-Roma students in classrooms with a high (rather than a low) share of Roma students. No similar findings were found concerning the share of poor students.

KEYWORDS:

• Compositional effects
• vignette experiment
• stereotypes
• teachers’ perceptions

1. Introduction

Perceptions drive stereotypes, and stereotypes can influence perceptions. Stereotypes—perceptions of individuals based on group belonging (Fiske, 1998; McCauley et al., 1980; Hinton, 2017)—speed up information processing. People often rely on stereotypes about a particular social group to judge strangers who belong to that group group (Guryan & Charles, 2013; Macrae & Bodenhausen, 2000). Stereotypes can reflect accurate perceptions of real group differences, but they can also be biased and unrelated to actual group differences (Hilton and von Hippel 1996; Riegle-Crumb and Humphries 2012).

Research in teachers’ perceptions of students has shown that teachers’ grading practices and track recommendations are biased against low-status and minority students: teachers assign lower grades to minority students (Sprietsma 2013) and low-status students (Hanna and Linden 2012), even if their academic achievements are similar to their non-minority and high-status peers. Randomized experiments have also demonstrated that teachers recommend the vocational secondary track over the academic secondary track more often to minority students, even when these students have the same academic achievement as their non-minority peers (Van Ewijk 2011; Wenz and Hoenig 2020).

Previous research has examined variations in teachers’ perceptions of students’ academic performance based on factors, such as students’ actual performance (Oates 2003), gender (Bonefeld, Kleen, and Glock 2022), socioeconomic status, and personality attributes (Rubie-Davies 2010). Teachers tend to rate students from low socioeconomic backgrounds as less intelligent than their classmates with high socioeconomic backgrounds (Alvidrez and Weinstein 1999). Furthermore, teachers held elevated expectations for students they perceived as possessing self-confidence and positive work habits. (Timmermans, de Boer, and van der Werf 2016).

Classroom composition may also influence teachers’ perceptions of students’ school performance (Glock, Kovacs, and Cate 2019; McKown and Weinstein 2008). As recent network research has highlighted, categorization processes lead to the attribution of similar attitudes to individuals who are connected. Thus, social ties (like being classmates) provide informational cues regarding underlying qualitative similarities (Halgin, Borgatti, and Huang 2020) that may lead to a similar perception of those who share the same classroom context.

For various reasons, a high share of disadvantaged students might suggest a low overall classroom quality to teachers. If teachers have substantial experience that students in the classroom context with a high share of disadvantaged students tend to perform worse on average than students in a classroom context with a low share of disadvantaged students, this experience might be reflected in their perceptions. Furthermore, if teachers hold biased perceptions of disadvantaged students, they might extend these toward other students who share the same classroom context with disadvantaged students. The share of disadvantaged students may reinforce this tendency.

This paper addresses how classroom composition affects teachers’ perceptions of students’ school performance (hereafter referred to as compositional effects) and makes two specific contributions. First, the paper introduces a distinction between direct and indirect compositional effects by examining how classroom composition (the share of disadvantaged students in the classroom) affects teachers’ perceptions of disadvantaged students (direct compositional effect) and non-disadvantaged students (indirect compositional effect). Second, the paper disentangles two mechanisms that can drive compositional effects and differentiates between social-status- and ethnicity-based compositional effects. By doing so, the paper focuses on one of the most disadvantaged ethnic minority groups in Central and Eastern Europe, the Roma,¹ and builds on the fact that, in Hungarian schools, the high share of Roma students is intertwined with the high share of poor students.

A vignette experiment was conducted involving primary school teachers (N = 562) in a large number of Hungarian primary schools (N = 332, approximately 10% of all primary schools). It was hypothesized that in a classroom where the share of disadvantaged students (Roma/poor) is high, teachers would perceive students’ school performance to be lower than in a classroom where the share of disadvantaged students is low².

The findings indicate that in classrooms with a high share of Roma students, teachers rated the school performance of both Roma and non-Roma students lower than in classrooms where the share of Roma students was low, indicating ethnicity-based direct compositional effects (targeting Roma students) and indirect compositional effects (targeting non-Roma students). This is consistent with the hypothesis. However, teachers’ perceptions of poor and non-poor students’ school performance did not differ between classrooms with a high share of economically disadvantaged (poor) students and classrooms with a low share of economically disadvantaged (poor) students. Therefore, contrary to the hypothesis, the results do not support social-status-based direct compositional effects (targeting poor students) and indirect compositional effects (targeting non-poor students). When comparing the ethnicity-based (Roma) and social-status-based (poor) compositional effects, statistically significant differences were observed in the indirect (but not direct) compositional effects. This suggests that the indirect compositional effects are mostly driven by Roma ethnicity rather than by the correlated economic disadvantage.

The identification of lower perceptions of non-Roma students in classrooms with a high share of Roma students is a novel finding which may not have been established in previous studies. As teachers’ perceptions influence their stereotypes (and vice versa), the findings may suggest the emergence of stereotypes (and potentially even discrimination) against non-Roma students in classrooms with a high share of Roma students. For instance, non-Roma students in schools with a high share of Roma students might receive lower grades, have less favorable school recommendations, and have a reduced likelihood of admission to higher-quality schools. In sum, the lower perceived performance of non-Roma students in classrooms with a high share of Roma students is an important topic for future research, especially as uncovering teachers’ stereotypes may be an effective intervention to reduce discrimination (Alesina et al. 2018).

This paper distinguishes itself from previous experimental research on teachers’ perceptions of students’ school performance in several ways. It utilizes a larger sample of 562 teachers; previous studies typically have used smaller samples, ranging from around 40 teachers (Costa, Pirchio, and Glock 2022; Glock and Krolak-Schwerdt 2013) to 200 teachers (Study 2 in Glock, Kovacs, and Cate 2019). Moreover, the sample analyzed in this paper was recruited nationwide, with invitations sent to all Hungarian primary schools; previous studies have recruited teachers through personal contacts (Bonefeld, Kleen, and Glock 2022; Costa, Pirchio, and Glock 2022; Glock and Krolak-Schwerdt 2013; Glock, Kovacs, and Cate 2019). Lastly, this paper examines teachers' perceptions of Roma minority students in Hungary, in contrast to previous studies that have examined teachers' perceptions of students with immigrant backgrounds, such as Turkish students in Germany (Bonefeld, Kleen, and Glock 2022; Glock and Krolak-Schwerdt 2013; Glock, Kovacs, and Cate 2019) or Arab students in Italy (Costa, Pirchio, and Glock 2022).

2. Previous literature on teacher perceptions of students’ performance

This section provides a literature review, identifying four limitations in prior studies that will be subsequently addressed in the current study. First, previous studies have primarily focused on examining the accuracy of teachers’ perceptions of students’ performance (Hoge and Coladarci 1989). Studies have typically involved teachers who already had pre-existing contact with the students they were rating (Alvidrez and Weinstein 1999; Oates 2003; Timmermans et al. 2018). Some studies have indicated that teachers’ perceptions of achievement gaps among students may align with the actual performance gap (Ferguson 2003; Jussim and Harber 2005)³. However, teachers’ initial perceptions about the students they know in person could influence the observed differences in students’ performance, making teachers’ perceptions endogenous. Therefore, new research should investigate teachers’ perceptions of hypothetical students’ performance to ensure that teachers’ pre-existing knowledge about specific students does not influence their perceptions.

Second, only a few studies have investigated the influence of compositional effects on teachers’ perceptions of students (Glock, Kovacs, and Cate 2019; McKown and Weinstein 2008). These studies support the idea that composition-specific information—particularly the classroom’s ethnic composition—can affect teachers’ perceptions. For instance, McKown and Weinstein (2008) found that in classrooms with high ethnic diversity, the ethnic gap in teachers’ expectations for student achievement was larger than in classrooms with low ethnic diversity. Moreover, Glock, Kovacs, and Cate (2019) found that German teachers who read a vignette describing an ethnically mixed classroom context exhibited more negative implicit attitudes (assessed by an Implicit Association Test) toward Turkish minority students than teachers who received a vignette describing an ethnically homogeneous classroom. The limited number of papers focusing on the compositional effects in students’ teacher-perceived school performance calls for more research.

Third, even in experimental research, it can be challenging to disentangle the differences in teachers’ perceptions attributed to students’ ethnicity and those attributed to students’ social status (Janky et al. 2014). Considering the frequent correspondence between students’ ethnicity and social status, it remains uncertain whether the ethnic differences observed in teachers’ perceptions are attributable to students’ ethnicity or to their associated social status. Therefore, new research should disentangle ethnicity and social status as drivers of teachers’ perceptions.

Fourth, previous research on compositional effects has overlooked the potential existence of indirect compositional effects. While the share of disadvantaged students can directly impact the perception of disadvantaged students (direct composition-specific effect), it may also have an indirect effect on non-disadvantaged students (indirect composition-specific effect). This calls for new research aimed at providing a more comprehensive understanding of indirect compositional effects.

3. Institutional background

This section provides institutional background information in three areas: primary education (1), the Roma ethnic minority (2), and schools’ ethnic composition in Hungary (3).

3.1. Primary education in Hungary

Hungarian primary education offers untracked, undivided education, encompassing general and lower-secondary education (ISCED 1 and ISCED 2 levels), comparable to elementary and middle school in the United States. The compulsory primary education lasts for eight years (from age 6–14).

Though primary education is undivided, the general (1–4 grades) and lower-secondary (5–8 grades) parts of the education are often differentiated. Teachers at the general and lower secondary levels might have different qualifications. Classes at general and lower-secondary levels are sometimes held in different buildings. In most cases, students have the same homeroom teacher in the first four years, and then another homeroom teacher for the next four years (grades 5–8). The homeroom teacher in the lower-secondary part of primary education is responsible for assisting students in the transition to secondary education. Thus, feedback and contact with the homeroom teacher in grades 5–8 could have a larger impact on students’ later educational decisions than their homeroom teacher from the first four grades.

3.2. Roma ethnic minority

The Roma are spread throughout Europe. There is a particularly large Roma community in Hungary, ranking among the highest in Europe after Bulgaria, Slovakia, Romania, and Serbia, relative to the size of the population.

The Roma are the largest and poorest ethnic minority in Hungary. Most Hungarian Roma speak Hungarian as their mother tongue, but most non-Roma Hungarians can recognize them by their appearance (Kemény and Janky 2006). The Roma minority makes up approximately 8 percent of the Hungarian population (Morauszki and Papp Z 2015; Pénzes, Tátrai, and Pásztor 2018). This proportion is higher among primary school-aged students. According to estimates provided by school principals in 2021, approximately 21% of students in Hungarian primary schools are Roma.

Roma people face significant economic exclusion (Kertesi and Kézdi 2011a) and residential segregation (Kemény and Janky 2006). The majority of Roma households experience poverty; in the 2010s, approximately 80% of Roma households were classified as poor, compared to around 25% of non-Roma households during the same period (Bernát 2018). Roma people are also the target of ethnic prejudice (Simonovits, Kézdi, and Kardos 2018) and discrimination (Buda et al. 2023), even relative to economically disadvantaged (poor) people (Janky et al. 2014). Illustrating the extent of entrenched prejudiced attitudes toward Roma people, a nationally representative survey found that 60 percent of Hungarians agree that ‘criminality is in the blood’ of the Roma (Bernát et al. 2012).

Roma students demonstrate lower academic performance than non-Roma students, reflected in lower grades and test scores. The ethnic test-score gap between Roma and non-Roma primary school students is comparable to the disparity observed between white and African-American students in the United States during the 1980s (Kertesi and Kézdi 2011b). Roma students also frequently encounter school segregation (Kertesi and Kézdi 2012) and suffer substantial discrimination, even from their own non-Roma classmates (Elwert, Keller, and Kotsadam 2023). Roma students attend lower-quality education and select school tracks with lower earning potential (Hajdu, Kertesi, and Kézdi 2019; Kertesi and Kézdi 2010, 2011a, 2011b). A significant proportion of Roma students do not complete secondary education, and only a small number pursue tertiary education (Kertesi and Kézdi 2016), even though Roma students hold similar attitudes to their non-Roma peers regarding school achievement, effort, and school behavior (Keller 2020).

3.3. The ethnic composition of schools in Hungary

The school composition influences students’ educational achievement and attainment in Hungary (Hermann and Kisfalusi 2023). Administrative data provide further insights into composition-specific differences among Hungarian primary schools. According to school principals’ estimates, a strong correlation exists between the shares of Roma students and poor students within a school.

The left panel of Figure 1 shows that in schools with a high share of poor students (over 80%), almost all students are Roma (median: 96%)⁴. In schools with a low share of poor students (less than 20%), only a few Roma students attend (median: 3%). The right panel of Figure 1 shows that in schools with a high share of Roma students (over 80%), the majority of students are poor (median: 60%)⁵, whereas in schools with a low share of Roma students (less than 20%), only a small percentage of students are poor (median: 9). In summary, Figure 1 indicates that schools with a significant proportion of Roma students also tend to have a high proportion of economically disadvantaged students, and vice versa. As a result, it is necessary to disentangle the ethnicity- and status-related determinants of teachers’ perceptions.

Figure 1. Violin plot indicating the connection between the share of poor students and Roma students within the schools:school principals’ estimates.

Note: Data: NABC 2021, 8th grader sample, own calculation.

4. The experiment

The study underwent ethical review and received approval from the Institutional Review Board (IRB) at HUN-REN Center for Social Sciences. Supplementary materials, data, and all analytical scripts are archived on the study's project page at the Open Science Framework (OSF): https://osf.io/6z7c4/.

4.1. The survey

A vignette survey experiment was conducted among Hungarian primary school teachers from November 2021 to May 2022. An invitation letter was sent to all Hungarian primary schools through the school’s central e-mail address. The invitation letter asked the school principal to forward the letter to homeroom teachers in grades 5 to 8. The letter informed teachers of the purpose of the study: to explore teachers’ grading practices. Among those teachers who participated, five randomly selected teachers received a gift voucher worth 25 EUR; teachers were informed about the lottery in the invitation letter.

Teachers participated in the survey voluntarily and accessed the questionnaire online. The survey included one vignette and two related questions about perceptions of students’ school performance. The teachers were also asked to complete a short background questionnaire regarding their gender, age, work experience, the grade level at which they serve as homeroom teachers, and the subject they teach. On average, it took approximately three minutes for the median teacher to complete the survey.

4.2. The sample

The sample comprises 562 teachers from 322 schools across 273 settlements. The 322 schools representabout 10% of all Hungarian primary schools. Among the schools in the sample, 61% had only one participating teacher, 25% had two participating teachers, and the remaining 14% had three or more participating teachers. The maximum number of teachers participating from the same school was 11.

The sample of 322 schools is not representative of Hungarian primary schools. Table 1 compares the characteristics of the sample schools to all Hungarian primary schools, using register data from 2021. The comparison highlights that schools in the sample are 16 percentage points more likely to be village schools and small-sized schools (by 6 percentage points), with a higher percentage of poor (by 3 percentage points) and Roma students (by 7 percentage points) than schools not included in the sample. Consequently, in the sample schools, the social status index of 8th-grade students is 0.32 standard deviation units lower than in schools not in the sample. Moreover, 8th-grade students in the sample schools exhibited lower math (by 0.19 standard deviation units) and reading performance (by 0.21 standard deviation units) compared to schools not included in the sample.

Table 1. The characteristics of sample schools, relative to all Hungarian primary schools.

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)	(13)	(14)
	Rich students^(a)	Poor students^(b)	Roma students	Math test score, 8^th grade	Reading test score, 8^th grade	SES	Village	Town	County seed	Budapest (capital city)	Very small school^(c)	Small school	Medium-sized school	Large school
Mean in school, not in the sample	0.08	0.18	0.20	0.00	0.00	0.01	0.40	0.33	0.15	0.13	0.02	0.31	0.33	0.35
Mean difference	−0.01	0.03*	0.07**	−0.19**	−0.21**	−0.32**	0.16**	−0.07**	−0.05**	−0.03+	−0.00	0.06*	0.02	−0.08**
	(0.01)	(0.01)	(0.02)	(0.06)	(0.06)	(0.08)	(0.03)	(0.03)	(0.02)	(0.02)	(0.01)	(0.03)	(0.03)	(0.03)
Standardized mean difference	−0.07	0.16	0.25	−0.19	−0.21	−0.33	0.33	−0.16	−0.16	−0.10	−0.02	0.12	0.05	−0.17
N of schools	2,530	2,531	2,522	2,702	2,702	1,676	2,812	2,812	2,812	2,812	2,812	2,812	2,812	2,812

Robust standard errors, clustered at the school level, in parentheses.

**p < 0.01, *p < 0.05, + p < 0.1

Own calculation using data from the Hungarian National Assessment of Basic Competencies (NABC); the cohort of 8^th graders in 2021.

The NABC assesses students’ mathematics and reading comprehension achievement with a PISA-like test. Mathematics and reading comprehension achievement are assessed by two 90-minute long in-class tests supervised by the classroom teachers and marked centrally. The NABC has a take-home supplementary questionnaire regarding students’ social backgrounds and home environments. The NABC also has a school questionnaire for the school principal, with questions about the school context.

The source of data in Columns 1-3 and 7–14 is the NABC school questionnaire, answered by the school principal.

The source of data in Columns 4 and 5 is students’ individual test scores, aggregated to the school level.

The source of data in Column 6 is individual students’ background data from the NABC background questionnaire, aggregated to the school level.

^(a) The translated version of the original Hungarian question asked principals to rate the share of students in the school who had ‘above average material circumstances’ (átlag feletti anyagi körülmények között élő).

^(b) The translated version of the original Hungarian question asked principals to rate the share of students in the school who ‘are living under extraordinarily bad material circumstances’ (nagyon rossz anyagi körülmények között élő).

^(c) A very small school is a school with a maximum of five teachers (as defined by the NABC).

Standardized mean differences are expressed in standard deviation units.

Panel B in Appendix Table A1 shows that none of the listed school-level characteristics in Table 1 consistently influence teachers’ perceptions of students’ school performance across different specifications⁶. Therefore, based on teachers’ observed characteristics, the non-representativeness of the sample should not undermine the conclusions that can be drawn from this sample⁷.

Using census data (the last publically available census is from 2011) further confirms that schools in the sample are situated in small settlements with an above-average share of the Roma population. While 27% of Hungarian primary schools are located in settlements with less than 2,000 inhabitants, this shareis 34% in the sample. Differences are less pronounced but still present regarding the schools’ share of Roma students. Specifically, 4% of Hungarian primary schools are in settlements where the share of the Roma population is more than 20%, compared to 6% in the sample. Furthermore, 21% of Hungarian primary schools are in settlements where the share of the Roma population is more than 5%; this percentage is 25% in the sample.

The primary school teachers in the sample (N = 562) were homeroom teachers at grades 5 (23%), 6 (19%), 7 (28%), and 8 (30%), teaching students from ages 10–14 at ISCED 2 level. The 562 teachers represent about 8% of all the teachers from the sample schools. Teachers in the sample are not representative of Hungarian primary school teachers. Women are underrepresented; the sample contains 79% female and 21% male participants, while nationally ,the proportion of primary school teachers that are women is around 85% (Varga 2021). The sample also contains a greater proportion of experienced middle-aged teachers than the national average; the mean age among teachers in the sample is 48 years (SD = 8.6 years). In 2020, 30% of Hungarian primary school teachers were between 40 and 49 years old; in the sample, it is 37%. In contrast, the shares of young and old teachers are both 3 percentage points lower in the sample than in the general population (Varga 2021). Teachers in the sample are more likely to teach Roma students and poor students. Based on school principals’ reports in 2021, 4% of Hungarian primary school teachers work in schools with more than 80% Roma students; this figure is 8% in the sample. Furthermore, 1% of Hungarian primary school teachers work in schools where the share of poor students exceeds 80%, compared to 2% in the sample. Throughout the analysis, these differences are accounted for by controlling for the participating teachers’ background characteristics and incorporating school-level fixed effects.

4.3. The experimental design

The experiment had a between-subject design. Every teacher received one vignette, and different teachers might receive different vignettes. This design allows for comparisons between teachers. Teachers were tasked with evaluating the school performance of hypothetical disadvantaged and non-disadvantaged students.

To disentangle the differences between social-status-based and ethnicity-based compositional effects, the vignettes labeled disadvantaged students as either ‘poor’ (representing the disadvantaged group in general) or ‘Roma’ (representing the disadvantaged ethnic minority group). The depiction of the classroom context also varied, describing a high or low share of disadvantaged students⁸. In the ‘poor’ vignette, the classroom context described a high or low share of poor students; in the ‘Roma’ vignette, the classroom context depicted a high or low share of Roma students. This meant having four vignettes in the survey corresponding to the four possible combinations of disadvantaged students’ labels (Roma or poor) and the share of disadvantaged students (high or low). Each teacher received one random vignette out of the four possible options.

Teachers were asked to answer two consecutive questions regarding the same vignette: one concerning the school performance of disadvantaged students (Roma/poor) and another concerning the school performance of non-disadvantaged students (non-Roma/non-poor). This design element made it feasible to distinguish between the direct and indirect compositional effects. Sample vignettes are shown in Figure 2⁹.

Figure 2. Sample vignettes to rate (non)-poor and (non)-Roma students.

Note: Words that appear in the [squared brackets] are the alternative word used in other vignettes.

The experiment was specifically designed to address the four outlined limitations identified in previous studies. Teachers were asked to rate the school performance of fictitious students based solely on the information provided in the vignettes. This design element ensured that teachers’ prior experiences with specific students would not influence their ratings; they had no contact with the (fictional) students they were asked to evaluate (1). Furthermore, since the described share of disadvantaged students in the classroom varied across vignettes, comparing teachers’ perceptions in classrooms with a high share of disadvantaged students to classrooms with a low share of disadvantaged students reveals how compositional effects influence teachers’ perceptions of students (2). In addition, the research design disentangles compositional effects related to social status and ethnicity by leveraging the semantic distinction between ‘Roma’ and ‘poor’. Since the labeling of disadvantaged students varied randomly across the vignettes, comparing the compositional effects between the ‘Roma’ and the‘poor’ vignettes can reveal how the Roma ethnicity—rather than correlated characteristics of Roma (such as being poor)—affects teachers’ perceptions (3). The study also differentiates between direct and indirect compositional effects. By comparing teacher-perceived school performance of disadvantaged (Roma/poor) students in classrooms with a high versus low share of disadvantaged (Roma/poor) students, the direct compositional effects can be revealed. Similarly, by comparing the teacher-perceived school performance of non-disadvantaged (non-Roma/non-poor) students in classrooms with a high or low share of disadvantaged (Roma/poor) students, the indirect compositional effects can be exposed (4).

4.4. Treatment, randomization, and balance

The treatment in the experiment was the assignment of teachers to a vignette. Teachers were allocated vignettes based on a randomly generated number. The share of the four deployed vignettes is equally distributed in the sample as 26, 26, 24, and 24 percent of teachers answered the above-described four vignettes, respectively. Thus,145, 149, 134, and 135 teachers are assigned to vignettes #1, #2, #3, and #4, respectively.

The randomization resulted in a good balance of teachers’ observable characteristics. Teachers randomized to different vignettes were similar in terms of gender, age, and the grade for which they were homeroom teachers. Teachers were also balanced in terms of the schools in which they teach, with similar shares of Roma students, rich and poor students in the school, and school-average math and reading test scores. Concerning these variables, the pairwise mean difference between teachers assigned to different vignettes was statistically insignificant in most cases (see Table A2 in the Appendix)¹⁰. Therefore, the applied between-subject design does not appear to be undermined by the different baseline characteristics of the teachers.

4.5. The outcome variable

The outcome variable is students’ teacher-perceived school performance, rated by teachers after the treatment. To assess teachers' overall perceptions of students' academic performance, they were asked about students' general school performance without focusing on any particular subject. Teachers used a slider to indicate students’ school performance that could theoretically range between 1 (worst) and 5 (best), corresponding to the commonly used grading scheme in Hungarian schools. The slider had a sensitivity of one decimal, meaning that there were nine categories between two integer grades.

Figure 3 shows the distribution of the dependent variable. Although the most frequent response was 3 (indicating average performance), there was considerable variance in how teachers estimated the fictitious students’ school performance. The teachers’ mean estimate of students’ school performance was 3.3 (SD = 0.71).

Figure 3. Teacher’s perceptions of students’ school performance.

5. Empirical strategy

The following OLS model was used to estimate how the classroom’s composition affects teachers’ perception of students’ school performance.

In Equation (1), Y is teachers’ perceptions of students’ school performance. Variable S marks whether vignette v assessed by teacher t was a ‘poor’ or a ‘Roma’ vignette using the label of poor ( = 0) or Roma ( = 1) when describing the fictitious students¹¹. Variable C represents the classroom context (0 = low share of disadvantaged [poor/Roma] students; 1 = high share of disadvantaged [poor/Roma] students). The vector X captures teachers’ background variables (gender, age, fixed effect for the taught subjects, and the grade level where the teacher is a homeroom teacher, introduced in dummies), ${\phi }_t$ represents school fixed effects and ${ϵ}_{\mathrm{vt}}$ is the individual error term. Standard errors are clustered at the school level. (1) $Y_{\mathrm{vt}}={\beta }_1+{\beta }_2\times S_{\mathrm{vt}}+{\beta }_3\times C_{\mathrm{vt}}+{\beta }_4\times S_{\mathrm{vt}}\times C_{\mathrm{vt}}+{\beta }_5\times X_t+{\phi }_t+{ϵ}_{\mathrm{vt}}$(1) The coefficient ${\beta }_2$shows how teachers rated students labeled as Roma (rather than those labeled as poor) in a classroom context where the share of disadvantaged [poor/Roma] students is low. This coefficient does not have a substantive interpretation regarding the compositional effects.

The coefficients ${\beta }_3$ and ${\beta }_4$ refer to the compositional effects, which are the focus of the analysis. The coefficient ${\beta }_3$ shows the differences in students’ teacher-perceived school performance if the classroom has a high (rather than a low) share of poor students. The coefficients ${\beta }_3+{\beta }_4$ show the differences in Roma students’ teacher-perceived school performance if the classroom has a high (rather than a low) share of Roma students. Coefficient ${\beta }_4$ measures the difference between the two compositional effects: the social-status-based (poor) and the ethnicity-based (Roma). This coefficient tests whether the compositional effects are driven by the Roma ethnicity or rather by being poor as a correlate of Roma ethnicity.

Equation (1) is estimated for both the disadvantaged (Roma/poor) students and non-disadvantaged (non-Roma/non-poor) students in independent estimates to reveal direct and indirect compositional effects, respectively.

6. Results

Figure 4 shows the (non)-Roma and (non)-poor students’ average school performance in different classroom contexts: where the share of disadvantaged (Roma/poor) students is low or where the share of disadvantaged (Roma/poor) students is high. Table 2 shows the compositional effects estimated by Equation (1).

Figure 4. Teachers’ perceptions of (non)-poor and (non)-Roma students’ average school performance under different classroom compositions.

Table 2. Compositional effects in students’ teacher-perceived school performance.

	(1)	(2)
	Direct compositional effect Teacher-perceived school performance of disadvantaged students	Indirect compositional effect Teacher-perceived school performance of non-disadvantaged students
Roma students [${\beta }_2$]	−0.07	0.03
	(0.17)	(0.15)
Social-status-based compositional effect [${\beta }_3$]	−0.03	−0.07
	(0.18)	(0.16)
Ethnicity-based compositionaleffect [${\beta }_3$+${\beta }_4$]	−0.37**	−0.51**
	(0.15)	(0.17)
Difference between Roma/poor compositional effects [${\beta }_4$]	−0.35	−0.44*
	(0.25)	(0.22)
N of teachers	562	562

Robust standard errors (clustered at the school level) in parentheses

**p < 0.01, *p < 0.05, + p < 0.1

Note: coefficients in the table are estimated with Equation (1).

All models contain a constant and the following control variables: teachers’ gender, teachers’ age, dummy variables signaling the grade in which the teacher is a homeroom teacher, and fixed effects for the taught subjects. Models also contain school fixed-effects.

Column 1 in Table 2 estimates the direct compositional effects. In classrooms with a high (rather than low) share of Roma students, teachers rated Roma students’ school performance by 0.37¹²units lower (p = 0.01) on a scale ranging from 1 to 5. Correspondingly, there is a remarkable difference between the two black bars in Figure 4. Expressing the ethnicity-based direct compositional effect in the units of the dependent variable’s standard deviation results in a medium-sized (−0.57) standardized effect. By contrast, in classrooms with a high (rather than low) share of poor students, teachers did not rate students’ performance differently (${\beta }_3$ = −0.03; p = 0.89). The difference between the two dark-gray bars in Figure 4 is small. There is only a small standardized effect of 0.04 for the social-status-based direct compositional effect. The two (Roma and poor) direct compositional effects are statistically not different; ${\beta }_4$ is statistically not significant (${\beta }_4$ = −0.35; p = 0.16). This comparison is equivalent to the difference between the two black bars relative to the difference between the two dark-gray bars in Figure 4. Thus, the decrease in teachers’ ratings of Roma students in classrooms with a high versus low share of Roma students was statistically similar to the decrease in ratings of poor students in classrooms with a high versus low share of poor students.

Column 2 in Table 2 estimates the indirect compositional effects. In classrooms with a high (rather than low) share of Roma students, teachers rated non-Roma students’ school performance significantly lower ($[{\beta }_3+{\beta }_4]$ =  −0.51; p > 0.01). Correspondingly, there is a remarkable difference between the two white bars in Figure 4. The estimated ethnicity-based indirect compositional effect is equivalent to a −0.78 standardized effect, expressing the point estimate in the standard deviation unit—a large effect. By contrast, teachers did not rate non-poor students’ school performance lower in classrooms with a high (rather than low) share of poor students (${\beta }_3$ = −0.07; p = 0.65). Thus, the difference between the two light-gray bars in Figure 4 is small. The standardized effect is also small-sized: −0.11. The two (Roma and poor) indirect compositional effects differ statistically since ${\beta }_4$ is statistically significant (${\beta }_4$ = - 0.44; p = 0.05). This indicates that the decrease in teachers’ ratings of non-Roma students in classrooms with a high versus low share of Roma students was larger than the decrease in ratings of non-poor students in classrooms with a high versus low share of poor students.

Sensitivity analyses (in Tables A3 and A4) reveal that both the direct and indirect compositional effects are robust concerning teachers’ real experience with Roma. When the same models are estimated on various subsamples based on the actual size of the Roma minority in the settlement, the estimated coefficients do not change substantially. Differences between the coefficients are never statistically significant and are attributed to the smaller sample size, making the point estimates more imprecise. This indicates that the compositional effects are not influenced by teachers’ actual experiences of Roma people.

As an aside, and not related to the compositional effects, Table 2 indicates that labeling disadvantaged studentsas Roma or poor did not translate to any difference in teachers’ perceptions, as the coefficients are not statistically significant in either of the two columns of the table.

7. Discussion and conclusion

This study investigated how classroom composition affects teachers’ perceptions of students’ school performance. The classroom’s ethnic and social status composition has the potential to influence teachers’ perceptions, as categorization processes often lead individuals to assume similar attitudes between people who share social connections (Halgin, Borgatti, and Huang 2020). The deployed experimental design elucidates how Roma ethnicity and the correlated economic disadvantage drive compositional effects. It was hypothesized that teachers in classrooms with a high share of disadvantaged students (Roma/poor) would perceive students’ school performance as lower than teachers in classrooms with a low share of disadvantaged students.

The results supported the hypothesis concerning Roma students. Teachers rated the school performance of both Roma students (direct compositional effect) and non-Roma students (indirect compositional effect) lower in classrooms with a high share of Roma students than in classrooms with a low share of Roma students. However, no evidence emerged for social-status-based compositional effects. Teachers rated economically disadvantaged, poor, and non-poor students similarly, regardless of whether the classroom had a high or low share of poor students. The comparison between ethnicity (Roma) and social-status (poor) driven compositional effects shows a statistically significant difference in indirect compositional effects, while no difference was found in direct compositional. The significant difference observed between ethnicity-based and social status-based indirect compositional effects suggests that teachers’ perceptions may be particularly driven by the ethnic Roma context rather than by economic disadvantage, often correlated with the Roma context.

Evidence of ethnicity-based indirect compositional effects could have important policy consequences. It could signal the boundaries of desegregation programs and explain why some high-status parents oppose integrated education (Goldring and Hausman 1999). For example, if non-Roma students’ performance suffers in a Roma-dominated classroom, parents may be opposed to sending their children to segregated Roma schools. The indirect compositional effects may also lead to discrimination against non-Roma students in situations where teachers assess students’ performance, such as grading, giving school recommendations, or school admissions. Therefore, non-Roma students in these schools might receive lower grades, less advanced school recommendations, and a reduced likelihood of admission to higher-quality schools.

The non-representativeness of the sample used in the analysis may limit the external validity of this research. Although some observed characteristics of teachers have been controlled for, there could be other unobserved teacher-level characteristics influencing teachers’ perceptions that are not accounted for. While teacher-fixed effects could address this issue, they are not available here, as teachers responded to one vignette. A less profound bias might be caused by the non-representative sample at the school level. While the schools in the sample are small village schools with a higher share of Roma students, any biases that the particular school context might introduce to teachers’ perceptions are controlled for by school-fixed effects, allowing for comparisons of teachers from the same school.

Another issue is whether teachers’ perceptions are informed by their experiences or their biases. In this regard, sensitivity analyses indicated robust results concerning teachers’ real experiences with Roma students. The observed direct and indirect compositional effects remained stable, regardless of the teachers’ prior experiences with Roma people. This suggests that prior experience with Roma people has little influence on the observed compositional effects¹³.

In conclusion, the finding of disadvantageous perceptions of non-Roma students in high-Roma-share schools is a remarkable finding that, to the best of my knowledge, has not been established before. Nevertheless, further research should delve deeper into how the composition of classrooms influences teachers’ perceptions of students’ school performance—specifically exploring ethnicity-based indirect compositional effects. The Hungarian context and the focus on Roma holds significant importance for countries with a large Roma minority, such as Bulgaria, Slovakia, Romania (Plainer 2021; Rauh 2018), and Serbia (Battaglia and Lebedinski 2022; Baucal 2006). Understanding the dynamics of how classroom composition affects teachers’ perceptions can shed light on the potential impact of ethnicity and social context on educational outcomes (Hermann and Kisfalusi 2023), providing valuable insights for educational policies and interventions aimed at promoting equity and inclusivity in schools (Alesina et al. 2018).

Acknowledgements

The research was supported by grants from the Hungarian National Research, Development and Innovation Office (NKFIH), Grant number K-135766; the János Bolyai Research Scholarship of the Hungarian Academy of Sciences (BO/00569/21/9) and the New National Excellence Program of the Ministry for Culture and Innovation from the source of the National Research, Development and Innovation Fund (Grant Number: ÚNKP-23-5-CORVINUS-149).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 The Roma’s disadvantaged position, especially within the school system, has been a topic of social research in Hungary and neighboring countries such as Romania (Rauh 2018; Plainer 2021) and Serbia (Baucal 2006; Battaglia and Lebedinski 2022) for many decades.

2 The hypothesis positing a positive correlation between classroom composition and teachers’ perceptions of individual students’ school performance differs from the predicted negative correlation posited in Davis’ (1966) frog pond theory or the related big-fish-in-little-pond (BFLP) theory by Marsh (1987). However, the negative correlation pertains to the classroom’s ability composition, whereas the positive correlation assumed in this paper relates to the classroom’s ethnic and social status composition.

3 Seemingly precise perceptions can potentially arise from self-fulfilling prophecies (Rosenthal and Jacobson 1968; Friedrich et al. 2015) or sustaining expectations (Rubie-Davies, Hattie, and Hamilton 2006)—processes that negatively impact students’ performance.

4 The violin plots in Figure 1 show the distribution of Roma students (left panel) and poor students (right panel) based on the categories of poor (left panel) and Roma (right panel). Violin plots integrate two types of information. First, they show symmetric density curves of the distribution of students (y-axis) according to the categories of variables on the x-axis. Second, within the center of each symmetric curve (violin), a box plot provides additional distributional detail about the median (white dots) and interquartile ranges (dark gray columns).

5 Schools with a high share of Roma students are mostly located in small villages—places where principals might not observe significant status differences that would justify classifying some students (relative to others) as poor. This might explain why the median percentage of poor students is ‘only’ 60% in schools where the share of Roma students is above 80%.

6 This is an unsurprising result as teachers received the vignettes randomly.

7 Nevertheless, the models in Table A1 reveal that female teachers rated students’ school performance systematically higher than male teachers did. Teachers’ gender is controlled in the estimations shown later in the results section.

8 The vignettes always used the terms ‘high’ and ‘low’ without connecting them to specific benchmarks.

9 Upon receiving vignette #1, teachers evaluated poor students, before evaluating non-poor students in a classroom with a low share of poor students. With vignette #2, they initially rated Roma students, and then non-Roma students in a classroom with a low share of Roma students. Vignette #3 led them to first assess poor students, and then non-poor students in a classroom with a high share of poor students, while vignette #4 prompted them to first evaluate Roma students, and then non-Roma students in a classroom with a high share of Roma students.

10 There are only sporadic statistically significant mean differences between teachers assigned to vignettes #1 and #4 concerning teachers’ gender (p<0.01) and the share of poor students in the classroom (p = 0.04). Furthermore, teachers assigned to vignettes #2 and #4 differ in their gender (p = 0.02). None of these statistically significant mean differences remain significant after correcting for multiple testing. Since teachers’ gender and school fixed effects appear in all models, even these small differences are controlled for.

11 Note that the same notion was used to describe the disadvantaged and the not-disadvantaged student in the two consecutive questions after each vignette.

12 ${\beta }_3$+${\beta }_4$

13 Nevertheless, the primary focus of this study was to explore teachers’ perceptions of students (which the study was designed to do) rather than exploring the reasons for these perceptions (which the study was not designed to do).