The influence of school climate and achievement on bullying: Comparative evidence from international large-scale assessment data

ABSTRACT

Background

Bullying is widely acknowledged as one of the most harmful events in a child’s life, leading to negative life experiences and outcomes. However, ‘school effects of bullying’ are rarely studied from an international perspective, especially with international large-scale data.

Purpose

In this study, we aim to look at bullying through an international comparative approach, focusing on the contribution of the school, the education system and culture, and pupil level factors such as socio-economic status (SES) and gender. Our key question is whether school factors can affect bullying prevalence.

Methods

We used data from six countries (USA, Finland, England, Romania, Korea and Italy) from the Trends in International Mathematics and Science Study (TIMSS), a large-scale international study. We used multilevel modelling to analyse the dataset.

Findings

We find little evidence of a relation between country policies and levels of bullying, though there are differences in the extent to which school and pupil factors are related to bullying. The most consistent relationship was between gender and bullying, with prevalence higher among boys, while school factors were not significant.

Conclusion

The findings indicate that ‘one size fits all’ school policies might not be the best course of action, and individual support might be a more fruitful avenue.

KEYWORDS:

• Comparative education
• bullying
• secondary schools
• large-scale assessment data
• international
• wellbeing

Introduction

Bullying is widely acknowledged as one of the most harmful events in a child’s life, leading to negative life experiences and outcomes not just in the short term, but, according to some studies, in the long term as well (Jenny, Hodges, and Salmivalli 2008; Smith 2016; Ttofi et al. 2011). It is therefore unsurprising that bullying is considered an urgent issue that schools should address, and a large number of school-based interventions aim to do exactly that.¹ This intensive activity begs the question as to the extent that schools, in their day-to-day operation, can create an environment which acts as a barrier to bullying. This would be observed as a school effect on bullying. This is an important question, not just in relation to bullying itself, but also as part of the broader call for educational effectiveness research (EER) to increase its focus on non-cognitive outcomes. In recent years, there has been a growth in studies of school effects on non-cognitive outcomes and there are some EER studies focused specifically on school-level effects on bullying. Although many of the relatively small-scale studies show quite differential effects (e.g. Olweus and Limber 2010; Salmivalli and Poskiparta 2012), one study, Kyriakides et al.’s (2013) five country study, provides the potential for evidence on a larger scale. Kyriakides et al.’s (2013) study is one of just a few international comparative studies of school effects on bullying, albeit limited to Europe. The lack of comparative studies is unfortunate in one sense, as our understanding of factors that might protect against bullying should encompass culture and policy at the system level as well as at the school level, to provide a deeper understanding of what the role of the school can be.

In this study, we therefore aim to look at bullying through an international comparative approach, focusing on the contribution of the school, the education system and culture, and pupil level factors such as socio-economic status (SES) and gender. Our key question is whether school factors can affect bullying prevalence. To address this, we use data from the Trends in International Mathematics and Science Study (TIMSS). While TIMSS and other International Large-Scale Assessment (ILSA) studies collect data on bullying through their surveys, this has not usually been the main focus of analysis of these datasets. By comparing six countries with differing known policy approaches to bullying, we contextualise our analyses in what Elliott et al. (2019, 5) called an ‘ecosystemic approach’. This will allow us to look at the extent to which individual and school factors may have different relations to bullying prevalence, which in turn may be related to policy, and will allow us to interrogate the extent to which relationships between pupil-level factors such as gender or social background, and school-level factors such as safety and discipline and aggregate pupil SES, have stable relations with bullying across contexts.

Background

In this section, we cover the literature on bullying. We start with a description of the importance of non-cognitive outcomes and our definition of bullying. We then describe a multilevel approach to bullying, covering individual, student and country factors related to bullying.

Non-cognitive outcomes

Internationally, attainment might be one of the most important influencers of policy, often to the detriment of analyses involving non-cognitive outcomes. Recent reviews of educational effectiveness studies show that there are four times more studies that focus on cognitive outcomes than non-cognitive outcomes (Chapman, Reynolds, and Muijs 2015). The studies that do exist focus mainly on dispositions, rather than pupil behaviours. Subsequently, it can be said that studies aimed at non-cognitive outcomes serve to fill a gap, as they concern many important phenomena that affect children and young people and therefore may be influenced by school effects. One such type of non-cognitive outcome is bullying. Bullying, as a persistent and highly harmful phenomenon in schools, might typify the relationship between school and classroom processes and non-cognitive factors (Kyriakides et al. 2013). From an international perspective, the Organisation for Economic Co-operation and Development (OECD) has worked with the definition of bullying as provided by Olweus (1993, 9): ‘A student is being bullied or victimized when he/she is exposed, repeatedly and over time to negative actions on the part of one or more other students. It is a negative action when someone intentionally inflicts, or attempts to inflict, injury or discomfort upon another’. The definition emphasises intentionality, implying a power imbalance between bully and victim. The causes of this are part of a complex set of factors, many of which pertain to individual contextual factors and patterns in bullying.

Bullying is not a purely unidimensional construct. Different forms of bullying exist and affect children in different ways, including verbal, physical, social and, increasingly, cyber-bullying. There is some evidence that experiencing multiple forms of bullying has stronger adverse effects on children (Smith 2018).

The multilevel nature of bullying

The previous sections have shown that bullying can be described at different levels – i.e. the individual, classroom, school and country level, corresponding with the multilevel nature of the Dynamic Model of Educational Effectiveness (Kyriakides and Creemers 2008). Classroom factors are included, with school-level factors providing conditions under which the effectiveness of classroom-level factors can be maximised by developing and evaluating policies on teaching, and creating a positive learning environment in the school. The model also takes into account the notion that the school situation is influenced by the wider educational context in which students, teachers, and schools are expected to operate. There is, however, a complicating factor when applying the model to large-scale assessment data, as researchers are limited to working with the data that are available. The model we are using is depicted in Figure 1.

Figure 1. Theoretical model regarding predictors of students’ perceptions of bullying in school.

Given the previously described literature, we posit that, with respect to bullying, national policies influence school and classroom factors, especially perceptions of having a safe and orderly climate in the school, and perceptions of school discipline and safety. These factors, together with the social background of the school (operationalised as school SES), in turn influence student perceptions of bullying. However, they also influence and are influenced by individual characteristics like gender, country of birth, SES and achievement level. We will elaborate more on the specific variables in the methodology section.

The model therefore suggests that bullying can be affected by factors at individual, classroom, school and country levels. At present, there are few studies that have taken this multilevel approach to the study of bullying. A range of studies exist that consider the impact of factors at individual levels in the system, rather than employing a multilevel approach, and we discuss some of these in the following section.

Individual factors and social patterns in bullying

Factors related to bullying are relatively stable over time and related to several individual characteristics (e.g. Cook et al. 2010; Scholte et al. 2007). Victims are often quieter and more sensitive, and more likely to be social isolates, while bullies tend towards aggressive behaviour patterns, a need for power, and display more positive attitudes towards violence as well as a lack of empathy towards the bullied (Olweus 1993). The stability of bullying behaviours can support an emphasis on early intervention. In our proposed model, we have not looked primarily at these personality characteristics, but have focused on demographic characteristics of pupils such as gender, country of origin and SES, and at their attainment levels.

The relationship between attainment and bullying is contested, with some studies finding no relationship, while others have found some relationship between being bullied and poor attainment among primary age students. There is also some evidence that anti-bullying interventions have been successful in raising attainment (Swearer et al. 2010). Perpetrators and victims of bullying are more likely to be boys than girls. Girls, however, appear more likely than boys to act as defenders of the bullies, while among boys, roles reinforcing bullying behaviours appear more prevalent (Salmivalli et al. 1996). Bullying can target minorities, and characteristics such as race, religion or sexual orientation (O’Malley Olsen et al. 2014; Connell et al. 2016; Scherr and Larson 2010; Brown and Taylor 2008). When children bully in groups, or where bullying is part of normal group behaviour, research suggests that friendships are made and maintained in part to show a distinction from other groups, or individuals (Duffy and Nesdale 2008). Traditionally, bullying was treated mainly as an individual phenomenon, but in recent years it has become clear that it is a collective problem in which group members play a crucial role, with the role of bystanders highlighted as being key in the promotion or prevention of bullying behaviours (Olweus 1993). As a result, it has become apparent that social and group norms are important in terms of pupils’ reaction and resilience to bullying, and several interventions have focused on changing such group norms (e.g. Flaspohler et al. 2009). An ecological developmental perspective, which posits that individuals are nested within a range of interlocking settings, including families and schools in the case of children, supports the possibility that schools may affect bullying prevalence (Bronfenbrenner 1979).

School factors related to bullying

The fact that bullying is a social phenomenon that heavily involves peers suggests that peer cultures may be influential, and also that school factors that influence peer cultures may serve as a protective factor (Sutton and Smith 1999). School culture and disciplinary climate may, therefore, be important here and are included in our model (see Figure 1). School and classroom climate and culture may influence the prevalence of bullying behaviours, as it may influence the attitudes of bystanders towards bullying, and their willingness to intervene in the bullying situation, whether as a student or a teacher. Prior research on the magnitude of school effects on bullying is not entirely consistent. Galand, Hospel, and Baudoin (2014), in their study of secondary school students in Belgium, found 5% of variance in bullying at the classroom level and less than 0.5% at the school level. Kyriakides and Creemers (2013), by contrast, found 20% school level and 25% classroom-level variance in bullying prevalence among primary school children in Cyprus. Similar findings are reported in studies by Kyriakides et al. (2013) for primary school pupils in five European countries (24% variance at the school‐level) and Muijs (2017) among primary school pupils in England (17% variance at the school level).

In one study, a positive school climate was related to lower rates of bullying and teasing and predicted higher graduation rates four years later (Dewey et al. 2013). School climate appears to be the most frequently encountered school-level factor related to bullying. Cook et al. (2010, 76), in their meta-analysis, suggest that bullying is more prevalent in ‘schools with a negative atmosphere’, while a large-scale study in Colorado found perceptions of a negative school climate measured at time 1 to be significantly related to self-reported bullying perpetration one year later (time 2), even after controlling for early incidents of bullying at time 1. As well as school climate, some evidence of relationships with related school and classroom practices exist. Galand, Hospel, and Baudoin (2014) found that in classrooms in which teachers were reported as directly intervening in bullying situations, less bullying was also reported, while the inverse was the case when class goals were strongly oriented towards attainment. Kyriakides and Creemers (2013) found that positive teacher–student relationships, policies for behaviour outside the classroom, partnerships, and evaluation of the school learning environment were significantly related to lower levels of bullying. Muijs (2017) also examined school policies and found a significant positive relationship between lower levels of bullying and policies on behaviour, including specific policies on bullying. All in all, these studies suggest that schools can make a positive and a negative difference when it comes to bullying, though the designs of prior studies do not allow us to make strong causal claims. As each of these studies employed cross-sectional designs, we cannot determine causality from them. Lower levels of bullying could simply be a correlate, rather than occurring as a result of school culture, although there is some limited evidence from longitudinal studies that could point to a causal relationship. Furthermore, most prior studies have considered a limited number of types of bullying, or have treated it as a unidimensional construct. Few studies include cyberbullying, which is the fastest growing problem in this area. Even where school and classroom effects are found, they do not necessarily result from differential school and classroom processes. The heterogeneity of schools and classrooms with regards to pupil characteristics, such as SES, means that variation in student intakes could equally be an explanation (Galand, Hospel, and Baudoin 2014). Aggregate measures of student social background have therefore been included as a key school-level factor in our model.

Country factors related to bullying

Few studies focus on country-level effects. Kyriakides et al. (2013) is one such study, evaluating school effects across countries, without explicitly focusing on country-level differences and how these are related to culture and policies. We hypothesise that national policies (for example, by raising awareness, supporting anti-bullying strategies or mandating anti-bullying policies) could influence bullying prevalence.

Purpose and contribution of this study

Our take-away point here is that multiple factors could be considered and compared internationally: at an individual level and at the level of classroom and school climate. Could large-scale assessment data be utilised? A recent report by the UNESCO (2019) combined data from two large-scale international surveys, the Global School-based Student Health Survey (GSHS) and the Health Behaviour in School-aged Children (HBSC) study, and found that almost one in three students (32%) reported having been bullied by peers at their school at least once in the previous month. The OECD has also conducted analyses in their Programme for International Student Assessment (PISA) study. According to PISA results (OECD 2017), the proportion of students who reported being victims of bullying is ‘larger in schools with high percentages of students who had repeated a grade, where students reported a poor disciplinary climate in class, and where students reported that their teachers treat them unfairly’ (15). Here again, we see school factors come into play and, as a result, differential effects within countries. As with the studies reviewed above, these international studies are cross-sectional, and we cannot therefore establish causality. What these large-scale assessments have in common is that they require a multilevel approach to international data.

The first contribution of this study is, therefore, to add to the knowledge base on the impact of schools on bullying, focusing on secondary schools, where few previous studies exist. A second contribution relates to the international comparative context, namely, the extent to which these effects differ by country. While the study by Kyriakides et al. (2013) considered five European countries, and found few differences, the countries involved did not contrast greatly in terms of culture or policies. In this study, therefore, we consider six contrasting cases: the United States, Finland, England, Romania, Korea and Italy. Finally, this study is one of the first to use large-scale international assessment data to look at bullying as a dependent variable. In particular, we looked at the TIMSS 2011 data. Although more recent data are available, the timing of 2011 data is more optimal in relation to implemented bullying programmes, for example from 2008 to 2011 in Finland.²

Our overarching research question was: What is the relationship between school factors and bullying? This study set out to answer this overarching question through two related research questions. Firstly, we wanted to explore, through multilevel models of data from six different countries, how much of the variance in grade 8 (13–14-year-old) students’ self-reported bullying can be explained at student and school levels. Secondly, we wanted to see what predictors at student and school level exist for self-reported bullying in the six counties.

Methodology

The current study utilised data from the 2011 Trends in International Mathematics and Science Study, TIMSS 2011 (Mullis et al. 2012). TIMSS 2011 is the fifth in the International Association for the Evaluation of Educational Achievement’s (IEA) series of international assessments of student achievement dedicated to improving teaching and learning in mathematics and science. First conducted in 1995, TIMSS reports every four years on the achievement of fourth- and eighth-grade students. This study uses grade 8 data, as we focus on secondary schools. Although there are some differences in the naming of grades (e.g. grade 8 is year 9 in England), grade 8 covers 13–14-year-olds. In analysing the data, we needed to take into account the complex sampling design of TIMSS. First, we used appropriate sampling weights, final weights, to cater for different probabilities of units being selected (Rutkowski et al. 2010). Second, we used multilevel models to account for variance estimation. We only included one plausible variable as predictor in our models, which risks underestimating standard errors. For this reason, and the relatively high sample sizes, we chose p < 0.01 as a more conservative level for the test of significance.

Country choices

As described above, we focused on data from six countries from TIMSS 2011. These were purposefully selected on the basis of the authors’ knowledge of the policies and practices related to bullying in these countries.³ The United States was included as a country with a firm tradition of anti-bullying programmes (AERA 2013).⁴ The United States, of course, does not have a national policy, with responsibility for education being devolved to each of the States, but as the AERA document (American Educational Research Association 2013) suggests, there is evidence of bullying issues in many schools and also of a large number of programmes and initiatives implemented to combat bullying in schools (see also Ttofi et al. 2011). We included England because the authors have studied the prevalence of bullying in this country. In England, based on the Education and Inspections Act 2006 (Section 89), it is compulsory for schools to enforce measures that will encourage good behaviour and prevent all forms of bullying (Department for Education 2017). Bullying prevention is therefore seen as being part of a school’s behaviour policy. Head teachers have a specific statutory power to discipline pupils for poor behaviour outside of the school premises. The national school inspectorate specifically includes bullying as part of its inspection framework.

Three other European countries, Finland, Italy and Romania, were included because they have experience with bullying programmes, and it therefore was relevant to explore how they fared in the TIMSS 2011 data. Finland has a national anti-bullying programme, funded by the ministry of education and culture, known as KiVa, a programme that has featured in this journal before. The KiVa anti-bullying programme was developed and evaluated in a stringent study including 234 schools representing all provinces of Finland (e.g. Antti. et al. 2011; Salmivalli, Garandeau, and Veenstra 2012). The findings showed that KiVa reduced bullying and victimisation significantly but also improved school liking, academic motivation and academic performance. KiVa is implemented in 90% of the schools. As this source is from 2011, it might be expected that TIMSS 2011 data would reflect the new Finnish situation, and a comparative view would therefore be useful.

In Italy, a number of national initiatives and laws exist, as well as additional regional strategies. At national level, there is a network of permanent regional observatories, which have research, training, monitoring and informative functions. There is also a national anti-bullying hotline and website, and collaboration with the police forces (i.e. postal police) in order to monitor aggression via web-based media and to inform students about the risks of the Internet and about the legal issues connected to cyberbullying (for a more detailed review, see Genta et al. 2009; Guarini, Brighi, and Genta 2010). In Italy, there are laws against using mobile phones in the classroom and there is also an anti-bullying week like in England. In Romania, policy is primarily aimed at preventing violence (Downes and Cefai 2016). The Minister of Education, Research and Youth established the National Council for preventing and fighting violence in the educational environment and instigated an anti-violence strategy. However, it is clear that this represents a somewhat limited approach in light of the prevalence of non-violent forms of bullying.

Finally, Korea was included as the highest performing TIMSS 2011 grade 8 jurisdiction, and as ‘representative’ of high performing Asian countries. Some research suggests that the high stakes culture in Asia might have negative effects on non-cognitive outcomes, and some studies suggest that Korea has high levels of bullying behaviour in secondary schools, which are sometimes linked to the high prevalence of teenage suicides (Sarzosa and Sergio 2015). While the government has taken action, including the introduction of cameras and guards in schools with particular problems, these fall short of the comprehensive strategies seen elsewhere. We would therefore hypothesise that those countries with more developed national policies and approaches (England and Finland) would see lower levels of bullying and lower relationships with school and pupil-level factors, while those with the least developed policies (Korea and Romania), would see the highest prevalence and relationships.

Variables

To create the models, we used the following variables from the TIMSS 2011 dataset (Foy, Arora, and Stanco 2013).

Dependent variable: As the dependent variable at the student level, we used the Students Bullied at School scale. This scale was created based on students’ responses to the six statements on being bullied or excluded: ‘I was made fun of or called names’, ‘I was left out of games or activities by other students’, ‘Someone spread lies about me’, ‘Something was stolen from me’, ‘I was hit or hurt by other student(s) (e.g. shoving, hitting, kicking)’, ‘I was made to do things I didn’t want to do by other students’. Students answered ‘never’, ‘a few times a year’, ‘once or twice a month’ or ‘at least once a week’. The scale was constructed using IRT scaling methods; specifically, the Rasch partial credit model (Masters and Wright 1997). Scales are standardised by a linear transformation to have a mean of 10 and a standard deviation of 2. Practically, it means that higher scores are ‘better’ with less reported bullying. We considered rescaling this scale, but as this is a scale provided by the IEA, we opted for keeping it the same but rephrased it as ‘Bullying Resilience’ scale (BR). The scale has been validated by IEA,⁵ with Cronbach alphas for the scales .69 and over (USA .78, FIN .76, ENG .77, ROM .73, KOR .75, ITA .69), variance explained between 40 and 50%, and factor loadings .51 and higher, making the scale suitable for analysis. The scale does have limitations in that it is both unidimensional and lacking items relating to cyberbullying.

Independent variables: At the student level we included:

• Male student (BOY)

• Whether the student is born in another country (born elsewhere).

• Home Educational Resources (SES) is a scale in the TIMSS 2011 dataset concerning the availability of three home resources, which we used as a proxy for SES. The scale contains items on the number of books at home, the highest level of education of either parent and the number of home study supports. As with the BR scale, the scale was standardised to a mean of 10 and standard deviation of 2. Although Cronbach alphas were relatively low (between .35 and .63), with lower factor loadings for the number of home study supports, the scale explained between 44 and 59% variance. However, to facilitate any future comparisons, we kept the scale intact. With this scale, higher scores mean ‘better’: more resources and thus higher SES.

• Mathematics achievement (MATHS, standardised around mean 500, standard deviation 100, first plausible value). The score is an imputed score, based on the achievement in a rotated-booklet design.

At the school level we included:

• Teacher scores for the Safe and Orderly School (SCHOOLSOS) scale were aggregated per school, by taking the mean of all maths teachers’ SOS scores in a school. The SOS scale consists of items scored by teachers in the school: safe neighbourhood, feeling safe, whether the school’s policies and practices are sufficient, orderly student behaviour and respectfulness towards teachers. As with the BR scale, the scale was standardised to a mean of 10 and standard deviation of 2. Cronbach alphas for the scales were .78 and over (USA .83, FIN .78, ENG .81, ROM .81, KOR .83, ITA .79), variance explained between 53 and 60%, and factor loadings .62 and higher. Higher scores are ‘better’, indicating a more safe and orderly school.

• The School Discipline and Safety (SCHOOLDAS) scale was constructed from items that were answered by school principals. The scale has 11 items asking the degree to which certain behaviours are prevalent among eighth-grade students in their school: arriving late, absenteeism, classroom disturbance, cheating, profanity, vandalism, theft, intimidation or verbal abuse among students, intimidation or verbal abuse of teachers or staff, and physical injury to teachers and staff. Principals would indicate whether these aspects were ‘not a problem’, ‘a minor problem’, ‘a moderate problem’ or ‘a serious problem’. As with the BR scale, the scale was standardised to a mean of 10 and standard deviation of 2. Cronbach alphas for the scales were .85 and over (USA .89, FIN .85, ENG .87, ROM .93, KOR .95, ITA .94),⁶ variance explained between 44 and 69%, and factor loadings mostly over .60. Higher scores are ‘better’, indicating positive school discipline and safety.

• Finally, school SES composition was included as the mean of all students’ Home Economic Resources in a school (MEANSES).

Analytical approach

To answer our research questions, we constructed multilevel models per country, with students nested in schools, as the data are hierarchical in structure. Although each of the students is taught in a classroom, which can be said to be nested in a school, the TIMSS 2011 data only sample one or two classrooms per school, so we ‘collapsed’ the classroom and school levels into one school level. Through multilevel modelling, an adaptation of the general linear model for hierarchical datasets, we can explore the unique impact of student and school characteristics upon students’ responses to items. Two-level models were used, as in many of the countries we sampled there is typically only one class per school in the sample, which means that the school and classroom levels are conflated (Snijders and Bosker 2012).

Building models

Analyses were conducted with MLwiN version 2.34 (Rasbash et al. 2016).⁷ We used IGLS estimation and obtained convergence on the results.⁸ For mathematics achievement as an independent variable, we used the first plausible value. Models were built by starting with the null model (model 1), then adding student characteristics variables (model 2) and then adding school level variables (models 3 and 4). Throughout the analyses, we used group mean centring at the student level and grand mean centring at the school level. We have assumed normally distributed dependent variables. The missing data for the variables was only between 0 and 2%. As this would not have a big effect on the estimates, we decided not to impute missing data (i.e. replace them by an estimate) and to use list-wise deletion (i.e. deleting all cases with a missing value) for model building. The model outputs, including parameter estimates, model fit, and significance at the 1% level (p < 0.01) are reported in the tables of results below.

Results

In this section we report the results for the analyses, first by presenting descriptive statistics, and then a summary table of the full, final models. We have included the full, four models per country separately in Appendix A. Table 1 presents descriptive statistics for dependent and independent variables.

Table 1. Descriptives of dependent and independent variables for six countries and the average for the total sample

Variables	USA		FIN		ENG		ROM		KOR		ITA		Average
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Student level
Bullying Resilience (BR)	10.10	1.93	10.52	1.83	10.43	1.92	9.67	1.89	10.29	1.85	10.68	1.77	10.28	1.86
Male student (BOY)	1.49	0.50	1.52	0.50	1.52	0.50	1.52	0.50	1.48	0.50	1.51	0.50	1.51	0.50
Born elsewhere	1.08	0.28	1.03	0.18	1.09	0.29	1.01	0.09	1.01	0.10	1.07	0.25	1.05	0.20
Home educational resources (SES)	10.89	1.78	11.22	1.46	10.79	1.62	9.91	1.73	11.36	1.75	10.30	1.65	10.74	1.66
Mathematics achievement (MATHS)	510.99	76.09	514.39	64.81	507.74	84.82	461.04	99.99	612.93	89.75	498.40	73.20	517.58	81.44
School level
Mean home educational resources	10.77	1.02	11.21	0.47	10.74	0.86	9.93	1.16	11.37	0.68	10.27	0.77	10.71	1.00
Mean safe & orderly school score (SCHOOLSOS)	10.38	1.82	9.42	1.57	10.58	2.17	10.49	1.80	8.46	1.60	8.88	1.55	9.70	1.82
Mean discipline & safety score (SCHOOLDAS)	10.26	1.42	9.89	1.14	10.75	1.55	10.50	2.14	10.52	2.55	9.75	1.86	10.28	1.78

Mean scores on the bullying scale differed between countries, with Italy higher than the average and Romania lower. The other countries, the United States, Finland, England and Korea, lie in-between. In addition, the levels for five out of six countries were above the overall standardised mean of 10 for the TIMSS 2011 sample. This is an indication that these countries from our sample experienced higher resilience to bullying than other countries in the TIMSS dataset. Nevertheless, the difference was not so large that they might suggest that national policies are necessarily strongly related to levels of bullying prevalence at the country level. Genders were approximately equally balanced in each of the countries. There were some differences in the numbers of students not born in the country, with Romania and Korea having somewhat lower percentages. Average SES was highest in Korea and lowest in Romania. The final student-level variable, mathematics achievement, was highest in Korea, with a mean over 600 points, while Romania and Italy were the only two countries below 500 points. At the school level, there were also substantial differences: for example, for the Safe and Orderly School scale, Korea and Italy both scored much lower than the other countries, well below average, while England and Romania scored higher than average. This corresponds with the School Discipline and Safety scale, for which England and Romania also scored highly, although on this measure Korea scores highly as well, contrary to Italy, which again scored relatively low. MEANSES, the country mean of school SES, is our indicator of schools’ SES. As expected, it was close to country means based on individual students’ SES. The level of variance of school mean SES differed across a set of countries, with Finland having a low standard deviation accompanying a high school mean SES, while Romania had the opposite combination. At first glance, if there is one thing the descriptives might tell us, it is that there seems to be a less-than-straightforward pattern between these variables. For the six separate countries, we can see this in more detail in Appendix A; for comparisons between countries, we offer plots of 95% Confidence Intervals for the predictor variables in Appendix B. Table 2 provides an overview of the full models for all six countries.

Table 2. Multilevel model 4: + school climate for the six countries in one table

	USA		FIN		ENG		ROM		KOR		ITA
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	10.182**	0.038	10.855**	0.058	10.579**	0.065	9.812**	0.049	10.66**	0.058	10.734**	0.063
Student level
Male student (BOY)	−0.111	0.054	−0.572**	0.08	−0.227**	0.085	−0.23**	0.066	−0.807**	0.069	−0.081	0.075
Born elsewhere	−0.147	0.092	−0.372	0.239	−0.625**	0.135	−0.598	0.33	−0.455	0.255	−0.67**	0.111
Home educational resources (SES)	−0.06**	0.018	−0.065**	0.02	−0.085**	0.031	0.028	0.019	−0.02	0.019	−0.032	0.022
Mathematics achievement	0	0.001	0.003**	0.001	0.002	0.001	0.002	0	−0.001	0	0.001	0.001
School level
Mean home educational resources	−0.022	0.035	0.126	0.102	−0.038	0.065	0.142**	0.032	0.042	0.059	−0.009	0.059
Mean safe & orderly school score	−0.004	0.013	0.054	0.038	0.007	0.033	−0.006	0.023	0.077	0.044	0.021	0.034
Mean discipline & safety score	0.008	0.021	0.032	0.051	0.061	0.033	0.05	0.027	−0.001	0.021	−0.022	0.027
Random part
Level: School	2.611	0.494	0.45	0.115	2.597	0.659	1.041	0.269	1.651	0.286	3.609	0.636
Level: Student	37.697	1.291	11.568	0.387	21.125	0.887	5.113	0.263	27.364	0.908	18.96	0.819
−2*loglikelihood:	32,181		15,550		14,556		21,840		19,963		15,160
Number of schools	368		134		107		144		144		185
Number of students	7490		3847		3496		5288		4941		3709

** significant at 0.01 level (p < 0.01).

When we looked at predictors at the school level, we found that Romania was the only country where school MEANSES was a significant predictor of BR, with higher school SES predicting a higher value for self-reported BR, and thus less bullying. In Finland, achievement was predictive of BR. However, this result must be approached with caution, as the coefficients are small over all countries. Individual SES was a negative predictor in the United States, Finland and England, indicating that more affluence leads to lower value on the BR scale, so lower bullying resilience (more self-reported bullying). Not being born in another country had a small negative effect on BR, but only in England and Italy was this effect significant. Gender (BOY) was a negative predictor of BR for four out of six countries, with the USA and Italy negative but non-significant. In all countries, the variance at the school level was relatively low; only in Romania did the school-level variance go above 20%. Such a percentage is more in line with classic school effect studies, like Scheerens and Bosker’s (1997) meta-analysis of school effects (see also Sammons 2007), where schools accounted for 19% of the variance in unadjusted student achievement levels, reducing to 8% after controlling for initial differences between students. Additionally, the two school factors of Discipline and Safety and a Safe and Orderly Climate were not significant predictors of reported bullying in any of the countries studied.

Discussion and conclusions

The first finding was that overall levels of bullying did not differ markedly between countries and appeared unrelated to the extent of anti-bullying programmes or initiatives in these countries, in contrast to what we had hypothesised. We set out to answer two questions. Firstly, we wanted to explore, through multilevel models, how much of the variance in grade 8 students’ self-reported bullying can be explained at student and school levels. This showed that there were quite large differences in the school-effect between countries, with school-level variance lying between 4% in Finland and 27% in Romania. The United States was also at the lower end of the scale, with Korea, England and Italy taking up intermediate positions. This is interesting, as it does appear that the two countries with the most developed anti-bullying programmes saw lower levels of between-school variance, while the country with the least-developed understanding of bullying at national policy level saw the highest. However, on the basis of one dataset and not having access to other factors, it would be remiss to draw firm conclusions based solely on these data.

Secondly, we wanted to see what predictors at student and school level there were for self-reported bullying. The main predictor in all countries, except Italy, was gender, with boys reporting lower levels of bullying resilience (i.e. more bullying). In England and Italy, there were relatively strong negative effects on bullying resilience (BR) scores if a student was not born in the country, suggesting higher levels of reported bullying for these young people. In Finland, the United States and England, Home Educational Resources, our proxy indicator for SES at the student level, was a negative predictor. In addition, in Finland, maths achievement predicted the score for the BR scale, but the coefficient was small, and we only used one plausible value, so we must be cautious in interpreting this. For all six countries, variance of BR was mainly at the student level. The student level variables included in the models only explained a small proportion of variance. This is in accordance with most previous research, which has likewise shown only limited impacts of factors such as SES on the prevalence of reported bullying. Only in Romania did the school composition variable MEANSES, the mean of all students’ SES scales, turn out to be a significant predictor of reported bullying. The teacher and principal-reported scales on Discipline, Safety and a Safe and Orderly Climate were both non-significant predictors. This invites the conclusion that targeted programmes, aimed at individuals, might be more appropriate than classroom or school-level interventions, though it is important to note that some previous studies have pointed to the benefits of group approaches, so care is needed in the interpretation of these findings.

In relation to the other studies of school-level effects on bullying, this study paints a mixed and subtle picture. The school effect was weak to moderate, echoing other studies looking at school effects on non-cognitive variables, but it was significantly higher than the school effect reported by the only other study of secondary schools, by Galand, Hospel, and Baudoin (2014). There were also some country differences, which suggests that effects are not universal but may be mediated by each nation’s culture, systems or policies, although from this study we cannot say which of these is the key source of influence.

This study concurs with previous work showing a relatively limited relationship between bullying and SES (e.g. Ma 2001; Tippett and Wolke 2014; Woods and Wolke 2004) and confirms the differential relationship with gender (e.g. Ma 2002; Silva et al. 2013; Wang, Iannotti, and Nansel 2009). Our results only partly coincide with previous literature when it comes to the relationship between achievement and bullying, with one of the six countries’ mathematics achievement positively predicted bullying resilience, and so lower levels of bullying. Targets of bullying often suffer diminished academic achievement, whether measured by grades or standardised test scores (e.g. Nakamoto and Schwartz 2010). Another major difference with previous studies of primary schools was the lack of explanatory power of the school-level climate and behaviour variables. This could be a result of the secondary school as opposed to the primary school composition of the sample, and also of the way the variables are operationalised in TIMSS. There was only a weak relationship between country and the prevalence of reported bullying, though some indication that there was lower between-school variance in countries with more developed anti-bullying strategies. This conclusion must remain highly tentative in light of the lack of controls for other factors, but it does open up a very interesting avenue for further research. This study also suggests that research on bullying should employ a multilevel approach to recognise the structure of the data, taking the school level into account, since variance ranged between 4 and 27%, and so the hierarchical structure of the data should not be ignored. Methodologically, this study also shows the value of the rich datasets provided by ILSAs, to look at non-cognitive dependent variables, and depart from the standard focus on academic attainment outcomes.

There are some policy lessons to be drawn from this study. It provides further evidence that the gendered nature of bullying, or perceptions of bullying, requires attention, and that, in particular, boys may need more support in this area. Another noteworthy element that would deserve attention is the negative influence of SES. Of course, given the negative effects SES has on achievement (e.g. see Sirin 2005) this should not be seen as a plea to lower SES to counter the negative influence of bullying. As these analyses are correlational, caution is needed. However, we could contemplate the negative influences on students’ wellbeing caused by wealth. In particular, it might be useful to look into the influence of social media in the most affluent countries and a relationship with (cyber-)bullying. For example, data from seven countries in Europe show that the proportion of children aged 11–16 years who use the internet and who had experienced cyberbullying increased from 7% in 2010 to 12% in 2014 (UNESCO 2019). The lack of variables measuring cyberbullying in the dataset used makes it hard to draw strong conclusions on this, however.

There also are more methodological reasons to be somewhat cautious. Chalamandaris and Piette (2015) noted the wide variability of research methodologies employed to study bullying. This study extends that variability by using data from a large-scale assessment. Most of the measures used in this study are based on self-report, and as a result we cannot be sure whether the responses give a representative picture of what is actually experienced in schools (Branson and Cornell 2009; Sawyer, Bradshaw, and O’Brennan 2008; Smith 2016). This also relates to what occurs in the classroom. The role of the teacher in the classroom is substantial: teacher behaviours can ‘make or break’ bullying behaviour in the classroom (Veenstra et al. 2014). After all, a school can adopt a bullying programme, but if the teacher does not implement it in their classroom practice, students will not experience any change. If we see ‘behaviour management’ as a key feature of teaching quality, then this needs to be taken into account.

In the methodology section, we alluded to some limited missing data. We think it will have had a minor influence on at least the direction of the predictors in our study. We have also stressed the fact that these analyses are correlational in nature. Achievement can influence non-cognitive outcomes, but the direction of causation is likely to be bi-directional. To explore the causality of these relationships, it would be possible, in future studies, to use techniques such as Structural Equation Modelling (SEM), although the cross-sectional nature of datasets like those in TIMSS, PIRLS, and PISA gives rise to structural limitations. A further methodological point is that research on the relationship between school characteristics and bullying, including this study, is overwhelmingly cross-sectional in nature. This limits our ability to determine whether or not the school characteristics studied are correlates or causes of bullying prevalence. It would therefore be valuable for more longitudinal research to be done in this area. It is also, of course, the case that, unlike more recent scales, the TIMSS 2011 bullying scale that we used in this study does not include cyberbullying, which is now a very prevalent form of bullying.

Taken together, given the subtle nature of the findings, we are nonetheless convinced that analysing large-scale data, as we have done in this study, provides an important tool in the analyst’s toolkit to better understand the complexities of bullying in secondary schools. Our findings suggest some variance across countries, which should lead us to caution when importing policies and interventions across national boundaries, since these may not sufficiently match the context of the country.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. There are numerous reviews, e.g. Vreeman and Carroll (2007) and Hall (2017); also in low-and middle-income countries (Sivaraman, Nye, and Bowes 2019).

2. e.g. see http://www.kivaprogram.net/is-kiva-effective

3. We do not describe the broader education systems in the countries. For this we refer to a range of general sources, including UNESCO (https://education-profiles.org/), Eurydice (https://eacea.ec.europa.eu/national-policies/eurydice/national-description_en), and the OECD (https://www.oecd.org/education)

4. For example, section 9 in the AERA review (AERA, 2013) mentions several databases that include such programmes.

5. Details on all the scales at https://timssandpirls.bc.edu/methods/t-context-q-scales.html

6. Note that the tables with reliability coefficients only display 10 items.

7. http://www.bristol.ac.uk/cmm/software/mlwin

8. IGLS (iterative generalised least squares) is a maximum likelihood (ML) method based on estimating the random and fixed parts of the multilevel model alternately assuming the estimates for the other part are correct. This involves iterating between two GLS model fitting steps until the estimates converge to ML estimates.

Appendix A.

Four multilevel models for the six countries

Tables A1 to A6 present the four multilevel models for each of the six countries. Asterisks ** denote significant results at the 1% level.

Table A1. Four multilevel models for the United States

	Dependent variable BR - USA
	Model 1: null		Model 2: +student characteristics		Model 3: +school controls		Model 4: + school climate
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	10.094**	0.025	10.141**	0.033	10.141**	0.033	10.182**	0.038
Student level
Male student (BOY)			−0.073	0.046	−0.073	0.046	−0.111	0.054
Born elsewhere			−0.13	0.078	−0.13	0.078	−0.147	0.092
Home educational resources (SES)			−0.065**	0.015	−0.065**	0.015	−0.06**	0.018
Mathematics achievement			0	0.001	0	0.001	0	0.001
School level
Mean home educational resources					−0.001	0.028	−0.022	0.035
Mean safe & orderly school score							−0.004	0.013
Mean discipline & safety score							0.008	0.021
Random part
Level: School	2.845	0.457	2.839	0.451	2.839	0.451	2.611	0.494
Level: Student	37.728	1.175	37.506	1.167	37.506	1.167	37.697	1.291
−2*loglikelihood:	44,847		44,414		44,414		32,181
Number of schools	501		501		501		368
Number of students	10,338		10,252		10,252		7490

** significant at 0.01 level (p < 0.01).

Note on Table A1: In the United States, the null model the school level accounts for around 7% of the variance in bullying prevalence. When pupil level predictors are added, only SES, our indicator for socio-economic status, is a significant negative predictor; in other words, the more affluent students are the lower the score on reported BR levels, which indicates more self-reported bullying. After adding school controls and variables on school climate, gender appears significant predictor, with boys reporting lower levels of resilience to bullying, but this was not significant at the .01 level. The remaining predictors are not significant.

Appendix A2.

Table A2. Four multilevel models for Finland

	Dependent variable BR - FINLAND
	Model 1: null		Model 2: +student characteristics		Model 3: +school controls		Model 4: + school climate
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	10.528**	0.046	10.824**	0.058	10.824**	0.057	10.855**	0.058
Student level
Male student (BOY)			−0.54**	0.076	−0.536**	0.076	−0.572**	0.08
Born elsewhere			−0.345	0.221	−0.341	0.221	−0.372	0.239
Home educational resources (SES)			−0.062**	0.019	−0.061**	0.019	−0.065**	0.02
Mathematics achievement			0.003**	0.001	0.003**	0.001	0.003**	0.001
School level
Mean home educational resources					0.125	0.092	0.126	0.102
Mean safe & orderly school score							0.054	0.038
Mean discipline & safety score							0.032	0.051
Random part
Level: School	0.518	0.128	0.49	0.117	0.488	0.118	0.45	0.115
Level: Student	11.77	0.41	11.386	0.378	11.382	0.378	11.568	0.387
−2*loglikelihood:	17,208		16,997		16,995		15,550
Number of schools	145		145		145		134
Number of students	4223		4206		4206		3847

** significant at 0.01 level (p < 0.01).

Note on Table A2: In Finland, the school level accounts for less unexplained variance than in the United States, at just over 4%. Gender is a significant predictor, with boys reporting lower levels of resilience to bullying. SES and maths achievement are significant negative and positive predictors of BR.

Appendix A3.

Table A3. Four multilevel models for England

	Dependent variable BR - ENGLAND
	Model 1: null		Model 2: +student characteristics		Model 3: +school controls		Model 4: + school climate
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	10.418**	0.048	10.585**	0.064	10.584**	0.064	10.579**	0.065
Student level
Male student (BOY)			−0.222**	0.079	−0.223**	0.078	−0.227**	0.085
Born elsewhere			−0.584**	0.123	−0.586**	0.123	−0.625**	0.135
Home educational resources (SES)			−0.099**	0.03	−0.099**	0.03	−0.085**	0.031
Mathematics achievement			0.002	0.001	0.002	0.001	0.002	0.001
School level
Mean home educational resources					−0.018	0.055	−0.038	0.065
Mean safe & orderly school score							0.007	0.033
Mean discipline & safety score							0.061	0.033
Random part
Level: School	3.016	0.705	2.91	0.68	2.908	0.68	2.597	0.659
Level: Student	21.445	0.875	21.12	0.834	21.119	0.834	21.125	0.887
−2*loglikelihood:	15,864		15,727		15,726		14,556
Number of schools	118		118		118		107
Number of students	3805		3786		3786		3496

** significant at 0.01 level (p < 0.01).

Note on Table A3: In England, school level variance is 12% of the total unexplained variance. Gender, being foreign-born (born elsewhere) and SES are the main pupil level predictors. Again, boys report lower resilience to bullying, and the same is true of pupils not born in England. These variables only explain around 3% of total bullying prevalence.

Appendix A4.

Table A4. Four multilevel models for Romania

	Dependent variable BR – ROMANIA
	Model 1: null		Model 2: +student characteristics		Model 3: +school controls		Model 4: + school climate
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	9.654**	0.041	9.786**	0.052	9.811**	0.048	9.812**	0.049
Student level
Male student (BOY)			−0.236**	0.065	−0.229**	0.065	−0.23**	0.066
Born elsewhere			−0.665	0.331	−0.67	0.331	−0.598	0.33
Home educational resources (SES)			0.032	0.019	0.032	0.019	0.028	0.019
Mathematics achievement			0.002	0	0.002	0	0.002	0
School level
Mean home educational resources					0.152**	0.03	0.142**	0.032
Mean safe & orderly school score							−0.006	0.023
Mean discipline & safety score							0.05	0.027
Random part
Level: School	1.901	0.336	1.811	0.327	1.038	0.253	1.041	0.269
Level: Student	5.182	0.261	5.114	0.26	5.118	0.259	5.113	0.263
−2*loglikelihood:	22,498		22,201		22,178		21,840
Number of schools	147		147		147		144
Number of students	5420		5367		5367		5288

** significant at 0.01 level (p < 0.01).

Note on Table A4: In Romania, school attended is more strongly related to bullying prevalence than in the other countries in this sample. Gender is the main predictor, with girls reporting more resilience to bullying. This is one of the few countries where there is a school SES effect, demonstrated by the mean school SES being a significant predictor of self-reported bullying: higher school SES means higher BR, and thus less bullying. The variables explain 13% of total variance in Romania, more than in any of the other countries. This reduces the proportion of unexplained variance at the school level to 17%. All in all, school level factors seem to have differential effects on bullying in Romania.

Appendix A5.

Table A5. Four multilevel models for Korea

	Dependent variable BR – KOREA
	Model 1: null		Model 2: +student characteristics		Model 3: +school controls		Model 4: + school climate
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	10.269**	0.055	10.666**	0.057	10.668**	0.057	10.66**	0.058
Student level
Male student (BOY)			−0.812**	0.067	−0.816**	0.067	−0.807**	0.069
Born elsewhere			−0.498	0.252	−0.502	0.251	−0.455	0.255
Home educational resources (SES)			−0.022	0.018	−0.022	0.018	−0.02	0.019
Mathematics achievement			−0.001	0	−0.001	0	−0.001	0
School level
Mean home educational resources					0.06	0.057	0.042	0.059
Mean safe & orderly school score							0.077	0.044
Mean discipline & safety score							−0.001	0.021
Random part
Level: School	3.802	0.53	1.715	0.272	1.665	0.269	1.651	0.286
Level: Student	28.147	0.931	27.415	0.877	27.423	0.877	27.364	0.908
−2*loglikelihood:	21,056		20,836		20,835		19,963
Number of schools	150		150		150		144
Number of students	5160		5157		5157		4941

** significant at 0.01 level (p < 0.01).

Note on Table A5: In the null model, the school level accounts for 12% of unexplained variance in Korea, though the significant gender predictor explains 9% of total variance, and most of this is at school level (reducing unexplained variance at the school level to 5.6%). Only gender is a significant predictor of bullying, with boys reporting more bullying, less resilience to bullying.

Appendix A6.

Table A6. Four multilevel models for Italy

	Dependent variable BR – ITALY
	Model 1: null		Model 2: +student characteristics		Model 3: +school controls		Model 4: + school climate
Fixed part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
Constant	10.655**	0.045	10.743**	0.062	10.742**	0.061	10.734**	0.063
Student level
Male student (BOY)			−0.077	0.073	−0.077	0.073	−0.081	0.075
Born elsewhere			−0.66**	0.108	−0.661**	0.108	−0.67**	0.111
Home educational resources (SES)			−0.022	0.022	−0.022	0.022	−0.032	0.022
Mathematics achievement			0.001	0.001	0.001	0.001	0.001	0.001
School level
Mean home educational resources					−0.021	0.056	−0.009	0.059
Mean safe & orderly school score							0.021	0.034
Mean discipline & safety score							−0.022	0.027
Random part
Level: School	3.838	0.659	3.571	0.608	3.568	0.605	3.609	0.636
Level: Student	19.166	0.801	18.987	0.787	18.987	0.787	18.96	0.819
−2*loglikelihood:	16,143		16,058		16,057		15,160
Number of schools	197		197		197		185
Number of students	3937		3928		3928		3709

** significant at 0.01 level (p < 0.01).

Note on Table A6: Italy shows the second strongest ‘school effect’ on bullying, which accounts for 16.6% of variance in the null model. In contrast to the other countries in the study, gender is not a significant predictor. Being born outside of the country is a significant predictor though, and it explains less than 2% of total variance.

Appendix B.

Forest plot of predictors

Figure A1. Estimates of predictors for six countries with 95% confidence interval.