Does educational leadership enhance instructional quality and student achievement? The case of Austrian primary school leaders

ABSTRACT

School leadership ranks as an essential factor for successful schools. The purpose of this study is therefore to investigate the effects of two central leadership practices – setting directions and managing instructional processes – on both instructional quality (mediator) and student achievement (target) in Austrian primary schools. The data for this study originate from the 2018 national educational standards test in mathematics, allowing the use of information from 3,785 teachers and 73,780 students from 2,961 schools. In line with theoretical leadership effectiveness models, the results of multi-level structural equation analysis reveal systematic relations of instructional quality with student achievement. However, no direct or indirect effects of the two leadership dimensions on student achievement are apparent when simultaneous effects of student GPA and student background variables, such as highest parental occupational status, books at home, migration background, and language spoken at home, are controlled for. We discuss these results as well as previous findings with respect to methodological issues and contextual conditions that deserve more attention in future studies.

Introduction

During the last decades, many school systems around the globe have undergone far-reaching transformations in their governance structure. In the quest for effectiveness, equity, and high quality in education, policymakers launched concepts of New Public Management, emphasizing patterns of decentralization, market orientation, comparisons, and accountability (Ball, 1998; Skedsmo & Huber, 2019). Schools received more competencies for decisions at the operational level, but at the same time, they were made more responsible for their success or failure in the performance of their students. Due to the shift toward more autonomy, the range of tasks for school leaders has increased as well. Their sphere of responsibilities now includes not only administrative tasks but also organizational and personnel management, as well as supervising the school’s pedagogical programme (Brauckmann & Schwarz, 2015). In this scenario, the importance of principals for the success of a school is widely accepted.

Irrespective of the current consent regarding principals’ role for achieving high educational standards, previous leadership effectiveness studies mainly focus on a handful of education systems, as they originate from predominantly English-speaking countries. In a recently published meta-analysis by Liebowitz and Porter (2019), for example, only 9 out of 51 considered studies are located outside the United States. Consequently, the generalization of leadership effects to other geographic regions is limited.

The purpose of our study is therefore to investigate the contribution of school leadership practices to instructional quality and student achievement in Austrian primary schools. This study extends the current knowledge in the field of school leadership research by examining an education system which has followed an ‘international’ reform agenda of introducing output-oriented concepts of educational governance but differs significantly from other systems, for example, by endowing school principals with both management and teaching tasks. We thus follow the call of Grissom et al. (2021, p. 86) to investigate specific conditions under which leadership effects may or may not emerge. Our discussion of the obtained results focuses on resources that are needed to enhance leadership effects in the studied context, which may inform decisions of educational policy makers and stimulate principals’ self-reflexion. Austria, like other German-speaking countries and contrary to most Anglo-American school systems, represents a typical low-stakes system in which school leadership has undergone extensive changes during recent years. In support of this assumption, Kemethofer and Weber (2020) reported differences in leadership behavior between German- and English-speaking countries (see also, Klein, 2017).

Following Liebowitz and Porter (2019, p. 793), who recommended to ‘focus more on specific leadership practices rather than overarching styles or models’, we use setting directions and managing instructional processes as two core leadership practices identified in effectiveness research (e.g. Day et al., 2016; Leithwood, Sun et al., 2020). Our attempt to gain more insights not only on ‘what works’ in terms of specific leadership practices but also on ‘what works in which context’ is based on a multilevel analytic approach, controlling for confounding variables of outcome measures, i.e. of student achievement. Furthermore, we share the assumption of many previous leadership studies that the impact of principals on learning outcomes is indirect, i.e. mediated by other inner-organizational factors such as increased levels of teacher cooperation and instructional quality. Hence, our main research question is: Are setting directions and managing instructional processes as two core leadership practices systematically related to instructional quality (mediator) and student achievement (target) in Austrian primary schools?

Section 2 introduces characteristics of the Austrian school system that need to be considered when making assumptions about leadership effectiveness, for instance, the principals’ own teaching obligation. In Section 3 we elaborate different approaches to specify relations of educational leadership with organizational and pedagogical processes, and with student outcomes, theoretically. Moreover, we differentiate the dimensions of leadership behavior and instructional quality that are of prime interest to the present study. We continue by highlighting organization-specific features such as student composition that may affect both leadership behavior and achievement measures and therefore present important control variables in leadership effectiveness studies. We complement these considerations with fundamental methodological challenges of providing statistical evidence of leadership effects. In Section 4, we conflate the presented ideas in a conceptual model and basic assumptions that guide subsequent statistical analyses of representative data from Austrian primary schools.

School leadership in Austria

Traditionally, the education system in Austria was characterized by a bureaucratic-hierarchical organization. But in light of the country’s weak results in international comparative performance assessments (especially PISA and TIMSS), the Austrian education system has undergone extensive changes. Following international trends, a new governance architecture was introduced including performance standards, standard-related nationwide comparative assessments, a mandatory quality management for compulsory schools, and national education reports. Taken together, the reform agenda signals a shift toward more output orientation with increased accountability and a higher degree of autonomy (Altrichter, 2017). Despite these changes, Austria’s education system today ranks as a low-stakes system, which still grants comparably little autonomy for individual schools. For example, the school principals’ decision-making scope for budgetary resources is below the OECD average. Additionally, principals and teachers can modify binding national curricula only to a limited extent to create a school-specific pedagogical profile (OECD, 2016).

For a long time, a principal in an Austrian school – similar to Germany (cf., Warwas, 2012) – was deemed a primus inter pares. The position of a school leader resembled that of a teacher with additional administrative functions (Schratz, 2003; Stoll et al., 2008), and position holders mainly ‘served to implement official regulations as smoothly as possible’ (Rößler & Schratz, 2018, p. 283). In Austria, changes for school leaders went hand in hand with ‘a new culture of evaluation’ (Specht & Sobanski, 2012, p. 7), which gradually established the principal as the strategic head of the school and the immediate superior of all pedagogical and administrative staff working at the school. Accordingly, the tasks of a principal now include quality management, organizational, instructional, and personnel development,and external relationships. At the same time, he/she is responsible for the consistent implementation of laws and legal regulations, monitoring the instructional quality, and supervising the school’s pedagogical work. Still, until qualification programmes were revised in 2018/2019, training for Austrian principals was rudimentary (Brauckmann-Sajkiewicz et al., 2020). The selection criteria are described as advanced seniority (at least six years of teaching experience), a good record of accomplishment, and social integrity (Rößler & Schratz, 2018). Within four years after their appointment, candidates had to complete an in-service school management training with a workload equivalent of 12 European Credit Transfer System (ECTS) credits.

A particular difficulty for the Austrian school system is the age structure of principals. In 2012 the national education report provided evidence of ‘overaging’ (Schratz et al., 2016). Although no current data are available, it can be assumed that many principal positions have been filled in recent years. However, novice principals often face a number of challenges and therefore need time to work in the job. Among the difficulties in their new role are coping with an established culture in the school and introducing their own ideas about the school and teaching (Walker & Qian, 2006). In this scenario, Eder et al. (2011) describe an existing tradition of teacher autonomy with respect to instruction. In a sample of 537 teachers, 36% represented the autonomy–parity pattern and another 27% described themselves as ‘lone fighters’, whereas only 36% belonged to the group of teachers with a clear team orientation.

In addition to the above-mentioned tasks, principals in Austria also have a partial obligation to teach in small schools with fewer than eight classes. Law regulates the exact extent of teaching duties, which may be up to 18 hours per week. Besides the school size, the existence of afternoon care groups, affiliated special schools and the number of children with special educational needs have an impact on the total amount of teaching hours (Heißenberger, 2019; Schratz, 2016). According to Heißenberger’s survey and analysis of policy documents (Heißenberger, 2019), the main arguments in favor of a teaching obligation are that regular teaching practice supports the evaluation of teachers and that principals do not lose touch with the ‘core operations’ of an educational institution. Some interviewed persons, on the other hand, see no relevance of mandatory teaching for leadership tasks. In primary schools, which form the basis of our study, about two out of three principals fall into this category of leading a school whilst having teaching duties. In fact, the average number of classes in an Austrian primary school is marginally above six (Statistik Austria, 2020).

To sum up, several context specificities affect the ways in which ideal conceptions of effective educational leadership can be realized in the focal school system. Designing the instructional program is often reduced to implementing and supervising binding national curricula rather than a prolific area of activity for Austrian school leaders. Own restraints, a late introduction of preparation programs, as well as long-standing school cultures impede the necessary role change from an administrator and equal colleague to an inspiring and motivating leader. Giving mandatory lessons does not guarantee an individual ability to stimulate school-wide instructional improvements but certainly restricts time for managerial tasks such as planning, coordination and evaluation.

Modeling and measuring school leadership effects

Starting in the 1980s, models of leadership effects appeared in educational research that summarized empirical evidence to illustrate the conditions of successful leadership and its relationship to crucial processes in schools (e.g. Bossert et al., 1982). Since then, research results led to refinements and extensions of existing models as well as the emergence of new approaches (Gumus et al., 2018). Hallinger and Heck (1996) identified different assumptions about the impact of school leadership. Essentially, they distinguish between models that assume a direct effect of principal behavior on student performance and models that postulate indirect effects in which intermediate variables transmit leadership effects.

There are a number of reviews and meta-analyses in which the statistical effect of school leaders on student achievement has been evaluated systematically (e.g. Hallinger & Heck, 1996; Liebowitz & Porter, 2019; Marzano et al., 2005; Robinson et al., 2008; Scheerens, 2012; Witziers et al., 2003). In sum, most of these studies indicated relatively small total effects on achievement measures. Thus, a prevailing view among educational researchers today is that principals unfold most of their effects on learning outcomes indirectly, through a (context-specific) set of four core leadership practices: 1) building a vision and setting directions, 2) taking care and developing the personnel, 3) (re)designing the organization, and 4) managing instructional processes (Day et al., 2016; Hallinger, 2011; Grissom et al., 2021; Leithwood et al., 2008; Leithwood, Harris et al., 2020; Leithwood, Sun, et al., 2020).

Day et al. (2016) used a longitudinal mixed-methods approach to investigate the relationship between leadership and student outcomes. They conclude that successful leaders respond to the needs and possibilities of the school by focusing on improving the quality of teaching and learning. This is achieved through their influence on structures and culture. Similarly, Bruggencate et al. (2012) report school leadership effects on student engagement and average exam scores that were mediated by development orientation and classroom practices. Their findings indicate a strong relationship between leader behavior and a school’s culture. Drawing on quantitative data from Germany, Pietsch and Tulowitzki (2017) found effects of leadership on essential elements of a good school, such as collaboration, commitment, and job satisfaction. Additionally, their analysis indicates effects of instructional leadership on teachers' classroom management and student orientation.

To sum up: School leadership practices aim to enhance essential pedagogical activities of the teaching staff. More precisely, they aim to boost the quality of instructional processes, such as those postulated by Praetorius et al. (2018): classroom management, cognitive activation, and needs-based support. These instructional processes are most proximally related to the students’ learning success (e.g. Seidel & Shavelson, 2007) and shaped not only through a principal’s direct involvement in the school’s instructional processes but also through the conditions and visions he/she creates for these processes. In this study, we use a conceptual framework that includes essential elements of established leadership models. Figure 1 depicts the structure of this model, while the following sections of the paper elaborate the content and function of its constitutive elements.

Figure 1. Conceptual framework used in this study.

Predictor variables: leadership behavior

Setting directions addresses core elements of transformational leadership and assumes that school leaders who succeed in establishing goals and high expectations stimulate changes in instructional practices. School leaders decide which goals are set, communicate them, gain the commitment of those responsible for achieving these goals, and ensure that the teachers’ work is in line with the school’s aims (Leithwood, Sun, et al., 2020; Pont et al., 2008). Through their words and deeds, school leaders inspire the members of their organization and thus ensure that the values and moral purpose of their staff align with the overall vision of the school. By supporting teachers appropriately, their commitment and motivation to reach the organizational goals also increase (Leithwood & Sun, 2012). In addition, such goals help teachers to make sense of their work (Leithwood et al., 2004). Hence, school leaders stimulate student achievement indirectly through their impact on teachers’ work attitude and morale (Luyten & Bazo, 2019). In high-performing schools, there are high expectations and a broad consensus on goals. Moreover, principals ensure that professional learning opportunities for teachers match these goals (Grissom et al., 2021). Consequently, Leithwood et al. (2004, p. 6) argue that setting directions accounts ‘for the largest proportion of a leader’s impact’.

Managing instructional processes summarizes practices that primarily emphasize teaching and are therefore highly relevant to the learning progress of students. This includes to ‘align the strategies and activities of the school with the school’s academic mission.’ (Hallinger, 2005, p. 224). The related set of leadership practices consists of planning, coordination, and evaluation of teaching as well as developing the personnel (Robinson, 2010; Robinson & Gray, 2019). It also includes activities such as classroom visits, providing feedback, coaching and supporting teachers, promoting opportunities for professional learning, and establishing a data-driven instructional program (Grissom et al., 2021). According to Robinson et al. (2008), successful school leaders provide formal conditions (staff meetings and professional development programs) as well as informal conditions (discussions about specific teaching problems) to engage in teaching-related interactions. Although a literature synthesis by Grissom et al. (2021) points out that not all instruction-related activities have singular positive effects, students generally and strongly benefit from school leaders who pay attention to instructional quality.

Mediating variables: facets of instructional quality

As set out above, common models that postulate a principal’s indirect contributions to learning outcomes connect leadership behavior to organizational processes that again lead to high-quality instruction and thus promote student performance. Brauckmann and Pashiardis (2011, p. 15) mention a set of prominent mediating variables on the school and classroom levels that are affected by the work of principals. Among these variables are a climate that is orderly and conducive to learning, evaluation and feedback practices, job satisfaction and commitment to the school, and learning opportunities for the staff. But they also include the consistent implementation of specific, high-quality teaching strategies.

The present study concentrates on instructional quality as one particularly important path through which leadership is linked to outcomes (e.g. Bryk et al., 2010; Hallinger, 2011). This mediator plays a prominent role, since it contains variables that are demonstrably most proximally related to students’ academic success (Seidel & Shavelson, 2007). As Creemers and Kyriakides (2012) point out, effective learning and strong learning outcomes are the ultimate aims of any school. Accordingly, school leadership is expected, first and foremost, to improve instructional practice. Generic conceptual frameworks of instructional quality often demarcate three basic quality dimensions (Künsting et al., 2016; Praetorius et al., 2018; Schlesinger & Jentsch, 2016) with several inherent facets that also appear in Figure 1 above.

Firstly, classroom management comprises teacher actions that maximize students’ time on task and thus ensure an orderly learning environment free of disruption. High levels of discipline and attention arise, for instance, from teachers’ communication of clear rules and their monitoring of student activities.

Secondly, the multifaceted construct of student learning support entails various approaches to meet students’ basic psychological needs and thus approaches to foster self-regulated learning. To support students’ experiences of competence, teachers provide differentiated and adaptive instruction, align the pacing of instruction to the present learner group, and give constructive feedback. To enhance students’ experience of autonomy, teachers allow individual choices among (differentiated) tasks and create learning material that is of practical relevance and interest to students’ lives. To foster experiences of social relatedness, teachers demonstrate openness toward students’ opinions and contributions whilst encouraging the students to treat each other in a friendly, considerate, and helpful way. All these means aim to establish a warm and trustful learning climate.

Thirdly, cognitive activation results from cognitively challenging tasks, questions, or even demanding problems. Furthermore, teachers who take up and put students’ prior knowledge to the test, elicit and continually explore students’ lines of thinking, or stimulate discursive and co-constructive learning activities also take effective measures to foster students’ deep elaboration and understanding of the instructional content.

Control variables: student background characteristics

When examining school leadership effects empirically, local contingencies of leadership practices and outcomes deserve close attention. School leaders must adapt their behavior to the context of their school. As Hallinger (2018) points out, educational policy, the school’s environment, and the student’s economic, social, and cultural status represent some of the most prominent contextual factors. Moreover, these factors affect intermediate (classroom) variables and learning outcomes (Maeyer et al., 2007). Although the impact of school contexts is undisputed, it is often too complex to specify them adequately in empirical studies (Sebastian et al., 2017). A frequently used context factor is therefore the aggregated socio-economic status of the students (Maeyer et al., 2007).

We also follow the recommendation of Reynolds et al. (2016) to use an adequate sample and to control for socio-economic contexts and other relevant variables. We address this issue by drawing on a dataset that comprises all primary schools in Austria and controlling for student GPA and student background variables.

Basic assumptions for testing the conceptual model

When examining if the two essential leadership practices of setting directions and managing instructional processes systematically relate to instructional quality and student achievement in Austrian primary schools, we assume that both of these practices as well as a general leadership factor have an impact on the quality of instructional processes. Instructional quality, in turn, is expected to have a positive effect on learning outcomes. As depicted in Figure 1, our study accounts for several facets of instructional quality that obviously correspond to the three basic quality dimensions set out above. For example, practical orientation and differentiation indicate aspects of student learning support, problem solving falls into the category of cognitive activation, and perceived discipline refers to classroom management. Like other studies, we control for the socio-economic composition of the student body which demonstrably affects both teaching practices and learning outcomes.

We expect that when examining principals’ impact on instructional quality and student achievement in Austrian schools, consistently positive yet small indirect statistical effects of leadership will appear. This assumption not only conforms with the majority of available meta-analyses on leadership effectiveness cited above but is also grounded in the following rationale: The principals’ comparably limited decision-making scope and autonomy, decades of rudimentary qualification, and long-standing school cultures in which the principal acted simply as the primus inter pares might dampen leadership effects on core pedagogical processes that are traditionally shaped primarily by each teacher’s own decisions on instructional design. In addition, Austria ranks among the countries with the strongest relation between the socio-economic status of the students and their academic performance (OECD, 2019, p. 57).

Methodology

Some studies (e.g. Grissom et al., 2015; Maeyer et al., 2007) demonstrated that the effect of school leadership on student achievement depends on the choice of the specified model. We therefore concentrate on theoretically sound and (according to the above-mentioned meta-analyses) empirically robust constituents of successful leadership and mediating variables of instructional quality. We further comply with Scheerens’s call (Scheerens, 2012, p. 137) to use multi-level structural equation modeling (see also, Grissom et al., 2021, p. 86). This is done to account for the nested structure of the data (students within teachers, teachers within schools) and the measurement error. It also serves to model the theoretically assumed relations at the correct level of analysis, i.e. specifying effects of instructional quality at the group level. Furthermore, we draw on external assessments for all relevant constructs to prevent self-serving biases in reports of individual practices as a principal or teacher, to coalesce multiple views on particular behaviors, and to ignore divergent teacher- or school-specific norms of rating student achievement. That is, teachers evaluate leadership behavior, students rate instruction, and student achievement is assessed by standardized performance tests.

The data for this study originate from the national, standardized Austrian student assessment in mathematics conducted in 2018. All public and private schools at the primary level with students in grade 4 participated in the compulsory tests.¹ Additionally, all students and teachers filled out questionnaires on context information.² The student questionnaire was provided as a paper-pencil questionnaire after the test, and the questionnaire for teachers was administered as an online survey. Students were asked, among other things, to rate the instructional quality of teachers and their own socio-economic background. The questionnaire for teachers included items on the perceived behavior of the principal (BIFIE, 2019). For the present analysis, data are available for 3,785 teachers and 73,780 students from 2,961 schools. To increase validity, we use external evaluations only. That is, we model leadership behavior as perceived by teachers instead of using principals’ self-evaluations of their behavior. We argue that external evaluations are more valid than self-ratings, as self-ratings are more likely prone to socially desirable responses, i.e. teachers are more likely rate their instructional quality higher than it is. Student ratings are considered to be more objective in this regard (e.g. Kunter & Baumert, 2007). Moreover, the use of student ratings also allows us to draw on multiple judgments and thus to control for divergent, unreliable student judgments by using the mean student judgment as a measure of teaching quality.

School leadership

Eight items relate to leadership behavior. Four of them represent setting directions and inform about a principal’s effort to define, discuss, and ensure the achievement of educational goals (e.g. ‘The school leader defines goals that must be achieved by the teachers of this school’.). Another four items assess managing instructional processes (e.g. ‘The school leader gives the teachers suggestions to improve the teaching’.) and focus on classroom observations, providing feedback, and discussing teaching objectives. Teachers rated all statements on a four-point Likert-type scale ranging from (1) never to (4) very often.

Research offers heterogeneous approaches to model school leadership statistically. As described above, scholars of the field claim several conceptually distinct dimensions of school leadership. However, few studies have investigated whether this assumption holds in the light of empirical data structures. For instance, Pietsch and Tulowitzki (2017) found that a bifactor model (assuming a general factor and sub-dimensions) was the only model that adequately represented school leadership practices. The bifactor model was superior to a range of ‘correlated-factors’ models that are commonly applied in the light of high empirical intercorrelations between the theoretically differentiated facets of school leadership in previous studies (e.g. Bruggencate et al., 2012; Pietsch & Tulowitzki, 2017). These high correlations cause problems of multicollinearity when predicting outcomes of different leadership dimensions. Since bifactor models assume a general factor (g-factor) that is uncorrelated with (orthogonal to) the specific factors (s-factors) and that partials out the common variance of the s-factors (leading to low associations between the s-factors), multicollinearity is no longer an issue. For these reasons, we follow Pietsch and Tulowitzki (2017) and apply the bifactor model. As these researchers point out, this approach ‘assumes that principals can exhibit domain-specific leadership behaviour [(s-factor), ibid.] that is independent from a global (g) factor dubbed “leadership core” as well as active leadership on its own’ (Pietsch & Tulowitzki, 2017, p. 11). Moreover, it is expected that the s-factors exert incremental effects on covariates beyond the effects of the g-factor.

Given that we use teachers’ evaluation of school leadership behavior, we face nested data (i.e. teachers within schools). According to methodological literature (Lüdtke et al., 2011; Marsh et al., 2009; Stapleton & Johnson, 2019), any disagreement among the raters of a particular principal could be interpreted as measurement and sampling error; what counts is the agreement (i.e. the opinion shared by all teachers in a school). Many studies follow this idea and apply a modeling approach that accounts for the nested data and the measurement and sampling errors by specifying a so-called shared construct (e.g. Marsh et al., 2009; Stapleton & Johnson, 2019).

However, for several reasons, we refrain from modeling school leadership as a shared construct at the school level: First, the intraclass correlation (ICC) of the items used to assess the two dimensions of school leadership is quite low in our dataset (10 to 12%), indicating that up to 90% of teachers’ evaluation of the school leader’s behavior varies within schools, which is between the teachers of the same organizational unit. For this reason, a construct of shared opinion about the principal as the driver of a teacher’s instructional activities is not commensurable with the present data. Rather, we assume the main influencing factor to be each teacher’s individual perception of leadership behavior. Second, modeling school leadership at the school level would imply a three-level model, since school leadership, instructional quality, and student achievement are located and assessed at three different levels. This would be too complex, i.e. too many parameters would have to be estimated. Third, modeling leadership behavior at the school level would force us to specify its effects on instructional quality on the school level correspondingly. But such a statistically consistent model is hardly compatible with plausible and recurring findings that instructional quality varies greatly between different teachers (even within a school) and thus represents a class-level construct. Hence, instructional quality at the school level has limited meaning. From an empirical point of view, very low ICC values (1 to 3%) of the items on instructional quality in our own dataset further corroborate that these items do not vary at the school level, rendering quality differences a phenomenon that exists mainly between teachers and not the investigated schools as ‘internally homogeneous’ units. For these reasons, we stick to modeling teacher ratings of leadership as single-level bifactor models, located at the class level just like instructional quality differences. Table 1 lists the psychometric properties of the bifactor model based on the two leadership scales.

Table 1. Psychometric properties of the leadership and instructional variables

Model	# Items	# Par	Chi^2	df	Chi^2/df	CFI	TLI	RMSEA	SRMR	Mean	SD	Alpha	Omega
School leadership
Setting directions	4	32	28.43	12	2.37	.984	.964	.015	.026	2.912	0.611	0.774	0.800
Managing instructional processes	4									2.748	0.523	0.683	0.680
Instructional quality
Practical orientation	4	22	10.73	2	5.37	1.000	.999	.008	.015	1.49	1.27	0.792	0.960
Cognitive activation – elaboration	4	22	19.81	2	9.91	1.000	.998	.011	.024	1.34	1.06	0.698	0.888
Differentiation	3	15	0.00	0	-	1.000	1.000	.000	.000	0.82	1.87	0.869	0.960
Perceived clarity	3	15	0.01	0	-	1.000	1.000	.000	.001	1.74	1.22	0.636	0.889
Cognitive activation – problem solving^a	3	15	0.00	0	-	1.000	1.000	.000	.000	1.50	1.84	0.884	0.985
Support of meta-cognition	3	15	0.00	0	-	1.000	1.000	.000	.000	1.30	1.77	0.893	0.984
Support of motivation^a	3	15	0.00	0	-	1.000	1.000	.000	.000	1.63	1.42	0.742	0.949
Consideration of students’ voice	4	22	172.99	2	86.50	.999	.994	.034	.022	1.17	1.87	0.908	0.967
Perceived discipline	3	14	1.80	1	1.80	1.000	1.000	.003	.002	1.45	1.60	0.878	0.986
Perceived pace of teaching	2	22	10.73	2	5.37	1.000	.999	.008	.015	1.91	1.73	0.874	0.991

^aError variance of one indicator item was fixed to zero for convergence reasons. While alpha assumes a) that each indicator variable contributes equally to the factor, i.e. same loadings (tau-equivalence) and b) that the error variances of the indicators are uncorrelated, omega is based on neither assumption. Omega represents a (hierarchical) coefficient that gives the proportion of variance in scale scores accounted for by a factor (McDonald, 1999). Regardless of whether alpha or omega is used, low values indicate that it makes no sense to compute a sum score.

Instructional quality

The student questionnaire included 39 items related to instruction. Based on research about instructional quality and teacher effectiveness (e.g. Baumert et al., 2010; Kunter et al., 2013), we selected 32 items referring to 10 distinguishable aspects of desirable instructional practices which themselves are well compatible with the three overarching ‘basic dimensions’ of instructional quality we outlined in chapter 2 (see, Table 1): practical orientation (e.g. ‘The teacher explains mathematics by examples from everyday life’.), cognitive activation – elaboration focus (e.g. ‘The teacher gives tasks that make us think’.), differentiation (e.g. ‘We work in groups, and each group has different tasks’.), cognitive activation – problem solving focus (e.g. ‘The teacher is interested in the fact that we can explain how we solve tasks’.), perceived clarity (e.g. ‘In class, we are clear about what we are supposed to do’.), support of meta-cognition (e.g. ‘The teacher tells us how we can improve’.), support of motivation (e.g. ‘The teacher makes the lessons really exciting’.), consideration of students’ voice (e.g. ‘I get to choose whether I work alone, in pairs, or in a small group’.), perceived discipline (e.g. ‘We can always work in a concentrated way’.), and perceived pace of teaching (e.g. ‘The lessons are designed so that we can follow along well’.). Students rated all items on a four-point Likert-type scale, with the response categories ranging from (1) in every lesson to (4) never or almost never. For easier interpretation of the results, we inverted these items. To account for the hierarchical structure of the data (students nested in classes), the measurement, and sample error, we specified one separate shared construct for each of the 10 instructional quality dimensions (Stapleton & Johnson, 2019). See, Table 1 for the psychometrics properties of the scales (model fit, descriptive statistics, and reliability indices).

Student achievement

The outcome variable is student achievement, which is represented by the performance in mathematics. As part of the educational standards test in mathematics, every student had to solve about 70 items divided into two test booklets of 40 minutes each. Each test booklet roughly had the same difficulty including multiple-choice and open-ended items. For comparison purposes, the test scores were transformed to a scale similar to PISA. On average, students scored 551 points with a standard deviation of 99 points (BIFIE, 2019). Further information on the data (e.g. composition of students) and the instrument used to assess students’ mathematics achievement is provided in the national results report (BIFIE, 2019) and by Breit and Schreiner (2016).

Control variables: composition of the student body of a class and student grades

To account for the social composition of the student body of a class, we included measures that represent established indicators of students’ socio-economic status in our statistical model. These indicators are the highest parental occupational status (HISEI), books at home, migration background, and language spoken at home. Moreover, we included students’ school grades to control for a class average academic achievement level.

Analytic procedure

As argued above, given the nature of the available data (low school level variability of the indicators) and our theoretical considerations (assessing instructional quality and its effects as constructs that vary primarily between classes), it is more appropriate to concentrate on the teacher and student levels in the present study. To investigate direct and indirect effects of school leadership, we perform multilevel structural equation models. More concretely, we perform a separate multilevel mediation analysis for each of the 10 instructional dimensions that presumedly play a mediating role between school leadership and student achievement.

All analyses were conducted using the R package ‘MplusAutomation’ (Hallquist & Wiley, 2018) in combination with Mplus (Muthén & Muthén, 1998–2017). Since the BIFIE provided full census data, no weighting was needed.

Use of plausible values: Students’ mathematics performance is generated by multiple imputations based on the assumption that these values are conditionally distributed depending on student test data (item responses) and other student data (covariates; Foy et al., 2012; Robitzsch et al., 2016). The plausible values provided by the BIFIE are then evaluated using the pooling method (Rubin, 1987). This was performed by the Mplus command TYPE = IMPUTATION.

Missing data: All variables used except mathematics performance have missing values. The item with the largest share of missing values (11%) is ‘We work in groups, and each group has different tasks’. Subsequent analyses apply the Full Information Maximum Likelihood (FIML) method. ‘In this method, all students are included for an analysis and the missing data is “integrated out”’ (Robitzsch et al., 2016, pp. 290 f.). A central assumption of the FIML procedure is that, considering all variables in the analysis, missing values are randomly distributed (i.e. independent of the levels of other variables in the analysis). Since our analyses include student achievement and student socio-economic background, we believe that we do consider central variables that are predictive for missing student data.

To determine the model fit, we used common cutoff criteria (Hu & Bentler, 1999; Little, 2013) – Bentler’s comparative fit index (CFI ≥ .90), the Tucker–Lewis index (TLI ≥ .90), the root mean square error of approximation (RMSEA ≤ .08), and the standardized root mean square residual – at both the student and class levels (SRMR L2 ≤ .10).

Results

All measurement models (Table 1) and all structural models (Table 2) have an adequate model fit. Most of them even have a very good model fit, as indicated by meeting or surpassing the criteria mentioned above (CFI, TLI > .95, RMSEA, SRMR < .05). Only the model ‘perceived clarity’ conveys a TLI value below the cutoff. However, all other fit indicators point again at a well-fitting model for this mediator.

Table 2. Model fit indices

	Models without control variables								Models controlling for the social composition of the student body
Model	# Par	Chi^2	df	Chi^2/df	CFI	TLI	RMSEA	SRMR L2	# Par	Chi^2	df	Chi^2/df	CFI	TLI	RMSEA	SRMR L2
1	64	1445.58	55	26.28	.986	.977	.019	.030	104	1845.60	119	15.51	.983	.972	.014	.028
2	64	3677.55	55	66.86	.942	.907	.030	.037	104	4455.30	119	37.44	.931	.889	.022	.041
3	55	1876.15	45	41.69	.984	.974	.023	.025	95	2173.15	101	21.52	.982	.970	.017	.021
4	55	3282.01	45	72.93	.916	.866	.031	.054	95	3790.52	101	37.53	.910	.850	.022	.045
5	55	1519.65	45	33.77	.988	.981	.021	.024	95	1865.31	101	18.47	.986	.977	.015	.018
6	55	2039.83	45	45.33	.985	.976	.025	.026	97	2353.35	99	23.77	.983	.972	.018	.021
7	55	1975.35	45	43.90	.970	.952	.024	.035	95	2355.00	101	23.32	.967	.944	.017	.026
8	64	1795.04	55	32.64	.991	.986	.021	.023	104	2077.85	119	17.46	.990	.984	.015	.018
9	53	1425.29	47	30.33	.987	.981	.020	.025	93	1616.98	103	15.70	.987	.979	.014	.017
10	49	2541.72	34	74.76	.965	.941	.032	.029	89	2762.64	82	33.69	.965	.937	.021	.019

Models 1 to 10 differ only with respect to the mediator variable: 1 practical orientation, 2 cognitive activation – elaboration, 3 differentiation, 4 perceived clarity, 5 cognitive activation – problem solving, 6 support of meta-cognition, 7 support of motivation, 8 consideration of students’ voice, 9 perceived discipline, 10 perceived pace of teaching. # Par = number of parameters, Chi² = chi square value, df = degrees of freedom, CFI = comparative fit index, TLI = Tucker–Lewis index, RMSEA = root mean square error of approximation, SRMR = standardized root mean square residual, L2 = level 2 (here, class level).

Direct effects

In Table 3, we report the direct effects of school leadership and the instructional quality on students’ mathematics achievement in grade 4. The findings (see the left-hand side of Table 3) suggest that school leadership only very weakly predicts students’ math achievement beyond the effects of instructional quality. While the global factor of leadership shows small but significant positive effects – slightly varying between β = .049 and β = .066, depending on which instructional quality dimension is included in the model – the specific school leadership factor ‘managing instructional processes’ shows a small but significant negative effect, slightly varying between β = −.073 and β = −.106, depending on which instructional quality dimension is included in the model. However, these effects vanish if students’ socio-economic background is controlled for (see the right-hand side of Table 3).

Table 3. Standardized direct effects (β) of the three school leadership factors and instructional quality on students’ mathematics achievement

	Models without control variables								Models controlling for the social composition of the student body
Model	Global	p	SetDir	p	InstPro	p	InstQua	p	Global	p	SetDir	p	InstPro	p	InstQua	p
1	.059	.007	−.001	.961	−.101	.000	.143	.000	.019	.264	.009	.686	−.043	.079	.090	.000
2	.060	.007	.001	.980	−.106	.000	.128	.000	.019	.263	.009	.669	−.044	.069	.106	.000
3	.049	.022	.002	.936	−.099	.001	.220	.000	.015	.382	.013	.566	−.043	.077	.106	.000
4	.058	.008	−.002	.927	−.092	.002	.210	.000	.019	.261	.008	.732	−.038	.120	.136	.000
5	.059	.006	−.009	.709	−.089	.002	.318	.000	.027	.106	−.002	.923	−.041	.091	.193	.000
6	.058	.008	−.002	.952	−.103	.000	.178	.000	.018	.289	.010	.657	−.044	.072	.101	.000
7	.055	.010	.002	.941	−.073	.013	.341	.000	.019	.264	.011	.616	−.031	.209	.188	.000
8	.050	.019	.000	.988	−.098	.001	.246	.000	.023	.165	−.002	.900	−.046	.060	.118	.000
9	.066	.001	−.018	.347	−.027	.164	.384	.000	.027	.103	−.003	.851	−.001	.924	.241	.000
10	.066	.002	−.023	.240	−.031	.124	.430	.000	.027	.101	−.002	.915	−.006	.715	.239	.000

Models 1 to 10 differ only with respect to the mediator variable: 1 practical orientation, 2 cognitive activation – elaboration, 3 differentiation, 4 perceived clarity, 5 cognitive activation – problem solving, 6 support of meta-cognition, 7 support of motivation, 8 consideration of students’ voice, 9 perceived discipline, 10 perceived pace of teaching. Global = global school leader factor, SetDir = specific school leader factor ‘setting directions’, InstPro = specific school leader factor ‘managing instructional processes’, InstQua = instructional quality, p = p value.

In contrast to the many non-significant direct effects of school leadership, all dimensions of instructional quality are weakly to moderately related to students’ mathematics achievement (β = .128 to .430, p < .000). The strongest association is between ‘perceived pace of teaching’ and math achievement (β = .430, p < .000), and the smallest between ‘cognitive activation – elaboration focus’ and math achievement (β = .128, p < .000). These findings are stable, even if students’ background (e.g. highest parental occupational status, language spoken at home, books at home) and grades are controlled for, though they slightly decrease in their magnitude.

In Table 4, we report the standardized effects of the three school leadership factors on instructional quality in mathematics in grade 4. Neither the global factor of school leadership nor its specific factors are significantly related to instructional quality, with few exceptions: The specific factor ‘instructional leadership’ is significantly and negatively related to ‘support of motivation’ (β = −.104, p = .006), ‘perceived discipline’ (β = −.086, p < .000), and ‘perceived pace of teaching’ (β = −.069, p = .010). However, the effect size is negligible for all three associations.

Table 4. Standardized effects (β) of the three school leadership factors on instructional quality in mathematics

	Models without control variables						Models controlling for the social composition of the student body
Model	Global	p	SetDir	p	InstPro	p	Global	p	SetDir	p	InstPro	p
1	−.003	.915	.035	.243	−.046	.216	−.019	.450	.047	.149	−.028	.448
2	−.009	.766	.018	.586	−.008	.855	−.017	.533	.031	.396	.000	.999
3	.047	.063	.005	.853	−.038	.281	.030	.228	−.001	.985	−.017	.630
4	.002	.944	.029	.396	−.077	.057	−.015	.586	.040	.270	−.055	.168
5	.001	.960	.039	.153	−.060	.091	.001	.984	−.018	.524	−.043	.281
6	.009	.744	.026	.360	−.028	.433	−.003	.892	.029	.368	−.012	.730
7	.015	.596	.002	.957	−.104	.006	−.006	.825	.007	.842	−.076	.040
8	.039	.116	.014	.614	−.040	.251	.031	.179	−.022	.378	−.017	.639
9	.021	.408	−.006	.796	−.086	.000	−.002	.942	−.015	.558	−.071	.001
10	.017	.532	.007	.760	−.069	.010	−.005	.848	−.024	.376	−.054	.028

Models 1 to 10 differ only with respect to the mediator variable: 1 practical orientation, 2 cognitive activation – elaboration, 3 differentiation, 4 perceived clarity, 5 cognitive activation – problem solving, 6 support of meta-cognition, 7 support of motivation, 8 consideration of students’ voice, 9 perceived discipline, 10 perceived pace of teaching. Global = global school leader factor, SetDir = specific school leader factor ‘setting directions’, InstPro = specific school leader factor ‘managing instructional processes’. p = p value.

Indirect effects

In Table 5, we report the unstandardized indirect effects of the school leadership factors on students’ mathematics achievement before and after controlling for the social composition of the student body of a class. As can be seen, there are only three statistically significant and negative indirect effects of the specific factor ‘managing instructional processes’ via ’support of motivation’ (b = −.020, p = .006; controlling for the social composition of the student body: b = −.007, p = .045), ‘perceived discipline’ (b = −.020, p < .001; b = −.008, p = .002), and ‘perceived pace of teaching’ (b = −.010, p = .011; b = −.006, p = .032). Again, these effect sizes are weak and negligible. This finding contradicts our assumption that school leadership (as perceived by teachers) is related to student math achievement via instructional quality (as perceived by students).

Table 5. Unstandardized indirect effects (b) of the three school leadership factors via instructional quality on students’ mathematics achievement

	Models without control variables						Models controlling for the social composition of the student body
Model	Global	p	SetDir	p	InstPro	p	Global	p	SetDir	p	InstPro	p
1	.000	.915	.000	.260	.000	.223	−.001	.455	.002	.180	−.001	.453
2	.000	.766	.000	.588	.000	.856	−.001	.535	.002	.405	.000	.998
3	.010	.067	.000	.853	.000	.279	.002	.239	.000	.985	−.001	.629
4	.000	.944	.000	.401	−.010	.061	−.001	.588	.003	.285	−.004	.181
5	.000	.960	.010	.157	−.010	.090	.000	.984	−.002	.526	−.004	.280
6	.000	.744	.000	.366	.000	.431	.000	.893	.001	.374	−.001	.730
7	.000	.596	.000	.957	−.020	.006	−.001	.825	.001	.842	−.007	.045
8	.010	.119	.000	.615	−.010	.248	.002	.186	−.001	.384	−.001	.638
9	.000	.408	.000	.795	−.020	.001	.000	.942	−.002	.559	−.008	.002
10	.000	.532	.000	.760	−.010	.011	−.001	.848	−.003	.378	−.006	.032

Models 1 to 10 differ only with respect to the mediator variable: 1 practical orientation, 2 cognitive activation – elaboration, 3 differentiation, 4 perceived clarity, 5 cognitive activation – problem solving, 6 support of meta-cognition, 7 support of motivation, 8 consideration of students’ voice, 9 perceived discipline, 10 perceived pace of teaching. Global = global school leader factor, SetDir = specific school leader factor ‘setting directions’, InstPro = specific school leader factor ‘managing instructional processes’. p = p value. Control variables are GPA, highest parental occupational status, books at home, migration background, and language spoken at home.

Discussion

Gained insights

In the present study, we explored the effects of school leadership on instructional quality and student (mathematical) achievement in Austrian primary schools. In doing so, we drew on two central leadership strategies: setting directions and managing instructional processes. Based on a framework of indirect effects, we hypothesized that school leadership affects student achievement via instructional quality. However, against the systemic background that the Austrian school system represents a low-stakes context and that its primary schools in particular have a long-standing tradition of principals acting as teachers with additional administrative functions, we also assumed that leadership effects might be small.

For our analysis, we used data obtained in national tests in 2018 allowing access to information from over 3,500 teachers and 75,000 students. The sample covers a whole cohort of Austrian primary students and their teachers. In accordance with methodological guidelines for leadership effectiveness studies, our analytical procedures entail multilevel structural equation modeling and draw on external assessments for all relevant constructs.

The results of our study support the assumed positive relations between the mediator variables (dimensions of instructional quality) and the outcome variables (student achievement). In particular, ‘perceived discipline’, ‘perceived pace of teaching’, and ‘cognitive activation – problem solving’ are most strongly related to students’ test scores in mathematics. However, we find negligibly small and non-significant direct effects of leadership on student achievement before and after controlling for the socio-economic composition of the student body. No statistically significant indirect effects of leadership (via instructional quality) appeared either.

Limitations of the study

One explanation for the lack of demonstrable leadership effects lies in the very nature of school- or class-level effects. As Scheerens (2017) points out, factors on the student level are responsible for variation in student achievement to a much greater extent. In fact, our results resemble prior leadership research with effect sizes that were often classified as small. Another explanation, which we anticipated in the theoretical groundings of our analyses, could be the fact that the studied principals received only rudimentary leadership training before taking office. A targeted training programme for school heads was introduced only in the 2018/19 school year. This programme is more comprehensive than previous ones and includes internationally approved leadership concepts. In addition, the Ministry of Education published a profile for school leaders in 2019. Both the new training and the school leadership profile are meant to provide guidance and support for school leaders. Principals in our sample, however, were unfamiliar with both.

Although using data from a large-scale survey offers the opportunity to apply appropriate and complex statistical methods, reference must be made to the cross-sectional character of the data, which inhibits causal interpretations. Nevertheless, it should also be stressed that cross-sectional studies can provide meaningful insight into possible longitudinal relations of variables if statistical analyses are rooted in solid theoretical assumptions about predictors and outcomes and if crucial control variables are specified. In the present study, we claim both. Given that we could demonstrate theoretically plausible relations between all dimensions of instructional quality and student achievement but not between dimensions of leadership practices and either of these constructs, it is questionable if a longitudinal approach would reveal these missing associations over time. In other words: Are changes in the principals’ behavior expected to evoke changes in teachers’ instructional work if differences in leadership practices are not associated with differences in instructional quality? In addition, the full census data also have merits. Advantages of the present large-scale cross-sectional study are the representativeness and the size of the sample (high power). In contrast, longitudinal panel studies are often not representative and small (too little power) due to costs of repeated data collection and study dropout. In this sense, we argue that both study designs have their justification. Of course, large panels would be best suited for our research question, but with the data at hand, we argue to provide an initial informative picture of systematic relations among educational leadership, instructional quality, and student achievement in the specific setting of Austrian primary schools.

Further restrictions concern the operationalization of the school leader dimensions. Although plausible in terms of content, the two scales used in our study do not represent validated scales. Moreover, they reflect arguably important yet not the full range of possible leadership behaviors that was mentioned in the first chapter.

Moreover, secondary data analyses set limits with regard to modeling theoretical assumptions. Leithwood and Jantzi (2006, p. 205), for example, mention motivation, capacity, and work settings as ‘key variables in a general model of employee performance’. This set of variables has an impact on classroom practices and, subsequently, on student achievement. For school leaders, it is much more likely to influence these factors than the actual teaching process. However, these intermediate processes could not be modeled with the available data.

Finally, our output measure is limited to mathematics achievement. Although studies on school leadership effects traditionally focus on student achievement, principals might also have an impact on other valuable educational outcomes, such as student well-being or school engagement.

Implications for research and practice

This research supports the assumption that the effects of leadership are context dependent. Policy makers have to be aware that what works in one education system is not readily transferable to another. In line with Brauckmann and Schwarz (2015) our results suggest that institutional frameworks including governance structures and regulations play a crucial role for leadership practices. In the case of Austria, reforms of educational governance have not been paralleled with specific training programmes for aspiring principals to respond to new duties and responsibilities. As Pont et al. (2008, p. 136) emphasized, a teaching background is not necessarily predictive of having the relevant skills to run a school. For both educational researchers and policy makers, it would certainly be revealing to conduct a replication or follow-up study to the one presented here several years after the very recent reform of training programs and job profile (see chapter 2). Another fruitful strand of inquiry pertains to the question if a marked reduction of Austrian (primary) principals’ teaching obligation, complemented by continuous in-service training, would benefit their leadership behavior and ‘pay off’ in terms of leadership effects. Although principals themselves, socialized as teachers, appreciate instructional interactions with the students, they also admit that preparing and giving lessons is time-consuming and keeps them from concentrating fully on their leadership tasks (Huber et al., 2013). Since teaching some hours ‘besides’ occupying a management position does not guarantee sophisticated lessons or even elaborated counseling skills toward the school’s teaching staff, it may well be effective to extend principals’ in-service training on topics of instructional/school quality, quality management, and feedback or coaching. However, a marked shift in required tasks and devoted time in Austrian schools’ management positions must also be accompanied by the principals’ self-reflection regarding their professional identity. If they see themselves mainly in the role of a primus inter pares (see chapter 3), they may not make full use of training contents and extended time frames.

Our final suggestion pertains again to methodological challenges of examining leadership effects and picks up another striking feature of the studied representative sample from Austrian schools. We drew on teacher ratings of educational leadership practices with good reasons (see chapter 4). Interestingly, we hardly found consensus among the teaching staff within each school on the requested indicators of these practices (see chapter 5 for information on methodological consequences). Under such conditions, the intended external assessments of educational leadership should also stem from trained observers. These suggestions and the mentioned limitations of the present study offer starting points for future research projects in this field.

Relevance of the study to the current situation: principals’ role during the COVID-19 pandemic

Many studies, particularly in German-speaking countries, highlight the increased importance of school leadership in meeting the challenges imposed on the school system by the COVID-19 pandemic (Huber et al., 2020; Jesacher-Rößler & Klein, 2020; Lavonen & Salmela-Aro, 2022; Reyes-Guerra et al., 2021; S-CLEVER Konsortium, 2021; Weiner et al., 2021). For instance, Weiner et al. (2021) argue that the school leaders’ task to create conditions so organizational members can reflect their current practice and to facilitate problem solving and innovation is particularly necessary during times of crisis. Moreover, the pandemic calls upon school leaders to best utilize ‘the efforts and skills of their workforce to adapt to changing conditions and perform under pressure’ (ibid., p. 2). To put it more concretely: Were school leaders able to create learning environments in which students, teachers, and parents felt safe to take risks, make mistakes, and learn? This addresses the role of school leadership in ensuring instructional quality during COVID-19-related school closures. Our study provides empirical evidence suggesting that a direct effect of school leaders’ behavior on instructional quality is rather unlikely. This may also be the case for teaching during COVID-19 as school leaders, teachers, and students were locally separated from each other. Rather, we assume that a school leader’s behavior affects teachers’ working environment during COVID-19, which in turn is a prerequisite for instructional quality. Early studies show that during COVID-19, school leaders’ behavior was associated with teachers’ well-being (Lavonen & Salmela-Aro, 2022), teacher cooperation (Huber et al., 2020; Jesacher-Rößler & Klein, 2020), teacher-student/parent cooperation (S-CLEVER Konsortium, 2021), implementation of online teaching and maintaining educational standards (Jesacher-Rößler & Klein, 2020), coordination of teachers’ actions (Huber et al., 2020), etc. Moreover, school leaders were responsible for ensuring a safe learning environment when schools were allowed to reopen. Hence, given the multiple tasks of school leaders during the pandemic, we argue – in line with Reyes-Guerra et al. (2021), pp. – that the crisis has corroborated that setting directions and managing instructional processes represent only two of many leadership approaches that are needed to tackle the challenges implied by the pandemic. Particularly under drastic contextual changes, additional leadership knowledge, skills, and dispositions come into play. Thus, future studies are necessary that integrate a broader set of leadership practices when investigating school leadership effects on teachers’ work (e.g. instructional quality) in particularly challenging conditions such as the recent pandemic.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

David Kemethofer

David Kemethofer is professor at the University College of Teacher Education, Upper Austria. His research focus lies in the areas of school management and leadership, quality management in the school system, school inspections, and educational standards and standard-based performance tests.

Christoph Helm

Christoph Helm is a full professor in the field of Educational Research. His research is on School and Teacher Effectiveness Research, Instructional Quality Research, COVID-19 and Education, and Competence Assessment.

Julia Warwas

Julia Warwas is a full professor in the field of Education for Business and Economics. Her research interests cover Educational Leadership and Governance, Teaching Quality and Teacher Cooperation, Stress and Coping in Working and Learning Contexts, and Competence Assessment for Vocational Education and Training.

Notes

1. Students with special educational needs who were taught mathematics according to the curriculum of special schools or a lower school level were excluded. The same counts for students with physical or sensory disabilities if they were not able to take part in the test under standardized conditions.

2. There was also a questionnaire for principals and parents, which is not included in the present analysis.