Negotiating between the accountability and the innovation mandates: evidence from Italian schools

ABSTRACT

In Italy, the National Evaluation System (SNV), encompassing both internal and external accountability components, is designed not only to hold school actors accountable but also to stimulate innovation and change in pedagogical practices. Nevertheless, existing literature presents inconclusive and contradictory findings regarding the effects and the relationship between school performance-based accountability (PBA) and educational innovation. Utilizing sensemaking and filtering theories, this paper analyses the negotiation of PBA and innovation mandates in Italian schools, emphasizing how school actors interpret, filter, and adapt the policy expectations to their organizational and educational practices. Special attention is given to the influence of local school contexts and performative pressures on this process. The primary data collection method involves interviews with school leadership teams and teachers analysed through an ideal type case analysis. Findings reveal diverse school responses to the double mandates of PBA and innovation. Varied logics of school enactment within the contemporary policy context are identified, influenced by teacher attitudes towards PBA and schools’ socio-economic conditions. Innovation is evident in both advantaged and disadvantaged school contexts, albeit with distinct applications and interpretations. Overall, the adaptation of PBA prerogatives in schools to foster innovation and change is neither uniform nor direct, although external pressures impact educational practices across contexts. Beyond socio-economic factors and administrative and marketing accountability pressures, additional intervening elements include school leadership and staff cohesion as well as school infrastructure and material resources.

KEYWORDS:

• School accountability
• education innovation
• performance-based reforms
• Italy
• policy enactment

Introduction

Over the past few decades, numerous reform efforts have been directed towards cultivating innovation processes in schools.¹ Educational innovation, intended as a significant change in educational practices (Vincent-Lancrin et al., 2019), despite its pedagogic focus,² is also a process that relies on school governance reforms. Specifically, the promotion of educational innovation is often accompanied by an increased autonomy granted to schools for the implementation of transformative changes. Schools with greater autonomy are expected to have more freedom to innovate, and to enhance their understanding of effective practices tailored to specific contexts and individuals (Looney, 2009). Yet, an increased autonomy for schools is frequently coupled with heightened external control tied to students’ performance in national large-scale assessments (NLSA). Governments are willing to delegate more autonomy to schools to the extent to which schools are willing to undergo more intensive monitoring based on their outcomes (Fahey & Koster, 2019; Verger & Parcerisa, 2017). These policy instruments typically manifest in configurations of test-based (TBA) or performance-based accountability (PBA), encompassing the delineation of learning standards, students’ performance measured through external standardized tests, the specification of school autonomy, and the implementation of accountability mechanisms (Verger et al., 2019).

The relationship between the two mandates, namely, PBA on the one side, and innovation and autonomy on the other, is however not clear. Research exploring the impact of PBA on educational practices presents conflicting outcomes (Mittleman & Jennings, 2018), also shaped by the interpretations and adaptations of policy environments within schools by educational actors (Diehl & Golann’s, 2023; Dulude & Milley, 2021). On the one hand, PBA pressures may serve to communicate good models of teaching and learning (Herman, 2004), motivating school actors to challenge conventional practices (Fahey & Koster, 2019). Conversely, these pressures can influence instructional practices (Avalos et al., 2020), by regulating the structure of curricular content, encouraging curriculum narrowing and endorsing teacher-centred transmissive instruction (W. Au, 2007; Barrett, 2007). Moreover, beyond these types of process innovations in education, administrative types of school innovations³ may have less direct relevance for the classroom changes, affecting instead the position of a school within a local school market (Lubienski, 2009). Additionally, the pressure to obtain favourable results in large scale assessments can have an adverse impact on the creativity of teachers (Appel, 2020) and their autonomy in educational planning (Farvis & Hay, 2020), by dissuading teachers from taking risks and limiting the time available for innovative or creative practices (Falabella, 2020; Knight, 2020; Sahlberg, 2009).

At the same time, there has been a gradual shift from traditional forms of accountability to more school-owned and school-driven models (MacBeath, 2008). This transformation is manifested through internal forms of accountability, where schools hold themselves accountable for their performance, often through internal processes and assessments, and where crucial elements of accountability are generated largely within school staff, fostering strong professional communities linked to enhanced student performance.⁴ Schools with established internal accountability tend to be more coherent and effective as organizations and more responsive to external accountability demands, since they are more skilful in choosing the curricular areas to prioritize, determine the instructional strategies related to performance measures, and learn how to handle external pressures in alignment with their own core values (Elmore, 2005a). However, research also indicates that the interaction between internal and external accountability is not always clear-cut. Without the appropriate internal standards, strong leadership, professional trust and staff capacity to collaborate, external accountability may not consistently promote school responses in alignment with external demands (Elmore, 2005a).

Furthermore, the impact of PBA and innovation policies may not be uniform across all schools. In accountability contexts, schools on probation often adopt strategic instructional approaches, tailoring teaching to match test content or narrowing the curriculum to boost test results while avoiding significant changes to their methods or principles (Mittleman & Jennings, 2018). The socio-economic school contexts may also influence teacher practices, such as using test data to categorize students (Hardy et al., 2019), tailoring instruction based on performance, and prioritizing practical-oriented teaching in lower-performing schools (Diamond, 2007).

Whilst previous research has focused on school enactment in relation to PBA policies in various contexts (e.g. Falabella, 2020; Gunnulfsen & Roe, 2018; Landri, 2021; Paletta et al., 2020), specific studies exploring the intricate relationship between PBA policies and innovation in education are scarce. In this context, Italy serves as an understudied and relevant case to comprehend how school actors interpret and enact the dual accountability-innovation mandate. Recent reforms in Italy have expanded school autonomy and PBA to incentivize pedagogical innovation and teaching flexibility (Checchi & Mattei, 2021; Paletta et al., 2020). Innovation is inherently part of the National Evaluation System (SNV) (Faggioli & Mori, 2018), an accountability model combining internal forms of accountability (through school self-evaluation reports and improvement plans) and external accountability components (through national student assessments and ministerial inspections) to foster reflexivity and change in pedagogical practices (Paletta et al., 2020). However, despite accountability and innovation being considered equally important dimensions in the Italian policy framework, implementation challenges emerge and the data-intensive external accountability seems to carry more weight than the innovation component (Mentini & Levatino, 2024).

The paper aims to analyse a) how the tension between PBA and innovation unfolds at the school level by examining how school actors understand and adapt their educational practices to the policy mandates, and b) how the local school context influences these dynamics.

The paper follows this introduction with an overview of the Italian reform context. Subsequently, it presents the theoretical concepts, and outlines the methods and data. Results are presented through an ideal case analysis (Stapley et al., 2022), beginning with an overview of the local school context and then illustrating school actor’s enactment strategies. The conclusion and discussion section includes reflections on the similarities and differences between schools, policy recommendations and potential future lines of research.

The Italian context and local meaning of educational reforms

The Italian educational system has been long characterized by a centralized and bureaucratic structure (Grimaldi & Serpieri, 2012; Mattei, 2012), and traditionally recognized as a late adopter of educational reforms focused on evaluation, accountability and innovation (Barzanò & Grimaldi, 2012; Kickert, 2007). Starting from the late 1990s, a sequence of performance-based reforms has been initiated to modernize the educational system (Checchi & Mattei, 2021; Grimaldi & Barzanò, 2014).

Assessment and accountability policies

In 2013, a National Evaluation system (SNV), was introduced, encompassing different policy instruments and integrating both external (national assessments and a sample-based ministerial inspection) as well as internal performance-based accountability mechanisms (through school’s self-evaluation reports and improvement plans) with a specific focus on school improvement and innovation (Presidential Decree 80/2013). In this comprehensive evaluation system, there are two distinct but complementary national organizations which are responsible for assessment and accountability on the one side (INVALSI), and improvement and innovation on the other (INDIRE). The system has been therefore defined as a ‘three-legged model’ where innovation is one of its main components (Mentini & Levatino , 2023).

The PBA process involves schools’ compiling of a self-evaluation report (called ‘Rapporto di Autovalutazione’ RAV), which includes a self-analysis of the schools’ resources, the organizational and pedagogical processes adopted, and the results achieved, including external INVALSI assessments.⁵ This is followed by external ministerial inspections that visits the schools to assess the reliability of the report and the stated goals, the development of improvement plans, indicating how the school is expected to reach the goals by defining strategic actions and priorities, and the publication of the documents in school websites or ministerial portals for transparency purposes.

The national student assessment (INVALSI), as one component of the accountability system, evaluates students’ competencies in Math, Italian and English language at primary, lower-secondary and secondary education. These national tests aim to measure students’ ‘fundamental competences, knowledge and abilities’ and are expected to lead to a change in pedagogy, because they are not tied to the evaluation of ‘simple knowledge’ acquisition, but rather to students’ problem-solving skills, their ability to apply knowledge, connect it to other domains and reasoning capacities (INVALSI, 2023).⁶

The PBA model therefore emphasizes diagnostic, equity and innovation logics (Mentini & Levatino, 2024), since it puts emphasis on changing professional practices and the ability of schools to adapt to their specific contexts (Paletta et al., 2020). The system serves to diagnose critical teaching and learning areas, the identification of which should be followed by interventions in schools, aimed at systematizing practices and reducing learning inequities (Mentini & Levatino, 2024). The apparatus is therefore expected to lead to a process of continuous reflection and improvement in organizational and pedagogical aspects (Faggioli & Mori, 2018), with no material consequences attached to the performance results (Paletta et al., 2020). On the contrary, the external standardized tests, fostering student reasoning and transversal competences, are often considered an innovative tool, in an educational context, such as the Italian one, characterized by a teacher-centred theoretical legacy (Mentini & Levatino, 2024). In fact, according to the official discourse, to prepare students for the tests ‘a slightly different, more engaging pedagogy may be needed, which stimulates students to think about what they are studying and make it theirs’ (INVALSI, 2023).

The Italian policy approach to innovation

Contemporary reforms in Italy also showcase increased interest in pedagogical and organizational innovation. Decentralization reforms (law 59/1999 and 107/2015), gave power to lower government levels and expanded the autonomy of schools especially in teaching, curricular and pedagogical matters. The innovation mandate in Italy is based on schools’ autonomy in defining of a three-year educational plan (PTOF) where teachers are expected to choose the most effective teaching strategies⁷ and plan their curriculum based on the European life-long learning skills, and a teacher evaluation based on innovative teaching methods and student performance (Checchi & Mattei, 2021).

The 107/2015 law (‘The Good School Reform’) initiated spaces for pedagogical innovation and teaching flexibility. Education innovation became associated with pedagogical changes and innovative teaching methodologies such as cooperative learning, competence-based education (lifelong learning skills), project-based and hands-on workshops, in the direction of overcoming a frontal teaching model (Biondi et al., 2009). At the same time, schools are granted higher autonomy to create partnerships with external stakeholders, and this type of network-based innovation should also lead to higher pedagogical innovation and digital innovation (law 107/2015).

In Italy, the innovation mandate is therefore composed of multiple and diverse pedagogical and organizational features, lacking conceptual clarity and consensus (Mentini & Levatino, 2024). In general, the concept of innovation in Italy is composed of four main dimensions: 1) Teaching and learning, which is related to transformations in teaching and evaluation methods (e.g. adopting problem-based teaching, debate, flipped classrooms methods, continuous formative assessment methods), flexible learning spaces and timetable, and the choice of (digital) educational materials, 2) Leadership and organizational development, related to a form of distributed leadership, the collaboration between leadership and staff and amongst staff, and the modernization of infrastructure and flexibilization of school calendar 3) Openness and relationship with the external community, as in schools being open up to the local community (including universities, industry and cultural institutions), collaborating and involving families and collaborating with external partners (such as agencies, institutions, local entities), and 4) Propensity to change, related to strategic planning, including the use of data for school improvement purposes, technological devices, professional development and curricular change (Nardi et al., 2022). ICT based, and e-learning activities are also part of the innovation mandate (National plan for digital education (Piano Nazionale Scuola Digitale), 2015).

School actors’ reception of the double-mandate

In Italian schools, teacher and school evaluations were never seen positively by teachers and unions given the intrusion felt in the professional autonomous field, leading to teacher resistance through forms of protests, strikes and oppositions (Barzanò & Grimaldi, 2013). According to previous research, the great majority of teachers was found critical and contrary to the INVALSI assessments (Martini & Papini, 2015), while over half of the Italian teachers expressed preference for internal school self-evaluations (De Angelis et al., 2015). Arguments against the implementation of external tests are related to having introduced competitive dynamics amongst schools (Peruzzo et al., 2022), the perception of being disconnected from school subjects and practices, having generated anxiety and doubts, or perceived as a top-down and bureaucratic requirement (Pastori & Pagani, 2016). The perplexities are also related to the aim and purpose of the evaluations, their supposed ‘objectivity’, and the change produced in the teacher’s autonomy and instruction (DiCresce, 2019).

In terms of innovation, although data shows that less than half of the schools implement a transversal competence-based curriculum (Poliandri, 2015), and that Italian teachers were tied to low innovative practices, such as frontal (face-to-face) teaching and the use of ‘less constructivist strategies’, including project-based, cooperative learning or inquiry-based learning (De Sanctis, 2010; OECD, 2013), Italy is also the country where schools are considered having strong capacity to adopt innovative practices (OECD, 2019), and is renowned for a vibrant teacher union activism and pioneering pedagogical experiences and figures, including Maria Montessori or Reggio Emilia experience.

However, the purportedly ‘equilibrated’ national evaluation model (SNV) integrating both accountability and innovation components, appears limited in practice (see Mentini & Levatino, 2024). Key barriers include incomplete school autonomy, resource deficiencies, a standardized teaching culture (OECD, 2013) and a supposed misunderstanding and misuse of the policy instruments among school actors (Mentini & Levatino, 2024). Furthermore, given that it is easier to objectify learning outcomes than other educational processes, there is the risk that schools prioritize test results instead of other processes when compiling their self-evaluations, potentially rendering the system as ‘crippled’, as the external accountability overshadows other components (Mentini & Levatino, 2024). Consequently, analysing how school actors interpret, receive and enact the institutional messages in their daily practices becomes a crucial avenue for research in this context.

Theoretical framework

To explain school’s policy enactment processes, the theoretical framework integrates interrelated mechanisms of sense-making (S. Ball et al., 2012; Coburn, 2001) and institutional filtering (Diehl & Golann’s, 2023; Dulude & Milley, 2021), with a heuristic approach to the enabling/constraining role of the local environment (Braun et al., 2011). Rather than referring to policy compliance or implementation, the theoretical framework considers the range of responses that schools give to external pressure, understanding ”the factors that affect those responses and, in turn, shape external pressure” (Elmore, 2005a, p. 137).

Making sense of the policy messages by school actors

School actors experience and interpret policy prerogatives in different ways actively shaping and transforming the resulting policy outcomes (S. Ball et al., 2012; Braun et al., 2011). The cognitive process of interpretation involves perceptions, opinions, and attitudes towards educational policies. When faced with a new policy mandate, school actors engage in interpreting and trying to make sense of the message, before putting the new mandate into practice (Coburn, 2001). Teachers and school actors actively mediate institutional pressures, a process framed by their pre-existing beliefs, worldviews and practices (Coburn, 2004). This implies that educators adapt, assimilate, and selectively enact policies and initiatives in alignment with other logics they have been socialized into (Thornton & Lounsbury, 2012). The process includes: a) noticing and selecting information and messages to put into practice b) constructing meaning and understanding of the information and c) negotiating details and acting upon those interpretations (Coburn, 2001; Porac et al., 1989). The process of meaning-making also includes how individuals think and feel. Thus, teachers’ emotions may influence the sense-making and reaction to the accountability reform (Kelchtermans, 2005). Thus, stress and anxiety related to external control and standardization (Schoen & Fusarelli, 2008) and a greater attention to students’ pace and in-depth curriculum (slow-teaching) may signify a risk for teachers, whilst standards, structure and direction may represent a desirable safe path for teachers, who are often overburdened (Falabella, 2020).

Institutional filtering into local realities

New theoretical approaches integrate policy sociology and enactment research into institutional theory to comprehend how schools navigate different institutional pressures (Diehl & Golann’s, 2023; Dulude & Milley, 2021). According to these perspectives, schools filter and locally assimilate the external pressures into their organizational reality and educational activities. Filtering encompasses various environmental aspects entering school organizations, while adaptation pertains to the local incorporation of these filtered aspects into daily operations by school actors. Crucial elements such as routines, networks, and sensemaking processes contribute to understanding how and why schools filter external demands, adapting them into their organizations (Diehl & Golann, 2023). Key actors, including principals or frontline workers such as teachers, actively frame external messages, noticing, selecting, and filtering policies in their organization in different ways (Diehl & Golann, 2023). Consequently, when confronted with external demands, especially when these cover multiple priorities and may even be perceived as contradictory, schools may exhibit a broader spectrum of interpretations and enactments to policy pressures, extending beyond resistance or alignment. As exemplified in the findings section of this article, the interplay between sense-making and filtering mechanisms plays a crucial explanatory role in understanding diverse school responses of schools to external institutional demands (Parcerisa & Verger, 2023).

The mediating role of school’s contexts

Previous research indicates that schools exhibit diverse responses to the policy demands, emphasizing the importance of considering school contexts, agencies and their unique characteristics (Coburn, 2004; Landri, 2021). The translation of policies into practice is a negotiated and intricate process often influenced by external circumstances encompassing institutional and socio-economic factors (Braun et al., 2011). The distinct attributes of schools and other contextual factors can thus elucidate and mediate school actors’ interpretations and behaviours in relation to institutional messages (Braun et al., 2011).

The socio-economic composition of the school, intricately linked to its intake, geographical site, location and historical background, contributes to the construction of narratives and institutional stories about the school (Braun et al., 2011). Research frequently observes that to respond to accountability pressures schools under probation are more inclined to adopt instructional strategies such as tailoring teaching to align with test content or narrowing the curriculum to focus on assessed subjects. This strategic approach aims to enhance test results, allowing schools to navigate punitive measures without fundamentally altering their teaching methods, educational principles, or organizational procedures (Mittleman & Jennings, 2018). This manoeuvring enables schools to maintain a status quo or continue their activities without disruption (Hallet, 2010). Moreover, the adoption of new organizational routines and responses to accountability demands may vary depending on whether schools are under probation, influenced by how school actors experience external pressure, and their adherence to the accountability policies (Verger et al., 2021).

A school’s socio-economic context also influences the teaching approaches and pedagogies. Teachers are found to use test data to categorize underperforming students and tailor their instruction accordingly (Hardy et al., 2019). Additionally, teachers tend to prioritize practical-oriented instruction (e.g. lectures, seat work, memorization, recitation) in lower-performing schools, preparing students for manual, clerical, or low-wage service-sector, while providing more challenging instruction to students from socially advantaged groups (Diamond, 2007). This highlights the intricate relationship that occurs between socio-economic inequalities, teachers’ expectations with their students’ potential, and the adoption of pedagogic practices.⁸

Material contexts encompassing school technology, resources and budget also play a key role, since the conditions of the school buildings, including their layout, quality, and spaciousness, have a considerable impact on policy enactments on the ground, including the capacity to attract and keep ‘good’ teachers or other staff (Braun et al., 2011).

Professional contexts, referred to as the values, ethos and culture of a school, as well as the school management and teacher experiences, ultimately shape how schools prioritize accountability demands compared to other societal expectations (Braun et al., 2011). Individuals adapt to new policies based on their interactions with others and the organizational cultural configuration. Existing networks of collaboration and support within schools, especially among teachers, significantly contribute to how schools translate these demands into specific educational practices and organizational routines (Diehl & Golann, 2023). This responsiveness stems from an openness to dialogue among colleagues and a teacher’s self-evaluation embedded in the day-to-day practice, mitigating anxiety and pressure (Elmore, 2005b). Additionally, to align internally with the external accountability demands, schools necessitate a shared culture with strong professional norms, a well-developed capacity for teachers to collaborate around shared values, and a ‘distributed leadership’ involving teachers more actively. The principal’s actions and leadership style may therefore be crucial in aligning and maintaining congruence between external and internal factors (S. J. Ball & Maroy, 2009).

In conclusion, the singularities of schools, manifested in concrete conditions and ‘professional cultures’, act as ‘frames’ and ‘filters’ to reform ideas, aiding in understanding the likely configuration of the translation (Landri, 2021).

Methods

The methodology employs a case study approach (Yin, 2009), based on qualitative research conducted in 12 lower-secondary schools in Rome, Italy. The selection of Rome is driven by both analytical considerations, such as feasibility of the study and familiarity with the context, and substantial factors, including ongoing accountability and innovation reforms in schools, and the socioeconomic disparities within the city that are taken into account when selecting schools. Rome serves as a compelling and representative case due to the presence of PBA and innovation reforms, along with a policy arena hosting key educational stakeholders, public authorities and institutions. The city encompasses 200 sub-urban zone areas (equivalent to city neighbourhoods) and 15 municipalities, displaying significant variations in social and economic indicators across these areas (Mappa Roma project⁹), proving a fertile ground for exploring differences amongst school enactment responses.

Public state schools exclusively form the focus of this study, given they constitute the majority (73%) of schools in the city of Rome as of 2018/2019 (Ministry of Education, 2023).¹⁰ In Italy, the state is the main provider of mass schooling, with only limited space for private (religious) schooling.

The study concentrates on lower-secondary education¹¹ (ISCED 2, age 11–14) commonly known as ‘middle school’, typically included in comprehensive school institutes covering ages 4–13.¹² The decision to focus on this educational level stems from the highly selective nature of the Italian system after lower-secondary schooling (Giancola & Salmieri, 2022), operating along three very hierarchical tracks: technical, vocational, lyceums (high schools). In the final year of low-secondary school, students undergo the INVALSI national test and a final oral and written examination, before transitioning to secondary education, obtaining a ‘diploma di licenza media’. Furthermore, the choice to emphasize lower secondary education is grounded in the belief that the relationship between accountability and innovation becomes apparent at this level. Primary schools typically have one teacher per class, whereas in lower secondary students exhibit more autonomy in their learning, and the teacher-student relationship is less mediated by families.

Sampling procedure

The research employed a two-step sampling process. Initially, a randomized sample of schools was chosen. In this phase, 100 public lower-secondary schools in the city (out of 370 total lower-secondary public schools) were selected based on ownership, educational level and further stratified by average income and performance on INVALSI national tests. Subsequently, utilizing the representative sample, schools were selected according to two primary variables: performance in standardized tests¹³ and the school average income, acting as a proxy of school’s socio-economic status.¹⁴ The data on school average income and performance results were divided into quartiles (low performance (Q1), middle performance (Q2 + Q3) and high-performance (Q4); working class (Q1), middle class (Q2 + Q3); and upper class (Q4). By crossing these two variables, a total of 12 schools were selected across four categories. The characteristics of these schools are presented in Table 1 below.

Table 1. Characteristics of selected schools, by income and performance.

School	Average Income	Performance category	Interviewed school leaders	Interviewed teachers (also with technical functions)
1	Upper class (Q4)	High Performing (Q4)	2	1
2 3 4	Upper class (Q4) Upper class (Q4) Upper class (Q4)	High Performing (Q4) High Performing (Q4) High Performing (Q4)	1	1 1 1
5	Upper class (Q4)	Low Performing (Q1)	2	1
6 7 8 9	Medium class (Q3) Medium class (Q2) Medium class (Q2) Medium class (Q2)	Medium Performing (Q2) Medium Performing (Q3) Low-performing (Q1) Low Performing (Q1)	2	4 1 2 2
10 11 12	Working class (Q1) Working class (Q1) Working class (Q1)	Low Performing (Q1) Low Performing (Q1) Low Performing (Q1)		2 1 3
Total			7	20

Source: own elaboration.

Data collection and participant selection

The study employs a case study approach, drawing on semi-structured interviews to principals, management team and teachers. Additionally, where feasible, documentary analysis was conducted using publicly available documents such as the school’s institutional project (PTOF), self-assessment reports (RAV) and fieldnotes.

A purposive sampling strategy was adopted (Patton, 2015). Principals and management teams were interviewed due to their central primary role in the organizational and managerial aspects, especially given recent reforms on school autonomy (law 1997; law 107/2015). For teachers, selection criteria included those responsible for a course or subject assessed in the national test. In some cases, interviews were extended to teachers not directly involved in assessed subjects. Teachers with a technical and/or informal leadership positions (i.e. part of evaluation committees, project planning committees, collaborators of the school principal, or instrumental functions) were included given their relevant managerial and coordination tasks.¹⁵

Access to schools

Initiating contact with schools involved sending emails and subsequent phone follow-ups, leveraging publicly available contact information on school websites. Clear communication of the project aims and interview details were clearly given to the principal, with personal visits in some instances to establish trust and rapport. While some schools, particularly middle-low and working-class areas, presented challenges for interviews (due to high rotation, other school priorities or unresponsiveness), document analysis served as a valuable complement to triangulate responses. However, I still considered and took this limitation into account in the analysis and interpretation of results.

Interview structure and data analysis

The interviews were conducted between November 2022 and January 2023, lasting an average of 40 and 60 minutes. Employing a semi-structured format, interviews followed specific scripts but remained flexible to delve deeper into emerging issues. Interview scripts were tailored for principals/leadership team covering modules that sought to investigate the relationship between school contingencies and enactment processes. Scripts addressed the opinion and translation into practices of the policy expectations on schools, and at the same time capturing schools’ contextual characteristics. The modules included: 1) Participant background, the years of experience working within the school and responsibilities 2) School context and resources – socio-economic composition of students, material and economic resources, school reputation, school climate – to investigate the cultural, social, historical and material elements that condition policy enactment 3) Opinions and perception about the accountability system – opinion about national tests and their publications, use and discussion of test results for improvement and innovation; perceived pressure – to capture opinions on the standardized test, the alignment of school goals with the accountability system, and perceptions about accountability pressures 4) Translation of the accountability system to pedagogical and organizational strategies and innovation practices – exploring the school’s and teachers’ educational approach, pedagogical practices, and innovation strategies 5) Market accountability and family relationship – to explore aspects that characterize the school’s reputation, marketing strategies, and relationships with parents with other schools.

For data analysis, an ideal case analysis was employed following a seven steps methodology (Stapley et al., 2022). Ideal types can be defined as ‘generalizations or mental representations of a social phenomenon that will never be identical with reality, but which will help to make that reality understandable’ (Stapley et al., 2022, p. 2). The ideal case analysis is based on the process of developing a typology by grouping cases or participants into different types on the basis of their common features (Stapley et al., 2022). First, I familiarized with the data through interviews, transcription and coding. Transcriptions were coded using ATLAS.ti (version 22) software, employing a flexible coding strategy (Deterding & Waters, 2021), which integrates existing codes from the ERC-funded project (see Parcerisa & Verger, 2023) and emerging inductive codes. Case reconstructions were then created, chronologically summarizing each participants’ narrative through approximately half a page per participant. By comparing and contrasting the school cases, different categories of enactment responses were developed, based on the school’s average income and student body population. Then, descriptions and names were assigned to these types and refined accordingly, followed by a detailed exploration of similarities and differences within and between school actors’ narratives.

Results

Privileged school settings

Privileged schools, characterized by an upper-class student body, are prestigious and in high demand. These schools face limited competition, boasting oversubscribed intakes and a favourable reputation, reinforced through positive word-of-mouth. The families choosing these institutions typically possess a ‘a rich cultural background’ and are financially invested in the educational offer (School 4, institutional project 2022–2025). Academic excellence takes precedence in privileged schools, reflecting heightened parental expectations for their children’s ‘brilliant futures’. Despite endorsing a traditional teaching approach, there is demand for project-based activities and extracurricular courses (Principal, School 4). The stability of teaching staff is also considered essential by parents in school choice and selection.

The reputation of privileged schools is based on student performance, emphasizing academic success. A wide offer of extracurricular projects that align with global citizenship themes, including environmental sustainability, wellbeing, and extra-curricular afternoon courses are offered such as music or sports. Innovation involves institutional and organizational strategies such as establishing partnerships with local entities (e.g. non-profit or religious organizations, music schools), participating in school networks (i.e. plastic free schools) and involving families in financial contributions for extra courses.

The professional contexts of these schools also play a crucial role in shaping the school’s capacity for change and innovation. They have clear leadership direction and cohesive staff (e.g. School 4), demonstrate a more positive attitude towards change and innovation, and are able to activate additional training courses for teachers and interdisciplinary projects. Staff members enjoy stable positions that foster an environment where collaborative learning and project support thrives, since working together for many years’’ represents ‘a stimulus to learn from each other and support each other’s projects’ (Teacher, School 4). Administrative staff in these schools also play a crucial role in supporting managerial and financial responsibilities, such as fund-seeking and project management. Despite sharing common characteristics, privileged schools exhibit varied approaches to accountability and innovation mandates, influenced by their performance level. This diversity underscores the nuanced ways in which schools navigate and respond to external demands.

High performance results and embracing innovation without tensions

In privileged contexts with high-performance results, school actors navigate the dual accountability-innovation mandate by favouring innovation over external accountability, with minimal perceived tension. The data obtained from the INVALSI tests functions as a monitoring instrument, but its systematic use for identifying performance gaps or planning educational improvements remains elusive, creating a gap between symbolic and substantive changes. The enactment of external accountability therefore resembles a formal bureaucratic ritual (Landri, 2021) rather than a catalyst for substantial change in teaching practices. Despite setting goals based on assessment data, the absence of significant positive or negative consequences attached to performance results diminishes the perceived pressure, rendering discussions more formal than transformative.

Maybe ideally, we think it’s right to take this into account, but then in practice, we don’t change much of teachers’ pedagogy based on INVALSI data.

(Teacher/Evaluation committee, school 4)

In schools with highly positive INVALSI results, the absence of consequential pressure contributes to a lack of systematic reflection on the data’s utility for identifying learning gaps or planning improvements. The positive results create a ‘happy island’ (Principal, school 2) where performance is already high compared to other schools, and the need for introspection is perceived as unnecessary.

Students have excellent skills, so we don’t even think ‘let me go and see where I went wrong’ or where the critical point is. (Principal, School 2)

However, while in school 4 the external test data is not perceived as important, nor by teachers or the school principal, and disregarded as a valid evaluation instrument because associated with a multiple-choice test, in school 1 and 2, the INVALSI test is perceived as more important by the school leadership team. This is because, in these contexts, it is associated with a ‘democratic evaluation method, which is the same for everyone in Italy’ (principal, school 2) or ‘a comparison parameter at national, regional and school level’, thus a variable that highlights internal class differences and which guides schools’ planning (principal, school 1).

Moreover, the innovation mandate in privileged schools is associated with various and different pedagogical and organizational features. Innovation in privileged contexts encompasses both educational processes such as the adoption of competence-based teaching and digitalization (school 1), as well as institutional and organizational innovations such as open schooling approaches (school 1 and 4), doing things that were never done before (school 2) and a willingness to adapt to societal changes (school 1). Beyond such features, while in school 4 and 1, innovation is particularly linked to schools’ capacity to establish relationships with external entities and local associations, enriching the educational offer with projects, and is intended as a pre-established practice adopted way before other schools, school 2 intends innovation as a ‘organisational revolution’, exemplified by the adoption of a model called ‘DADA model’ (Didattiche per Ambienti Di Apprendimento), which implies restructuring classroom settings for different subjects, and requiring students to move autonomously from one classroom to another, thus bringing substantial changes in school spaces, timetable and teachers’ professional development. Such inclinations and initiatives reflect a clear administrative school innovation (Lubienski, 2009) and are sustained by school’s pedagogical autonomy in diversifying the educational offer.

Innovation also means doing new things, which perhaps have never been done before, or which were a bit afraid to try. (Principal, school 2)

In terms of teaching practices, privileged schools emphasize lifelong-learning, citizenship competences (Principal, school 2), and competencies for the future, such as autonomy of learning and responsibility. Active and interdisciplinary learning take precedence over content-based teaching (Principal, school 1). Student-centred education, focusing on personalized teaching methodologies, is favoured to allow students to express their individual characteristics (Principal, school 1). Moreover, in such schools, diversifying, adapting and personalizing the teaching approaches and overcoming a traditional education model is seen as beneficial to enhance students’ learning, inclusion and to strengthen key competences. However, the role of teachers and teacher training in sustaining innovative pedagogical approaches changes according to the schools. In school 4, teachers are viewed as having a clear attention to pedagogical innovation and they do not adopt an ‘old school’ frontal teaching method (Teacher, evaluation committee, school 4). Teachers are continuously self-developing and are seen as proactive in suggesting new project-based activities, that is ‘a strength and is also a continuous stimulus for all school staff (…) and the reason why we have such a wide educational offer (…)’ (Teacher, evaluation committee, school 4). On the contrary, in school 1, the leadership team believes that some of the teachers are still tied to using ‘obsolete teaching methods’ associated with a passive transmissive educational model, linked to content-based teaching and student’s performance (principal, school 1). When this happens, teachers are given specific attention and the methodologies will be ‘aligned and adapted to that of the school’ through specific training courses (Vice-principal, school 1).

Moreover, PBA results are perceived as a reflection of innovative pedagogical and educational practices already in place. The conflict between the PBA mandate and innovation only arises when the former contradicts the evaluation of ‘more significant transversal and citizenship competences’ or continuous formative assessment (Vice-principal, school 1). The PBA mandate in these schools is hence adapted to meet the requirements of improvement and innovation.

INVALSI tests evaluate only basic skills, therefore do not reflect schools’ pedagogical priorities. (Teacher, School 2)

Following this reasoning, in such schools, as opposed to the other categories of schools, teachers do not report using instrumental strategies, such as teaching to the test, to enhance students’ performance. This specific practice is either completely disregarded and omitted by the interviewees, or considered to be in clear opposition to pedagogical values related to evaluation, which emphasize ‘open ended questions, reasoning, reflection and understanding as opposed to the utilization of true or false questions to test students’ preparation’ (Teacher, school 2).

Parental preferences also contribute to prioritizing the innovation mandate in these schools. The absence of pressure on INVALSI test results by parents underscores their major interest in other performative aspects, such as languages, internationalization (School 1), innovative or well-being educational dimensions (School 2). On the contrary, parents who subscribe children in these schools ‘do not look at INVALSI results’ (school 1) or ‘are actually critical to students taking the tests (…) and do not even understand why students should take them’ (Teacher, Evaluation committee, school 4). Only in one school case among such privileged schools, affluent parents also support the achievement of traditional academic goals in addition to innovation and hence are also ‘worried about INVALSI results’ (Principal, school 2), and academic ‘basics’ which are commonly seen as solid, tried-and-true educational practices (Kohn, 1998).

More often in privileged schools, qualitative and reputational instruments, including questionnaires to students and families, play a more prominent role in gauging parental satisfaction (Principal, school2) and innovative practices serve as a marketing strategy (Lubienski, 2009), attracting families and making schools more efficient and appealing.

These (innovative) methods have increased the family’s choice of our school… adopted to make the school more efficient but also more attractive. (Principal, School 2)

This interesting result is contrary to the often-assumed hypothesis by which schools with more reputation face lower levels of external pressure due to competitive dynamics, and that schools with higher SES student bodies do not have to divert resources for marketing campaigns since they tend to be oversubscribed (Lubienski, 2009). This result is attributed to a specific governance model in Italy, associated with a quasi-market model (Benadusi & Consoli, 2004), based on familys’ free choice, that creates a sort of demand-driven competition between schools (Colombo & Desideri, 2018).¹⁶ In addition, administrative types of innovations adopted in privileged schools are typically associated with quasi-market models (Lubienski, 2009). Indeed, in such privileged schools’, although intakes are already high and competition with other schools is not particularly felt, principals feel the need to maintain the positive reputation and believe that diversification¹⁷ (Lubienski, 2009) and a unique pedagogical offer is what attracts parents to the school, making the school being specifically ’chosen in comparison with other schools in the same neighbourhood, city or out of Rome’ (Teacher, Evaluation committee, school 4).

The families are interested to what we offer (…), during the open days they start asking about robotics or a specific language course or subject, so we have a certain educational offer, and all of it, including the three-year educational plan is visible on the school website. (Teacher, Evaluation committee, school 4)

I present my offer, and if a parent likes it, they come to our school, if not they go to another one (.) because other schools do different things compared to us, but that’s okay, otherwise we would all be photocopies, and then the parent chooses rightly. (Principal, school 2)

Another parameter of school choice is students’ appreciation of the school. Thus, keeping the school attractive to students also brings higher intake rates: ‘the innovative school method that we adopt keeps students happy and makes them self-responsible, and this is highly appreciated by parents’ (Principal, school 2). Privileged schools thus leverage their autonomy to diversify and adapt their educational offer in order to meet parents’ expectations, thus maintaining high subscriptions. They also leverage their high reputation and high student body to attract stable teachers. Such aspects, together with positive academic results, resemble family’s motivations for school choice and preferences, and do not require additional marketing efforts to attract students. These rationales altogether resemble a ‘universal’ and ‘a-contextual’ (Lubienski, 2009, p. 17) quasi-market mechanism and economist view of education, by which parents have the right to choose the best education for their children and by which choice and competition offer opportunities of innovation and responsiveness to consumers’ preferences (Lubienski, 2009).

Middle – income school settings

Middle-income schools are characterized by an average middle income, but exhibit varying performance results and socio-economic student population. Two schools (School 5 and School 6) cater to a homogenous middle-class student population, where parents typically prefer traditional teaching methods, summative assessments and extensive use of textbooks (Teacher, school 6). The teaching method remains traditional, emphasizing students’ academic success and results, while also incorporating inclusion strategies. In addition, extracurricular subjects and transversal projects related to environmental education are also implemented. Other schools in this category (School 7, 8 and 9) have a more mixed and heterogenous student population. Such schools experienced an influx of students with immigrant backgrounds, reaching up to 50% in some cases (e.g. School 9), contributing to a multiethnic and multicultural student population with a significant percentage of students from a socially disadvantaged background (e.g. School 8).

Heterogenous schools within this category face considerable challenges, including school disorganization, fragmentation, high teacher rotation, and principal turn-over. Teacher stability is compromised due to precarious contracts and budget cuts, leading to a limited personnel pool. The instability of teachers, ‘who stay in schools only for only a few years before moving to other schools’ (Teacher, school 8), makes long term planning and the implementation of long term-changes and innovations more difficult. In these contexts, there is an imperative for schools to proactively engage in self-activation and adapt to frequent personnel changes.

If there is a cohesive teaching staff with stable contracts, you can work during the years, but if there is continuous change in personnel, it is more difficult to implement an educational strategy. (Teacher, School 8)

Homogenous contexts: balancing innovation and performance amid internal conflict

Within homogenous contexts, characterized by average performance results, school actors tend to align innovation with performance goals, utilizing the external assessment data to drive both innovation and performance processes. However, this alignment gives rise to conflicting opinions amongst school staff.

At the managerial level, a significant emphasis is placed on the accountability system. Principals and management teams view accountability instruments as an objective and crucial measurement of schools’ and teachers’ quality. They perceive performance-based tools as essential for self-diagnosis, reflection and influencing teaching strategies. INVALSI assessments, seen as an indicator of teaching quality, are employed to understand the skills effectively taught to students.

Through INVALSI results we have the possibility of understanding on which skills the class has worked on well or on which ones it has not. (Principal collaborator, school 6)

Data from the PBA also serves as a basis for identifying improvement actions and planning. Such schools analyse learning objectives and gaps from INVALSI results, to strategically allocate efforts in the upcoming years, by supporting improvement processes within the school and establishing future priorities in the educational project. The PBA mandate, in this scenario, is embraced, since schools display agreement with the requirements and expectations of the policy message (Landri, 2021).

What interests us is the improvement beyond the standardized INVALSI tests (…) The improvement should be both at the level of the curricular and methodological skills of the teacher and at the level of the student, therefore the whole school should improve. (Teacher, evaluation committee, school 6)

When INVALSI results fall below expectations,¹⁸ schools take specific improvement actions based on the data such as adjusting the class compositions, providing psychological support, and offering targeted training for teachers with lower performance results. Moreover, extracurricular courses are activated in the three subjects evaluated in the national test and teaching is modulated to the content of the assessments, by strengthening some tested subjects or modifying teaching priorities on the basis of the test results.

We modify our teaching also on the basis of what are the answers given by the INVALSI tests. (Principal collaborator, school 6)

… There is greater attention to subjects evaluated in the external tests because for example, remedial courses are activated only and exclusively for the subjects covered by INVALSI tests. (Teacher, school 6)

While some teachers within these schools positively view national tests as objective measurements of student learning, positively evaluating competences which students will need throughout their whole life, others harbour negative perceptions, considering the evaluations as judgements of their profession or interpreting them as individual assessments detached from their own teaching and educational planning. Some teachers feel distant from the accountability system, believing it does not influence their teaching nor provides opportunities to improve students’ performance.¹⁹

Negative opinions on the external accountability are also reported in the high-income school with a homogenous student population, experiencing low performance results, and high performance-based pressure. Teachers in this school feel judged in their professionalism (Principal, school 5), and the reputational pressure deriving from external test scrutiny is perceived by teachers as a self-judgement, contributing to stress and anxiety (Principal, school 5). This performative pressure and ‘fear’ of being judged from the external assessments negatively influences teachers’ instruction, limiting creativity, personalized instruction and alternative teaching practices (Teacher, school 5). The tension between accountability and innovation becomes evident in this context, hindering teachers’ engagement with alternative practices or creativity.

The pressure you feel spills over to the manager, but also to the teacher. Without pressure you would probably have even more creativity in doing things. (Teacher in evaluation committee, school 5)

In homogenous school contexts teachers also report using intensive teaching to the test strategies, to ‘accelerate and motivate students with additional testing especially during that time of the year’ (teacher, school 5), or as a way to train students on the testing modalities, regardless of the testing content (school 6). Such practice, resonating with an item-teaching form of testing preparation (see Popham, 2001)²⁰ is also sustained by making students exercise often on school textbooks, which already incorporate exercises dedicated to the INVALSI tests.

In terms of innovation, a combination of both product²¹ and administrative innovations (Lubienski, 2009), can be observed in such contexts, such as the adoption of digital innovations (e-learning, digital technologies, coding), upgraded school facilities, (school 6), ‘a great focus on interdisciplinarity and collaborative projects’ (school 5). Schools are well equipped with digital infrastructures, additional classroom spaces and laboratories because “if a teacher has what she needs to work she will work better and students will study more peacefully (Principal, school 6).

This is a school that is trying to chase the wave of innovation. A school that has many new projects, especially on digitalization. (Principal collaborator, school 6)

Given the differing opinions among school staff on the PBA mandate, the role of the principal is important in aligning the internal culture with the institutional messages. Principals work to align the internal evaluation methods with the external assessment, and foster a shared understanding of the PBA instruments. Principals in such schools actively engage in communicating the purpose of external evaluations to teachers, emphasizing that INVALSI evaluations aim to monitor and recalibrate the teaching-learning processes (Principal, School 6). By addressing the negative perceptions, the principal advocates for teachers’ training and support, facilitating a collaborative approach to develop new instructional approaches and internalizing accountability expectations.

Heterogeneous contexts: divergent approaches and malleability of the double mandate

In socially mixed schools, with middle to low performance results, teachers exhibit opposing approaches to the double mandates. These responses depend on the alignment between accountability and the teachers’ preferred pedagogical approaches. Teachers may either express tensions or interpret performance data as an innovative or improvement device.

Some teachers view external assessments as unfair and unreliable. The association of national external tests with quiz-type and standardized evaluations (Teacher, school 8) contradicts their preference for personalized assessment and teaching methodologies (Teacher, school 9), believing they reduce students’ conceptual learning and critical thinking. The negative opinion results in open resistance and ‘opting out’ from the regime of accountability (Landri, 2021).

I’m critical of the INVALSI because they are quizzes that students have to deal with only during a short time; they don’t improve the reasoning capacities of students, they are often only quizzes. (Teacher, school 8)

Teachers openly express a desire to boycott the tests, citing fatigue, lack of awareness and the perceived futility of personalized teaching within a standardized framework (Teacher, school 9). This represents an explicit refusal of the testing logic and performative pressure underpinning the external accountability mandate.

Let’s say that I could boycott the INVALSI tests, because it is useless to talk about personalized teaching when you do things that have been the same for everyone. (Teacher, school 9)

Conversely, when teachers associate the accountability mandate with transversal or competence-based teaching, they display more positive attitudes. INVALSI tests, in this context, are seen as stimulants for reasoning capacities, beyond evaluating curriculum knowledge and content.

INVALSI is the result of when you work in terms of skills, transversally. It’s not a matter of curricular programs, it’s being able to think about what you know, being able to come up with a reasoning. (Teacher, school 7)

Interestingly, in some cases, teachers undergo significant transformations in assumptions due to INVALSI assessments, not only focusing on the surface level features of the message (Coburn, 2004). They use the national tests to stimulate students’ transversal competences, by integrate them in their teaching throughout the whole year in a form of curriculum-teaching (see Popham, 2001) or they use the simulations of the INVALSI tests to broaden students’ divergent thinking. Indeed, teachers seldomly use instrumental strategies such as teaching to the test, and this practice is adopted for different reasons and in less intensive ways than in homogenous schools. For instance, teachers report making students exercise on INVALSI tests only at times, but with no specific intention (Teacher, school 9), or with the rationale ‘to activate other kind of reasoning and make students evaluate their learning in other ways than the ones they are used to’ (Teacher, school 8). Interestingly, one teacher also perceives the external evaluations to be an ‘innovation’ that stimulates problem-based teaching, leading to a departure from a more traditional teaching model.

I must say I appreciate the work done by INVALSI in the sense that I actually see it as an innovation (…) When the abstract calculation is not required, students have to know how to use what you have in front of them (…) so you have to set up the teaching in a whole other way. (Teacher, school 9)

Teachers in heterogeneous contexts adopt socio-emotional strategies, focusing on equity and inequalities in education, for instance offering courses for students with a migration background in the Italian language. They take on an ‘activist’ and missionary approach, defending students’ interests and backgrounds, ‘by taking care of students and giving them as many experiences as possible’ (Teacher, school 9). Innovation in heterogenous contexts is associated with ‘not doing things in a traditional and passive way’ (Teacher school 9), and encompasses active learning, project-based and manipulative teaching, outdoor learning, small group work, and alternative assessment methods.

Students have a very low concentration level, so we need to get the message across in very different forms, whether it be in a visual way, perhaps through quizzes, sometimes even in the form of a game. (Teacher, School 8)

Some teachers adopt active-based methods to enhance students’ interests (teacher, school 9), or manipulative methods to overcome students’ learning difficulties or ‘absences of educational culture’ (Teacher, school 7). This approach is often justified by the fact that “maybe students who have more difficulties in Italian language or reading a textbook, will activate other types of skills through manual activities’ ’ (Teacher, school 8). As it is argued in the literature, working-class students often receive instruction that is more practically oriented, and prepares them for manual, or low-wage service-sector (Diamond, 2007). In this sense, innovation in heterogenous school contexts is associated with process innovations in education, which usually focus on different and innovative teaching methods and that enhance efficiency, within an improved pedagogical approach (Lubienski, 2009).

Finally, in these school contexts, the pressures on teachers to balance innovation and students’ individual development, alongside the demands for standardization and performativity, generates tensions amongst the school actors. Performative pressures lead to negative emotions in teachers, a decreased use of a creative or personalized instruction, and resistance to change and innovation in classrooms. A decline in teaching creativity is associated with mental tiredness, low energy (Teacher, school 7), loss of enthusiasm (Teacher, school 9), and performance-based anxiety and competitiveness” (Teacher, School 12). The tensions between accountability and pedagogical innovation thus become clear.

Disadvantaged school settings

Unprivileged schools, characterized by a working-class student body and very low performance, face significant challenges in implementing innovative pedagogical and organizational processes. These schools, located in the suburbs of the city, struggle with subscription due to a less attractive educational offer. Parents and the public administration are perceived by schools as absent and not supportive of schools’ actions. Schools’ ethos mainly focuses on disciplining and controlling students’ behaviour.

In unprivileged schools, a traditional educational approach prevails, and there are notable difficulties in creating innovative teaching environments. Personnel shortages and overworked leadership teams hinder the implementation of innovative practices. According to teachers working in these schools, a lack of cohesion between staff members further limits the schools’ capacity to introduce laboratory-type or transversal teaching, which demands additional efforts, collaborative planning, and exchanges among teachers.

The principal works in two schools and only comes once a week in this school, and therefore there is a lack of clear guidelines, there is no cohesion, and there is a strong disorientation which affects the possibilities that the school has (Teacher, School 10)

School buildings owned by the public administration are also perceived as ‘obsolete’, with inadequate spaces that hinder the stimulation of innovation in teaching ideas and students’ creativity (Teacher, school 9). The lack of functionality in terms of digital devices further impedes changes. Innovative educational practices like hands-on teaching, personalized instruction, or student group work are constrained by the learning environment and would also require a different type of classroom setting (Teacher, school 10).

For me there is a real space problem and there are very few additional classrooms. Sometimes it would be necessary to work with small groups of students and to have personalized lessons. It may seem obvious but it is not obvious in everyday teachingc. (Teacher, school 12)

The lack of suitable spaces for collaborative and innovative teaching methods thus becomes a significant barrier to the effective implementation of innovative practices. Teachers working in these school contexts emphasize the real challenge of limited space, particularly the shortage of additional classrooms to implement group work or provide personalized lessons, presenting a daily challenge in delivering a diversified and personalized type of teaching.

Experiencing contradictory pressures: filtering and decoupling policy

In terms of responses to the institutional mandates, in unprivileged school settings, teachers select some policy messages in and others out, either engaging or dismissing certain ideas and approaches (Coburn, 2001). The complex and contradictory pressures emanating from the dual mandates create a challenging environment for teachers to manage.

Teachers working in these schools are often against the PBA mandate for different reasons. They view the INVALSI tests as unfair, particularly when perceived as a top-down bureaucratic and statistical requirement that favours high-performing schools without deepening or analysing the contextual conditions of such performances. The standardized tests are either criticized for not considering the schools’ socio-economic background, because the results are too difficult to understand and analyse for teachers, or because they are unrelated to students’ interests and inducing stress and anxiety in students.

We have to lower the emotional filter that is created every time we have a test, we are all quite agitated. (Teacher, school 10)

Teachers in disadvantaged schools also downplay the significance of the PBA model when it does not align with broader school priorities (Teacher, school 12) or lacks an added value (Teacher, school 10). In disadvantaged school contexts, the disconnect between the external assessments and students’ life context, coupled with content misalignment with teachers’ curriculum diminishes teachers’ perceived utility of the PBA model.

The tension between accountability and pedagogical innovation also emerges clear in such school contexts. Teachers express reservations about standardized testing, citing its failure to stimulate students’ reasoning or high-order thinking skills. This conflict is illustrated by the struggle between ticking boxes and fostering deeper understanding and reasoning.

Through philosophy we learn how to think and reason. But if you ask me to tick the box, then I am against this type of testing (…) you see that students just proceed mechanically they answer without even understanding what is being asked of them. (Teacher, school 12)

However, this negative opinion on the PBA system differs slightly according to the school. While the majority of teachers in disadvantaged schools share such negative perceptions, some others witness the diagnostic role of INVALSI tests, believing that it could and should be more often integrated in teachers’ didactics (e.g School 14).

In terms of pedagogical approaches, a traditional teaching model prevails, with face-to-face instruction and a reliance on textbooks being the preferred method. While teachers express ideal preferences for innovative teaching methods (including real-life problems exercises, argumentative teaching and peer learning methodologies), the practical challenges of implementation, such as requiring additional effort or prior preparation, coupled with the perceived need for structured teaching to manage student’s energy and lack of autonomy, results in prioritizing and preferring traditional and structured teaching approaches.

A bit of traditional school must not disappear because otherwise students do not learn the content. (Teacher, school 12)

Yet, the adoption of innovative pedagogical approaches changes according to teachers and their levels of preparation within a school. While some teachers adopt creative teaching practices, the use of digital tools or cooperative learning methodologies, some others, also within the same school, recognize that such practices are not common, and that a traditional method is prevalent especially within the most aged teachers (School 12). In other school contexts, teachers autonomously adopt a combination of both traditional teaching and unstructured dialogic teaching, as well as diversified activities and group work (School 14). These practices are supported by a feeling of being adequately prepared to do so, although this often means ‘being the only one that makes students work in such a way’ (Teacher, School 14). Other barriers recognized by teachers towards the adoption of laboratorial or interdisciplinary teaching are the systemic conditions, such as high teaching autonomy, which renders teaching a solitary practice, as well as limited time for training and inadequate support for continuous professional development. Teachers in unprivileged contexts perceive the mandatory training as ineffective and struggle with a defensive climate against continuous institutional changes, impeding a positive relationship with innovation.

Schools are oppressed by continuous transformations that come from outside and the climate welcoming new things is very defensive. (…) in short, there is a difficult relationship with innovation. (Teacher, school 11)

Consequently, disadvantaged schools often respond symbolically to the institutional pressures, by indicating improvement actions on the internal evaluation (RAV) reports, but often without translating them into substantial instructional activities. This form of decoupling in schools reflects a pragmatic approach to meet external expectations without necessarily translating them into meaningful changes, but continue doing ‘business as usual’ (Hallet, 2010).

This year, there was a decline in math results. So, as a priority in the improvement plan, we needed to write that ‘we will try to improve results in Italian, in mathematics and in a foreign language’. But then, in terms of specific actions, nothing more is done. (Teacher, School 12)

In disadvantaged schools teachers also often adopt instrumental strategies in classrooms, such as extensive preparation focused on test items (see Popham, 2001) during the whole school year and selective use of texts to prepare for INVALSI tests. Such emphasis on direct and fact-centred instruction highlights the pragmatic adaptation of teaching methods to meet PBA demands, as witnessed from the following quotes.

I look for an example of texts that are more of interest to them, for instance selecting some about cyberbullying or drugs (Teacher, School 10)

Sometimes, instead of experiential and hands-on activities, we have to take the history textbook and work in a teacher-centered way (…). We do a lot of test exercises on the tests, since INVALSI provides example exercises, and throughout the year, I mainly work on reading and understanding texts. (Teacher, school 11)

Teachers in disadvantaged schools thus more often employ an ‘instrumental and technical orientation to change’ balancing conflicting priorities and aligning classroom structures with different mandates, even when this is in contradiction with educational and pedagogical values related to innovation (Niesz, 2010). However, despite reservations about the fairness or usefulness of INVALSI, some teachers recognize the need for students to excel within the existing evaluation system.

If the question is if I am against the INVALSI tests as an evaluation system, yes, I am against it and I do not consider it a useful or fair criteria or method of evaluating. But after having said that, these evaluations exist, and since it quantifies student learning, students have to know how to do them as best as possible”.. (Teacher, School 11)

Moreover, the rationale behind adopting teaching to the test strategies is not always the same and may also resemble educational and pedagogical aims, ranging from ‘making students understand the approach adopted by the INVALSI tests’ (Teacher, school 12) to adopting test preparation as a valid and additional way to evaluate students’ learning (school 14) or as a way to discuss about current issues, hence ‘practicing for the INVALSI does not become an end in itself to but also something more about civic education’ (Teacher, school 10).

Such conflicting and diverse responses reflect the challenges in reconciling pressures between accountability and pedagogical innovation that teachers in such settings face and the employment of adaptive strategies to navigate the competing pressures they feel deriving from the two diverse mandates.

Discussion and conclusions

This study explored the intricate relationship between PBA and innovation in the context of educational reforms, focusing on Italian schools, which is an underexplored case from this perspective. Framed by sensemaking and filtering theories, the paper analysed the tension between accountability and innovation at the school level. Through an ideal case analysis, it investigated how school actors interpret and adapt the dual policy mandates, emphasizing the influence of the local school context on these dynamics.

While the Italian PBA system aims to promote innovation (Mentini & Levatino, 2024), trying to align PBA and innovation in practice is not as straightforward. According to the analysis, at the implementation level, school actors reveal diverse interpretations and adaptations of the PBA and innovation mandates based on their understandings and interpretations. Teacher attitudes towards the PBA system play a central role in shaping their responses. Those teachers harbouring negative opinions are more inclined to resist and reject the mandate, adapting teaching-to-the-test and curriculum narrowing practices. Conversely, when teachers understand the PBA as fostering transversal competences and critical thinking, the alignment between policy and practice becomes more apparent. Moreover, given the traditional and content-based educational legacy in Italy, innovation is often associated with a new and ‘alternative’ way of teaching, including active learning, competence-based, project-based activities, hands-on teaching, and pedagogical changes. Thus, when the external assessment is understood in support of such types of approaches, teachers tend to integrate it more easily in their teaching and schools use it as an innovative and improvement device.

The socio-economic school context and performative pressures also emerge as key determinants, influencing how teachers interpret and adapt their practices to the institutional messages. Schools catering to middle and high-class students, partly because they experience less pressure, are also more able to align internal and external accountability to drive innovation, adopting various strategies that meet the expectation of school improvement based on the external test data. Conversely, in low-performing contexts with low-class students, teachers decouple and escape from institutional pressures, employing instrumental strategies to balance conflicting priorities and align with PBA demands. Moreover, ethnically diverse low-income school contexts witness high tensions between the PBA and innovation mandates. School actors in these contexts struggle to balance performative demands with the capacity for creativity, personalized instruction, and the stimulation of deeper understanding and high-order thinking skills. Undesirable outcomes such as intense student test training and traditional teaching approaches become more pronounced in such settings. Finally, in disadvantaged school contexts, despite expressing a preference for innovation, a perceived need for structured teaching as well as practical challenges, leads teachers to rely on traditional methods, highlighting the tension between their educational values and the demands of PBA. On the contrary, in high or middle-income homogenous contexts, with high performance results, the accountability mandate represents a stimulus for diversification in teaching and a competence-based transversal teaching. Yet, instrumental strategies such as teaching to the test are also found in middle income school contexts, however adopted for diverse rationales and intensity depending on the school student population.

Furthermore, a constant pressure to innovate and improve, coupled with the performative and reputational pressure stemming from the PBA mandate, is felt by the majority of school actors, irrespective of the schools’ socio-economic characteristics. Administrative and bureaucratic tasks stemming from the PBA create a burden in some schools, impact teachers’ emotional experiences and influence their practices. Feelings of self-imposed pressure and the negative emotional experiences deriving from the PBA influence the use of creativity and alternative or innovative teaching practices.

The innovation mandate in Italy is however nuanced and ambiguous, associated with diverse pedagogical and organizational features, thus it is understood and adapted to different school agendas and priorities, depending on schools’ socio-economic status. Disadvantaged schools tend to understand innovation as process innovation focusing on teaching and learning, with a greater focus on socio-emotional skills or manipulative and experiential teaching. In heterogeneous schools, teachers adopt alternative teaching methods to overcome student’s learning difficulties and behavioural issues since they are less demanding and respond to more ‘practically-oriented’ learning needs. Schools better positioned use innovative practices to teach high-level order and problem-solving skills or use organizational changes, network collaborations and extra-curricular activities to enrich their educational offer. Interestingly, in privileged school contexts, innovative practices are predominantly used to retain parents and maintain a high reputation, responding to a market- model based on family’s free choice (Lubienski, 2009). Indeed, given the Italian quasi-market education system, pressures do not only come from administrative and bureaucratic forms of accountability, but also from parents’ expectations, who play a role in influencing school practices, despite their educational demands and preferences are far from homogenous. Within upper and middle-class families, some groups of parents claim the use of traditional education models and are tied to academic goals, whereas other groups ask for more experiential-oriented and project-based activities in schools.

Beyond the broad socio-economic conditions, it also emerges that in the analysed dynamics the schools’ leadership and the cohesion among staff play a particular role in mitigating school responses. Schools exhibiting cohesive professional communities and visionary leadership, also display a higher adoption of innovative practices and less superficial responses to the PBA mandate. In fact, stable contracts for teachers and continuous professional self-development contribute to higher cohesion and collaborative teaching, supporting the adoption of institutional changes in schools, extracurricular and interdisciplinary teaching. This tends to disadvantage schools serving low-income students, given the fewer and less stable professional resources that can set a clear educational approach or support long-term educational planning. Moreover, a clear leadership direction in sharing a correct understanding of the accountability policy in schools seems to support an internal accountability and positive learning environment by which teachers are less likely to exercise control over student curriculum and performance.

In the study, it finally emerges that material elements, including school buildings and infrastructure, play a crucial role in supporting innovative methodologies, such as collaborative classes, laboratorial teaching, digital education or outdoor education. Policy implications suggest the need to reinforce structures and create conditions for change and innovation especially in more disadvantaged school contexts. Supporting school autonomy through increased financial and managerial resources and personnel is crucial, highlighting the importance of such type of support especially in larger schools, with higher need of organization, or in disadvantaged schools where management teams and teachers are subject to more frequent changes and internal rotations.

These findings highlight potential avenues for future research that could contribute to a more comprehensive understanding of the complex relationship between PBA and innovation in education. Firstly, there is a need to delve deeper into the emotional experiences of teachers under PBA pressures, examining the impact of PBA on teachers’ well-being, job satisfaction and teaching practices. Secondly, a closer examination of the interplay between schools’ socio-economic contexts and innovation would offer a deeper understanding of how innovation manifests in different settings and the role of the school socio-economic composition in influencing teaching and organizational school practices. Lastly, further research could adopt a comparative perspective and explore the relation between PBA and educational innovation across contexts, investigating how the discrepancies between policy discourse and practical implementation may differ according to countries, identifying both barriers and enablers and considering cultural, institutional and policy variations.

Acknowledgments

I would like to thank my supervisors for their invaluable input and support throughout the research process, the research team and the external reviewers for their insights, suggestions for improvement and expertise which were instrumental in shaping the direction of this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Doctoral Fellowship from La Caixa Foundation [ID100010434], code [LCF/BQ/DR19/11740004].

Notes

1. Process or product innovations in schools typically involves changes in teaching and learning (OECD - Oslo Manual for Measuring innovation).

2. Pedagogical innovations, intended as new changed processes for delivering services, such as new pedagogies and teaching practices (European Commission, 2018) usually includes learner-centred education and interactive instruction (Burns & Paniagua, 2018), active methodologies, student-centred learning and curricular integration (SITES, 2009), the application of skills and knowledge to real-life challenges, in opposition to a traditional teacher-centred classroom (Sahlberg, 2006).

3. Process innovations in education occur at the classroom level, involving teaching and learning, curriculum changes, teaching methods or other programmatic options. Administrative innovations (including marketing or other organisational innovations) instead typically occur in the areas of management, administration, governance of a school and involve substantive changes in the structures or organizational behaviour of schools (Lubienski, 2009).

4. Internal accountability is intended as the alignment of individual values with collective expectations, reinforced by the processes of accountability (Elmore, 2005a). Internal accountability, including forms of teacher and school self-evaluation and assessment, precedes and determines all school responses to their external environment (Elmore, 2005b).

5. The results include both internal student assessment results as well as results achieved in INVALSI national standardized tests, reporting the level of learning achieved in Italian, mathematics and English tests, in relation to schools with similar socio-economic and cultural backgrounds.

6. In literacy, INVALSI tests measures the ability to comprehend a text; in mathematics, the ability to solve a problem applied to real-life problems and the ability to argument; in English they are tied to listening and reading exercises based on real-life situation, and thus are less focused on the acquisition of knowledge, as for instance the grammar.

7. Teacher autonomy is disciplined through the 59/1999 law, which guarantees teachers with a high degree of flexibility in the timing, modalities and activities they consider more appropriate with respect to each student’s individual learning. It establishes that teaching practices are based on the school’s pedagogical plan (POF, Piano d’Offerta Formativa), elaborated by each individual school and encompassing diverse methodological options (art 4 law 59/1999).

8. Working class students typically acquire more context-dependent (restricted codes) of meaning and knowledge compared to privileged students (Hoadley, 2006) and teachers tend to hold lower expectations regarding the future careers and learning achievements of students from economically disadvantaged backgrounds (Tarabini, 2012).

9. https://www.mapparoma.info/mappe/mapparoma32-reddito-dei-romani/

10. https://dati.istruzione.it/opendata/opendata/catalogo/elements1/?area=Scuole

11. There are a total of 283 low-secondary schools in Rome (both public and private) amongst which 72% are public and 27% are private schools (dati.istruzione.it).

12. Comprehensive institutes are school buildings (plessi scolastici) which include all levels of education, with the aim to give educational continuity to students (age 3–14), and are usually located in the same neighbourhood and in proximity to each other. They are also run by the same principal and collegial body.

13. The school performance variable was calculated as the average performance that each school obtained INVALSI standardized tests (available for 2018/2019 or 2020/2021), in language, Italian and mathematics subjects. This data was retrieved from the Ministry of Education portal (Scuola in Chiaro), where public data of each school is available. However, it is important to clarify that while INVALSI data was utilized for school classification, it serves as a convenient starting point and does not signify an authors’ alignment with the epistemological foundations of INVALSI knowledge. On the contrary, the emerging categorization of schools provides a more nuanced classification beyond the confines of INVALSI data.

14. The average income by neighbourhood (urban area) where the school is located is retrieved from the Ministry of Economics and Finances public data, based on Revenue Agency Data (2019).

15. From the 107/2015 school autonomy law, a more complex school organization emerges. Some teachers are appointed as referees for some particular aspects and take part in evaluation/technical committees that are responsible for different bureaucratic aspects, such as the formal filling in of documentations required by the accountability system (Self-assessment report (RAV), and improvement plan (known as ‘Piano di Miglioramento’, PdM).

16. Whereas before the choice of a school was only possible according to the same geographical area, since the 1990s the choice is possible in relation to each school, public and private (Pandolfini, 2009).

17. Diversification (or differentiation) is defined as ‘an increase in the number of options available locally in schooling, usually from the parents’ perspective’. Innovation and diversification are distinct but inextricably related (Lubienski, 2009, p. 19).

18. The INVALSI results are given back to schools highlighting the difference in average compared to schools with similar economic, Social and Cultural Status (ESCS).

19. One of the reasons is related to the design of the tests, since they are given back too late to schools according to teachers (usually at the beginning of the following academic year), so teachers believe they cannot not work on improving their results with the same students. Another reason is related to the fact they are anonymous evaluations and results are not given back to teachers individually for each student. Finally, since 2018 INVALSI results are not high-stakes for students, hence both students and teachers often do not attribute particular importance to them.

20. Item-teaching is when teaching is focused directly on test items or on items much like them, while curriculum-teaching requires teachers to direct their instruction towards a specific body of content knowledge or a specific set of cognitive skills represented by a given test, while (Popham, 2001).

21. Process innovations in education also includes product innovations such as new or substantially different service offered to students (Lubienski, 2009).