Enacting school self-evaluation: the policy actors in Irish schools

ABSTRACT

School self-evaluation is a low-stakes policy recently mandated in Ireland and while schools are becoming more consistent in engaging in this internal mode of evaluation, their engagement has not been uniform. This paper provides new ways of thinking about, understanding, and explaining how school self-evaluation plays out in Irish schools. Subscribing to the view that policies are not simply implemented but enacted through the creative processes of interpretation and translation, this paper shows how school self-evaluation is performed in Irish schools in various ways by various people. We identify numerous policy actors in our qualitative data: narrators, entrepreneurs, outsiders, transactors, enthusiasts, translators, critics, and receivers. This assortment of actors helps to bring school self-evaluation to life but as it comprises heterogeneous entities with varying characteristics, levels of experience, and motivations it is simply not possible for this policy to be implemented in schools as policymakers envisage.

KEYWORDS:

• Education policy
• school self-evaluation
• policy enactment
• policy actors
• Ireland

Introduction

Policy texts are rarely the work of single authors or a single process of production but a series of compromises (Ball, 1993). The policies that are ‘done’ in schools are very often ‘written’ by governments, their agencies or other influential stakeholders, with school staff excluded and instead positioned as implementers (Ball et al., 2012). However, policies do not usually come with specific instructions (Ball, 1997; Braun et al., 2011; Maguire et al., 2015). As Ball et al. (2012, p. 3) explain,

This is in part because policy texts are typically written in relation to the best of all possible schools, schools that only exist in the fevered imaginations of politicians, civil servants and advisers and in relation to fantastical contexts. These texts cannot simply be implemented! They have to be translated from text to action – put ‘into’ practice – in relation to history and to context, with the resources available.

Policy sociology suggests that what happens inside a school in terms of how policies play out is complex and will be mediated by institutionally determined factors (Maguire et al., 2010, pp. 156–157). Policies are very often open to interpretation and thus are translated differently in different contexts. Schools can sometimes generate their own policies that elaborate on and embed aspects of national policymaking into their own cultures and working practices (Braun et al., 2010, p. 548) i.e. policies are enacted rather than implemented. Policy enactment involves the creative processes of interpretation and translation, the recontextualisation of the abstractions of policy ideas into contextualised practices (Ball et al., 2012; Braun et al., 2010, 2011). Interpretations are the ‘meaning-making’ processes where schools ask themselves ‘what does this mean for us?’, ‘do we have to do this?’, ‘does this fit with what we do already?’ (Maguire et al., 2013), ‘who will enact it?’, ‘what does it mean in practical terms?’ (Maguire et al., 2015). Translation, then, is the process of putting policies into action and assigning them symbolic value (Ball et al., 2011a, 2012).

Subscribing to the view that policies are not implemented but enacted, this paper shows how school self-evaluation (SSE) gets performed in Irish schools in various ways by various people. The Irish education system might be considered to be a highly centralised and standardised one in many respects but ‘there exists a considerable degree of discretion in policy and practice at the school level’ (Smyth, 2016, p. 10) and this is reflected in the reality of how SSE plays out in schools . Since 2012/13 post-primary schools in Ireland have been required to engage in SSE but, to date, they do not appear to have done so uniformly. As previous studies on SSE in Ireland have largely focused on its implementation and on the capacity of school staff to do so, this study makes a new and valuable contribution to the literature by providing new ways of thinking about, understanding, and explaining how SSE takes place in Ireland. We subscribe to the view that the staff members in schools are heterogeneous and diverse beings who view and respond to, interpret and translate (or not), policies in various ways. Drawing on qualitative data generated through interviews with staff members in five Irish post-primary schools as part of an Erasmus+ funded project entitled 'Distributed Evaluation and Planning in Schools' (DEAPS), this paper looks at how schools in Ireland put SSE into action through the activities of various policy actors. The project has previously found indications that principals are more positive than the staff in their schools towards the involvement of stakeholders such as parents and students in SSE (see Brown et al., 2020a) and with prior research suggestingthat SSE is not popular with the majority of staff in Irish schools (Brown et al., 2016) and that school leaders can be more supportive and enthusiastic than teachers about the SSE process (Harvey, 2015), this is an area that merits further unpacking.

This paper is particularly pressing as it emerges at a juncture when the system of school evaluation in Ireland, along with many other countries, is seeking to find a way of negotiating the seismic systemic changes posed by the COVID-19 pandemic. A very practical impact of this changed landscape on Irish evaluation and accountability structures has been the decision of Ireland’s Department of Education and Skills to postpone its planned update of the current 2016–2020 quality framework (Department of Education and Skills, 2016a) and SSE guidelines (Department of Education and Skills, 2016b) for post-primary schools until 2021. In June 2020 a circular was issued stating that

the second cycle of school self-evaluation (SSE) was due to cover the period from September 2016 to June 2020 and a third cycle was due to commence in September 2020. The Department of Education and Skills recognises that because of the extended period of school and centre closure that has taken place since 12 March 2020, students, parents, teachers, principals, coordinators and boards of management may have been unable to engage in SSE activity during this time. In view of this, the Department of Education and Skills is extending the second cycle of SSE until June 2021. Consequently, the third cycle of SSE, which had been due to commence in September 2020, will now commence in September 2021. (Department of Education and Skills, 2020, p. 2)

Thus, we argue that this paper is timely and topical and will serve as a useful forewarning to policymakers that schools and their staff members are not homogenous entities that can implement policies in coherent and straightforward manners.

School self-evaluation in Ireland

SSE is an integral part of the inspection and evaluation process in Ireland and is used to complement external inspections. As stated by the Department of Education and Skills, it is ‘primarily about schools taking ownership of their own development and improvement’— SSE is a ‘collaborative, reflective process of internal school review, focused on school improvement’ (Department of Education and Skills, 2016b, p. 6). It is an iterative six-step procedure (see Figure 1) and is mainly performed by school staff in collaboration with other school stakeholders such as parent and students, and typically takes place at the school as opposed to the classroom level via whole-school action plans (O’Brien et al., 2020). It is important to note, however, that, while mandated, SSE is relatively new in Ireland, evolving ‘quite suddenly from being a largely rhetorical concept to a very real imposition on schools and teachers’ (Brown et al., 2017, p. 74) in 2012.

Figure 1. The six-step SSE process (Department of Education and Skills, 2016b).

Previously, Ireland’s school inspection system had broken down by the 1990s and inspections at post-primary level nearly ceased entirely (McNamara & O’Hara, 2012) and at worst became non-existent (McNamara & O’Hara, 2008a). An influential and oft-cited review of education in Ireland by the Organisation for Economic Co-operation and Development (OECD) in 1991 found that the teaching approaches were from a previous century (Mooney Simmie, 2012) and that the teaching culture was very removed from any professional accountability (O’Grady et al., 2018). It was said that the ‘full potential’ of school inspectors was ‘far from being tapped’ and that there was a ‘lack of development of teacher and school self-evaluation’ (Sugrue, 2006, p. 186). The first quality framework was published in 2003 to assist schools in the inspection and evaluation of schools and was designed with an emphasis on cooperation and partnership as opposed to monitoring and accountability (McNamara & O’Hara, 2012; McNamara et al., 2011). Because inspection at post-primary level had been virtually non-existent for so long, and most teachers were able to proceed through their careers without ever having to experience an externally controlled evaluation of their teaching (McNamara et al., 2002), a legacy developed whereby many principals and teachers perceived evaluation as being something that was done to them rather than it being part of their professional responsibility (McNamara & O’Hara, 2005, 2006; McNamara et al., 2009). Thus, evaluation, was, from the start, ‘less threatening and intrusive than approaches taken elsewhere’ (McNamara et al., 2002, p. 202), with external inspection significantly downplayed in the framework (McNamara & O’Hara, 2005, 2006, 2012; McNamara et al., 2011). Given the prevailing culture in Ireland at this time, it is very plausible that SSE was not only considered to be the most effective form of evaluation but the only realistic and achievable approach (McNamara & O’Hara, 2005, 2006).While a culture of evaluation has been developing in Ireland over the past two decades, recent years have seen SSE accelerate. The current approach is relatively new in that the central idea of the original framework – that SSE would be an ongoing process – arguably failed to take hold (McNamara & O’Hara, 2006, 2008b, 2012; McNamara et al., 2011) as schools were under no obligation to engage with the framework. Since the 2012/2013 academic year however, schools in Ireland are now required to engage in SSE. According to Brown et al. (2017, p. 74), the reason ‘for this rapid change of policy can only really be guessed at’:

The Inspectorate would point out that self-evaluation had been part of school evaluation since 2003 and while more honoured in the breach than the observance was always likely to be stepped up at some stage. Also … the increased role accorded to SSE in Ireland was in line with similar developments elsewhere and it is very evident in recent years that inspectorates are now working more closely together and are heavily influenced by new policies and practices in other countries. Finally, the theory and practice of inspection becoming more indirect in the sense of being concerned primarily with overseeing SSE as opposed to conducting hands-on inspections is no doubt welcome in the context of limited resources and falling numbers of inspectors.

One likely reason, however, was Ireland’s poor performance in PISA (Programme for International Student Assessment) 2009. Administered by the OECD, PISA is a triennial international assessment of 15-year-olds’ knowledge and skills in reading, mathematics, and science, and after the release of Ireland’s results in 2010 a national strategy for improving literacy and numeracy was introduced in 2011, followed by compulsory SSE in 2012. For the first iteration of mandated SSE, schools were ‘expected to evaluate three themes in four years: literacy, numeracy and one other theme’ (Department of Education and Skills, 2012, p. 5).

Elsewhere, we have discussed how SSE takes different shapes and forms in different countries (see Brown et al., 2020b); a general outline of how SSE in Ireland compares with SSE in the three other countries in the DEAPS project is available in Table 1. While SSE discourse in Ireland has appeared to become imbued with tones of managerialism in recent years, Irish schools do still operate in a low-stakes accountability environment with little or no negative consequences for schools or teachers (Brown et al., 2020c; Gustafsson et al., 2015; O’Brien et al., 2019) and the emphasis in official discourse is still largely on improvement as opposed to accountability. Given the relatively recent introduction of SSE in Ireland, combined with the absence of strong accountability mechanisms or procedures, how schools perform SSE in Ireland has been rather inconsistent (see for example, Brown et al., 2017; O’Brien et al., 2019). Given that, to date, research in Ireland has largely focused on the implementation of SSE and the capacity of staff to implement it this paper provides new ways of thinking about, understanding, and explaining how schools in Ireland engage with and in SSE by looking at the various policy actors that ‘do’ this policy.

Table 1. SSE in Ireland and the other countries in the DEAPS project.

	Regulatory status of SSE	National guidelines/manuals available for schools	National support service for SSE in Schools	Recommended stakeholders in national SSE guidelines
Belgium	Recommended	No	Yes	None
Ireland	Compulsory	Yes	Yes	Staff, students, parents, board of management
Portugal	Compulsory	No	No	Staff, students, parents, local community
Turkey	Compulsory for primary and vocational technical education	Yes	No	Staff, students, parents

Method

This research is part of a broader Erasmus+ funded project on the role of students and parents in school decision-making in four partner countries. The data presented in this paper were gathered in 2019 through semi-structured interviews with senior leaders, teachers in middle-management roles, and classroom teachers in five post-primary schools in Ireland (information about the five schools and the 39 interviewees is available in Table 2). The research aimed to involve the three main post-primary school models in Ireland: voluntary secondary schools; Education and Training Board (ETB) schools, some of which are known as community colleges; and community/comprehensive schools. The voluntary secondary schools are denominational and although state aided are privately owned and managed, typically by the Catholic Church, while community schools operate under the joint trusteeship ofan ETB, a religious order and/or a bishop or another nominated person, and comprehensive schools operate under the trusteeship of religious denominations (Skerritt & Salokangas, 2020), usually the Catholic Church. While some ETB schools do also have a religious representative on their management board (Liddy et al., 2019), ETB schools are publicly managed and are therefore ‘a distinctive segment of the Irish schooling system’ (O’Flaherty et al., 2018, p. 318). Essentially, the different types of schools can be clustered into three traditions: voluntary secondary schools, ETB schools/community colleges, and community/comprehensive schools (Coolahan, 1995). These school types can also be formally recognised as serving areas of socio-economic disadvantage, and two of the schools in our sample have disadvantaged status.

Table 2. List of participants and participant codes.

Schools	Participant Codes
School A This school is a single-sex (female) voluntary secondary school serving an impoverished area, and the school has disadvantaged status. Less than 200 students attend this school and notably, many do later attend university.	Principal (SAP) Deputy principal (SADP) Home School Community Liaison (SAHSCL) Teacher with responsibility for student council (SASC) Four classroom teachers (SAT1, SAT2, SAT3, SAT4)
School B This school is a single-sex (male) voluntary secondary school. The school is classified as a disadvantaged school as it serves students from low-income families. More than 500 students are currently enrolled. The school often faces problems in terms of students’ behaviour and parents’ engagement, and a major problem is students’ attendance.	Principal (SBP) Teacher with responsibility for SSE (SBSSE) Teacher with responsibility for year group (SBYG) Home School Community Liaison (SBHSCL) Teacher with responsibility for student council (SBSC) Three classroom Teachers (SBT1, SBT2, SBT3)
School C This school is a co-educational ETB school in an affluent area. Over 1,000 students attend this school. The majority of students are coming from privileged backgrounds but students from disadvantaged backgrounds also attend the school, and a small proportion of students are technically homeless. The school is a high-performing one.	Principal (SCP) Three deputy principals (SCDP1, SCDP2, SCDP3) Teacher with responsibility for year group (SCYG) Four classroom teachers (SCT1, SCT2, SCT3, SCT4)
School D This is a single-sex (male) voluntary secondary school. Its 700 students come from various backgrounds— some come from middle-class backgrounds with well-educated parents in high-status employment, and others come from working-class backgrounds. This school is considered to be a high-performing school in terms of students’ academic attainment, and it also has a reputation for sporting excellence.	Principal (SDP) Deputy principal (SDDP) Teacher with responsibility for SSE (SDSSE) Two teachers with responsibility for year groups (SDYG1, SDYG2) Three classroom teachers (SDT1, SDT2, SDT3)
School E This is a co-educational community school serving more than 1,000 students. The students attending this school come from a wide range of socio-economic and ethnic backgrounds and the school caters for a very diverse range of students and academic abilities.	Principal (SEP) Teacher with responsibility for student council (SESC) Teacher with responsibility for year group (SEYG) Three classroom teachers (SET1, SET2, SET3)

Ethical approval was granted by the researchers’ institution (Dublin City University) and informed consent was sought from all interviewees. In advance, schools received a plain language statement about the project, a link to the project website, the researchers’ contact details, and the contact details of the Dublin City University Research Ethics Committee. The plain language statement outlined, inter alia, that participation in this research was completely voluntary, that there was no direct benefit for participants, that participation involved no risk beyond those of everyday life, that maintaining anonymity is of the highest importance, that all data and audio recordings would be stored in a secure location, that the research would be published in peer-reviewed journals, and that participants could, up until publication, withdraw from the process and that in such an event their data would be destroyed. As informed consent is an ongoing process, the interviewer talked interviewees through these points at the start and end of every interview.

Participants were interviewed about their views on and experiences of SSE and the involvement and inclusion of students and parents in school decision-making. With participants’ permission an audio recording was made of each interview and transcribed afterwards. Throughout the data collection, and particularly during data analysis, it became evident that SSE was not simply taking place in a coherent manner but being shaped through the policy work of many different people, and it was here that our attention turned to policy enactment and the work of policy actors. The data were subsequently organised into a predefined set of codes. This analysis was also undertaken in an iterative and cyclical manner however in that we frequently moved back and forth between the transcripts and the literature on policy enactment and policy actors which offered a practical and sensible way of thinking about, understanding, and explaining how SSE plays out in Irish schools. Guided by this scholarship, moments and mentions of enactment in our transcripts were arranged and categorised in line with the literature. As this analysis took place during the first half of 2020, and therefore at a time when we were working remotely due to the ongoing pandemic, reviews of the data and the codes occurred by electronic means. We deliberated over various categorisations and their comparative weightings, as well as the volume of quotes from particular participants. It is worth stressing, however, that the policy actors or positions outlined in Table 3 are not necessarily specific individuals nor are they fixed, unified or mutually exclusive, and some individuals will move between and fulfil multiple roles, while some roles have higher profiles in certain schools (Ball et al., 2011b, 2012), and this is very much reflected in our final presentation of the data.

Table 3. Policy actors and ‘policy work’ (Ball et al., 2011b, 2012).

Policy actors Policy work
Narrators	Interpretaion, selection and enforcement of meanings, mainly done by headteachers and senior leaders
Enterpreneurs	Advocacy, creativity and integration
Outsiders	Entreprenuership, partnership and monitoring
Transactors	Accounting, reporting, monitoring/supporting, facilitating
Enthusiasts	Investment, creativity, satisfaction and career
Translators	Production of texts, artifacts and events
Critics	Union representatives: monitoring of management, maintaining counter- discourses
Receivers	Mainly junior teachers and teaching assistants: coping, defending and dependency

Interpretation and translation: the work of policy actors

Policy enactment involves interpreting and translating policy i.e. turning general policy ideas into contextualised practices (Ball et al., 2012; Braun et al., 2011, 2010). Interpretation is an initial reading of policy, considering what the policy means and if it requires anything to be done; usually it is set over and against what else is in play and what consequences there might be from responding or not responding (Ball et al., 2011a, 2012). Interpretations are meaning-making processes (Maguire et al., 2013) and are instantiated and elaborated in management meetings, staff briefings, and by making someone responsible for a policy, positioning it as a priority, assigning it a value, and selling it to staff (Ball et al., 2011a, 2012). Translation, then, is the literal enactment of policy, using tactics, talk, meetings, plans, events and so on (Ball et al., 2011a, 2012). Although separate processes, interpretation and translation interwove and overlap in that they both involve producing institutional texts, doing professional development, changing structures, roles, and relationships, and very importantly, the identification and allocation of posts of responsibility and the allocation of resources (Ball et al., 2011a, 2012). Sometimes, assigning a staff member responsibility over a policy is the enactment of policy and its embodiment (Ball et al., 2011a, 2012). It is not only teachers involved in interpreting and translating policy, however, while not all teachers actively partake (Maguire et al., 2015). Thus, enactment is intricate, incomplete, and intersubjective in that multiple subjectivities and positions shape how policies are understood, and differences inevitably occur in enactments over time and in different settings (Maguire et al., 2015, p. 487).

According to Ball et al. (2011b, 2012), with the exception of school leaders who receive particular attention, the policy interpretation genre often tends to consider all policy actors to be equal and working on and with policy in similar ways but actors in schools are actually positioned differently and take up different positions in relation to policy, including positions of indifference or avoidance. In Ireland, the current SSE literature shows indications that the ways in which staff members inside Irish schools respond to and engage with and in SSE are indeed varied and complex. For a start, Harvey’s (2015) doctoral research indicates that school leaders can be more supportive and enthusiastic than teachers about the SSE process. Additional studies also find that SSE is not simply implemented in coherent and collective manners by school staff. For example, vignettes of three teachers provided by Murphy (2019) show that while some teachers are engaged in SSE, others engage little, or do not have opportunities to engage. Harrison et al.’s (2020) case study research in one particular school suggests that although some inexperienced teachers can be less involved in school decision-making than their more experienced colleagues it is not necessarily a case of exclusion as junior teachers can be satisfied with not being overly involved in such processes as they focus on finding their feet in the classroom. O’Brien et al.’s (2019) research found that some teachers can be motivated to engage with SSE by a belief that their participation will increase their chances of receiving a promotion. However, it was more commonly found that teachers were agreeing to be involved in SSE for the duration of one school year based on the belief that SSE would be passed on to another group of teachers once they had ‘done their bit’ for the school (O’Brien et al., 2019).

Ball et al., (2011b, 2012) have outlined various kinds of policy actors, or policy positions, involved in making sense of and constructing responses to policy through the processes of interpretation and translation. Using the typology outlined in Table 3, in what follows, this paper discusses the various policy actors involved in SSE in Irish schools, as found in our interview data. As outlined above, and as will be seen in our data, some individuals move between and fulfil multiple roles and some of the roles have higher profiles in some schools (Ball et al., 2011b, 2012).

The policy actors doing SSE in Irish schools

Narrators

According to the typology, narrators are usually school principals or senior leaders. These actors make interpretations of policy and decide what is required to be done and how it can be done. In the Irish context, although low stakes, SSE is now mandatory and schools are obliged to engage in this internal mode of evaluation. Thus, school principals and their teams must comply and ensure compliance throughout their schools. According to those in senior leadership positions, SSE was active and embedded in the fabric of their schools:

If I was to say ‘What is at the core of what we do now?’ I would say ‘It’s SSE’. (SBP)

It’s very live in the school … happening all of the time. (SADP)

An important part of this role is the creation of institutional narratives in relation to policies. Narrators recount narratives about their schools, how they operate and function, and how they strive for improvement. For example, the principal of one school, while being interviewed in a common room in the school pointed mid-interview to the school emblem and slogan on the wall behind the researcher. In doing this, this principal (SAP) referred to the school slogan, ‘Being Outstanding, By Doing Outstanding, Together’ (paraphrased for ethical reasons), as a way of conveying the value placed on not only SSE in this school, but stakeholders’ involvement in SSE. In another school the principal, being interviewed in his office, stopped the interview to provide the researcher with the school’s latest SSE report and School Improvement Plan, and to evidence his vision for the school pointed to the pieces of text he had highlighted and annotated. As part of this narrative, the principal positioned the school as being more advanced and experienced than other schools in terms of SSE:

You got to be mindful too that we were doing school self-evaluation long before anyone else was doing it. (SCP)

Indeed, such narratives can work to create links with or break away from the past. The narrative in this particular school was one of continuity and longevity. For example, one of the deputy principals in this school had worked in the school for several decades and emphasised the longstanding culture the school had of engaging with stakeholders:

I was here from year two of the foundation of the school and the first thing I was told when I came, the school was just a year open, was that we have an open-door policy for parents and that has always been preserved and that’s very much part of how we do things. (SCDP2)

Today, this school was one in which all stakeholders, it was said, were able to contribute to SSE. Another deputy principal explained:

It’s an open culture here whereby people can, and I know for a fact, come in, make a suggestion, or come in with a query, and we think ‘Do you know what?’ and that will end up on the agenda for the senior management team and we’ll say ‘That’s something that we actually do need to suggest’. (SCDP1)

Entrepreneurs

Entrepreneurs promote, advocate, and represent particular policies. These actors are committed to and associated with specific policies as they work to bring about change and engage others in their work. While there were many instances of motivated and committed staff in our data, we did not find many examples of entrepreneurs. In one particular school, however, two members of staff working closely together on the school’s SSE exhibited entrepreneurial characteristics. In the first example, a head of a year group explains how in supporting the staff member with responsibility for SSE they created a ‘SSE team’, recruiting a number of colleagues to help with policy enactment, while the staff body gets updated on their progress:

SSE has dominated the last two to two and a half years of school … we set up an SSE team which meets regularly. We try and bring our findings back to staff. All staff should be aware of it because there’s a massive drive and effort to make them aware. (SDYG1)

In the above example, this staff member is not only working to enact SSE in the school, but he is enlisting others and presenting to all. In doing so, this entrepreneur is making the policy widespread in the school and being recognised as a key actor. Similarly, the teacher specifically responsible for SSE explained that another version of this SSE team, one for students, was a work in progress:

We are looking at setting up something now but it hasn’t been happening up until now. We’re looking to set up a student team. So we have a teacher team at the minute and we’re saying ‘Look, maybe we should set up a student team where we would have a certain number of students who are quite interested in it’. (SDSSE)

Outsiders

The vast majority of policy actors are based inside schools, but not all are necessarily so located. Consultants, advisers, and providers of professional development programmes can, at times, all play important roles in the policy process. Schools often consider the interpretations of external professionals to be valuable, and translations can be supported and facilitated via this expertise. Most members of staff we interviewed had very little awareness of any training available to them in relation to SSE but some participants did speak of engaging in training outside of school. Programmes offered by official support services for teachers were criticised and were not deemed useful:

There’s no point someone coming in and telling us we need to evaluate and we need to reflect or so on—they need to tell us what we actually have to be doing possibly or how we could do this in a more efficient way. (SCT3)

In contrast, however, training being provided by an Irish university was considered useful, and was spoken of in very complimentary terms:

I did that SSE course. That would recommend that whatever theme you are looking at that you engage all the partners in whatever you’re looking at. That was excellent, that was an excellent course. (SADP)

X’s (university) general SSE training facilitated us very well. Every time you had a question you weren’t in the dark. (SDYG2)

For those with experience in both forms of training, it was clear that one mode was regarded as more beneficial than the other:

I did the (university) course with Dr Walsh and that was great but that was in my own time. I did one (official support service) course recently but even at that I felt like I knew it all. (SDSSE)

The influence of outsiders should not go unnoticed. It would appear that some staff members are engaging with outside support, and the interpretations and advice of these outside actors can very well filter into school life.

Transactors

Such is the pressure contemporary schools now face and the nature and reality of teachers’ work that teachers in many schools in various contexts around the world often find themselves having to record and document various practices. Transactors account, report and monitor work and with SSE now mandatory in Ireland a significant portion of interviewees stressed that there were targets and monitoring procedures for SSE in their schools:

We know all the targets, they’re all integrated into our departments. (SAT1)

We’ve also got a noticeboard in the staffroom that’s updated regularly … it’s something the whole staff would know. (SDT2)

Two schools had created posts of responsibility specifically for SSE, where the appointed person would oversee SSE in the school. In one of these schools in particular, many staff members mentioned that SSE was the responsibility of a designated person. Some examples include:

We are very clear that there is a senior post-holder who is responsible for school self-evaluation. (SBP)

It’s part of someone’s post, the school self-evaluation is important in the school. (SBSC)

One person is given overall responsibility to lead the school self-evaluation and that has certainly happened. (SBHSCL)

What was clear from the interviews with the designated post-holders, or the transactors, was that they were engaging in policy monitoring. In the following examples, both transactors explain how they issued questionnaires to students as a way of monitoring teachers:

Teachers might be a little bit more positive saying they’re doing more than they’re doing and students might be saying that we’re doing less then we’re doing, so I am relying on their opinion but sometimes certain things stick out and while it might be slightly exaggerated perhaps by students it gives you an indication of things that are happening or aren’t happening in the class. I do think it’s quite good for getting an insight into quality of teaching. (SDSSE)

The last survey we had was on literacy, numeracy, active learning. We have a big drive on active learning. We wanted to see are teachers actually implementing active learning methodologies, and who’s best to tell you? The teachers will always say ‘Aw yeah, I am’. (SBSSE)

Enthusiasts and translators

In our data we found many instances of enthusiasm for SSE. In all five schools, SSE was spoken of in very positive terms and was promoted as being very active. We found many enthusiasts engaging with narratives that portrayed their schools as being highly engaged with policy. Notable, however, was how the participants doing this were predominantly experienced teachers and not junior staff members¹. These actors spoke encouragingly about policies, suggesting that they are consistently spoken about, frequently the topic of meetings, and a product of school leaders’ visions. The successful establishment of SSE as an active initiative was attributed to the policy work of school leadership teams:

They’re (management) quite proactive. As far as the staff are concerned, I think everyone is on board. SSE is high on people’s minds. That’s down to management being very proactive. You can almost say, I mean, one staff meeting a month is taken up with it. (SDT1)

It’s something that management have really made us aware of. It’s a huge focus in the last couple of years. (SDT3)

All members of staff are engaged and knowledgeable around school self-evaluation. We are given a chance at every meeting to comment, ask questions, to get involved. (SET2)

The school’s approach to SSE is developing and has developed a lot over the last number of years in that people are very much involved in that process. (SET3)

As well as showcasing enthusiasm, enthusiasts very often incorporate policies into their practice. In these instances, we see teachers detailing how they, at the classroom level, contribute to SSE in their schools by enacting internal recommendations of engaging with student voice:

One of the things I would actually do at the end of every year term is, I would have a feedback sheet and so I’m giving the students voice at the end of every term. What am I doing well, what do I need to work etc. I think that’s hugely important. I’ve actually carried it over from my teaching practice days. (SDT1)

My students decided what way we would lay out the room and what way they felt was conducive to effective learning. (SET1)

Enthusiasts can also be translators i.e. they plan and produce events, processes, and texts for policy in their schools. Translation work can dovetail with aspects of the entrepreneurial and transaction work outlined already. For example, the teacher responsible for SSE in School D, as well as being an entrepreneur and transactor, is also a translator:

The surveys. I’ve tried a couple of different ones. I’ve done short ones. The recent ones I did had 30 questions in them. Like a range of different aspects within the classroom with the view to starting a new school improvement plan on active learning. (SDSSE)

The role of translators in particular is not to be underestimated, and principals in particular emphasised the importance of staff members doing what would be called ‘translation’ work. For example, in one school the principal commended the teacher responsible for the student council and the role he played in the school’s SSE: ‘The skill of the guy liaising with them. It’s not accidental that he’s doing that’(SEP). In another school, the principal contrasted two policies in the school, highlighting the important role of translators. A certain policy in the school had faded due to the absence of particular people but on the other hand SSE was active due to an important translator:

The people that we have had leading it have gone on maternity leave, then another person fell ill, we’ve had a bad run of it and it doesn’t get the impetus it needs … since Ms Murphy came on board there is a school self-evaluation plan. (SCP)

Critics

Union representatives and activists might be considered to be policy critics. They can play significant parts in the policy process, but their involvement tends to be minimal. There were mentions in our data of one school having ‘always been a strong union school’ (SBP) and remaining ‘quite a unionised school in many ways’ (SBYG) but as SSE is a considerably low-stakes policy, it is perhaps unsurprising that we found no policy critics per se. However, it is not plausible to think of every staff member being fully supportive of every policy in a school— in this case SSE— and indeed, there were various grievances aired during interviews. For example, it was commonly agreed that time constraints impeded teachers’ involvement in SSE. In other instances, some staff were more critical of how SSE was enacted. In this example, a participant sees SSE as being somewhat disingenuous, for optics, and in the case of stakeholder involvement, tokenistic:

You see the way schools are now sending out questionnaires to families and parents but it’s because either the Department of Education and Skillstold them to or because they know they are going to have a school inspection … .what we’ve been doing really has been ticking boxes up to now so there’s still big changes to be made. (SEYG)

Receivers

Receivers of policy tend to be less-experienced teachers. To these actors, policy is distant and emanates from those in much higher positions such as the government and school management. They rely more on the interpretations of others than their own and are somewhat dependent on their more experienced colleagues when it comes to policy. Teachers in the early stages of their careers can feel overwhelmed and inundated with their various tasks, duties, and responsibilities and so it is of little surprise that policy is not a key concern of theirs. Indeed, relative to other members of staff, teachers near the beginning of their careers tend to be far more removed from the policy process. In the words of one teacher with three years of teaching experience, ‘Coming from a junior staff member you don’t have the same involvement as other staff’(SDT1). Reflective of how some teachers can be passive and depend on others to guide them are the responses of four classroom teachers interviewed together as a group in School C when they were asked if the school had a clear SSE policy. The following exchange took place in response to the interviewer’s question:

It does but I actually haven’t read it. (SCT1)

I actually haven’t read it either … I’m not fully aware of the document itself. (SCT4)

I’m not aware of the document either. (SCT3)

I actually haven’t read the document myself. (SCT2)

It is worth pointing out that of these four teachers, three are relatively inexperienced —one has four years of experience while the other two teachers both have two years of experience. As mentioned above, receivers are usually junior teachers, but more experienced teachers can also be receivers of policy. This is indicated in the above excerpt where a teacher with eighteen years of experience (CST4) had the same policy position as the three junior teachers. Indeed, receivers might be more widespread than this. In the following quotes from the two teachers responsible for SSE in their schools, we see that they are of the view that most of their colleagues occupy passive positions in relation to SSE:

They’re consulted about different things. Whether they would know they’re consulted specifically about SSE I’m not sure to be honest. (SBSSE)

I think SSE is unfortunately something that people think that I do, slightly at the side. That I do these presentations and it’s great, but I don’t know if people fully believe it’s their responsibility as well. I’m just the coordinator, I shouldn’t be the only person driving the bus but I do think there’s a little problem there. (SDSSE)

Discussion

Based on our data, it is evident that there are a wide variety of policy actors in, and sometimes outside of, Irish schools, each playing a role in bringing SSE to life. The typology of policy actors outlined here not only supports previous work on policy enactment and policy actors but builds on and adds to our understanding of existing research in Ireland and accentuates the need to consider how SSE is shaped by various, mainly internal, players. As our data show, entrepreneurs are not necessarily found in every school while critics can be marginal (Ball et al., 2012) and outsiders sparse, but we did find strong evidence in particular of narrators, transactors, enthusiasts and translators, as well as receivers. We found that the staff members doing high-profile policy work in schools tended to be those in senior leadership positions and those assigned posts of responsibility, while for the most part experienced classroom teachers were confident in discussing SSE. On the other hand, junior teachers tended to be receivers of policy, although there are indications that receivers might be far more widespread than this. In particular, the aforementioned finding elsewhere that teachers were agreeing to be involved in SSE for one school year on the basis that SSE would be passed on to another group of teachers once they had ‘done their bit’ for the school (O’Brien et al., 2019) further fuels our suspicion that policy receivers are perhaps more common than first theorised and then found in our data; or, maybe more appropriately, they might be what Golding (2017) has called policy-neutral ‘survivors’. Particularly in the case of more experienced teachers, despite often demonstrating positivity and enthusiasm, it might be that they are actually passive and ‘policy-resistant’:

This role has something in common with that of the ‘receiver’, though the survivor is not trying to ‘receive’ and then enact the policy, but rather adopts a minimal adaptation in order to survive without external censure. (Golding, 2017, p. 927)

In any case, existing literature and the data presented here show that the actors in Irish schools are not equal and are not working on and with policy in similar ways (Ball et al., 2011b, 2012).

Some actors, such as those in senior leadership positions, have no choice but to comply with and enact certain policies— in this case SSE. For others, policies present career opportunities, promotions, and advancements (Ball et al., 2011b, 2012). A policy such as SSE can be used by some teachers to distinguish themselves from others. In the words of a year head with six years of experience:

I might know more about it because there’s promotions and posts and stuff going on in the school. You know, if you want to be going that side of it you should be reading these documents.(SBYG)

However, teachers are at different points in their careers, with different amounts of experience, responsibilities, and aspirations and many are tired and overloaded (Ball et al., 2011b, 2012) and it is perhaps not possible for classroom teachers in particular to be familiar with the ‘ins and outs’ of every policy (Braun et al., 2010). Teachers in contemporary schools can be overwhelmed by the sheer volume of policies they face. As an interviewee with more than 40 years of experience explained:

There’d be a policy in this school on everything. There’s so many policies you could nearly open a library. A lot of us wouldn’t necessarily be familiar with the details of every policy and we couldn’t be, it would be unreasonable. Usually you are familiar with the policies that are most relevant to your own areas of work … Teachers wouldn’t be thinking all the time about the school self-evaluation process. It wouldn’t impinge their, I mean if you’re by and large teaching kids you wouldn’t automatically be thinking about the school self-evaluation process. It’s there in the background and it does happen but it wouldn’t be the most immediate thing as you go about your day to day work.(SBHSCL)

How teachers interpret policies is influenced by existing values, interests and beliefs (McGrath et al., 2017) and recent research in Singapore (Tan, 2017) and Australia (Lambert & O’Connor, 2018) shows that policy enactment is personal. Thus, some staff members are narrators, some are transactors, some are enthusiasts and translators, but some are also receivers of policy, or policy survivors— and some can be combinations of roles. As others have found, positions can manifest and merge, cross and meld (Lambert & O’Connor, 2018), and teachers can adopt a predominant role or more than one role in relation to their enactment of policy (Golding, 2017). It is simply not possible for the staff body as a collective unit to respond to SSE in coherent and consistent ways, and instead, the complex and intricate policy work of various actors combine to make SSE happen.

In line with previous research in Ireland, we found that most schools do not have a specific SSE policy (Brown et al., 2016). We found that in most cases, while staff generally explained how active their schools were when they were asked if the school had a policy and gave the impression that there was indeed a very clear policy, senior leaders and teachers assigned posts of responsibility often conceded that there was no textual SSE policy in their schools:

Everyone knows what we are doing. Is it written down? I don’t think so. (SBP)

If you’re asking if we have a clear written plan, perhaps we haven’t. (SCP)

No, we don’t have a clear policy as such for staff. (SCDP1)

What do you mean by ‘policy’ now? … In terms of policy I would have done some research and then I would have developed a school improvement plan … but in terms of policy we wouldn’t have a set of guidelines. It’s left down to me as to how I want to manage the process. (SDSSE)

This, we suggest, is indicative of how schools in Ireland are producing their own ‘take’ on policy (Braun et al., 2010). There is much narration being done in Irish schools and while entrepreneurs can be elusive and policy receivers, and maybe survivors, operate in the background, and critics and outsiders on the margins, it is transactors, enthusiasts, and especially translators putting unwritten school policies into practice.

Conclusion

SSE is a low-stakes policy recently mandated in Ireland. While schools in Ireland are becoming more consistent in engaging in SSE, this engagement has not been uniform and this paper provides new ways of thinking about, understanding, and explaining how SSE plays out in Irish schools. This paper has shown that schools have collections of different staff members who engage with and in SSE in different ways. These actors are positioned differently and take up different positions in relation to policy and it is the combination of all of these positions that make SSE happen (Ball et al., 2011b, 2012). These policy actors do not simply implement SSE but enact it according to the meanings and commitments they hold, as well as their position in the hierarchy and their relative power (Maguire et al., 2010). It is too much to expect schools and teachers to simply perform a policy as it was expected to be performed when designed by policymakers. Policies cannot be performed in a rigidly uniform manner as schools and the policy actors within them are heterogeneous entities with diverse traits and characteristics and they work on and with policies in various ways. With the current cycle of obligatory SSE in Ireland now due to conclude at the end of the 2020/21 school year, it is imperative that policymakers consider how the staff members in schools should, can, and are likely to engage with and in SSE before the next iteration is rolled out for September 2021.

In focusing solely on policy actors, entirely absent from our analysis is how SSE plays out in different types of schools. Schools have individual histories, buildings and infrastructures, staffing profiles, leadership experiences, budgetary situations and teaching and learning challenges (Ball et al., 2012; Braun et al., 2011; Maguire et al., 2010) which will affect how they respond to policy. For example, new research on the enactment of quality assurance and evaluation policies in Brazil indicates that school actors do not share a homogeneous understanding of these policies and that their enactments can be influenced by their schools’ socio-historic contexts (Candido, 2020). There is a need for research in Ireland to begin exploring the similarities and differences in terms of how SSE and its features such as student voice and parent voice are enacted and experienced in voluntary secondary schools, ETB schools, and community/comprehensive schools, including in both disadvantaged and more privileged settings. We are also conscious of our own positions as external researchers in this study. An insider approach as taken by Golding (2017) in England and Lambert and O’Connor (2018) in an Australian university would help to advance our understanding of enactments in Irish schools. Case study research conducted by insiders offers unique opportunities to investigate schools in ways outsider research might not allow and there is likely to be a better understanding of schools’ histories and micropolitics (Perryman, 2011). With SSE becoming a popular topic for doctoral research in Ireland, we assert that practising teachers adopting insider approaches and investigating how SSE is performed in their respective schools is one way of addressing this and making valuable new contributions to knowledge.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Craig Skerritt

Craig Skerritt is a researcher at the Centre for Evaluation, Quality and Inspection, Dublin City University. He is also the Policy and International Programmes Manager at the Royal Irish Academy and a policy fellow at the Institute for Policy Research, University of Bath.

Joe O’Hara

Joe O’Hara is Professor of Education and Director of the Centre for Evaluation, Quality and Inspection, Dublin City University. A former President of the Educational Studies Association of Ireland, he is currently President of the European Educational Research Association.

Martin Brown

Martin Brown is Associate Professor of Education at Dublin City University and a member of the Centre for Evaluation, Quality and Inspection. He has planned and led evaluations in Ireland, Northern Ireland, Europe and the Middle East.

Gerry McNamara

Gerry McNamara is Professor of Education and Director of the Centre for Evaluation, Quality and Inspection, Dublin City University. From 1997-2007 he was Head of the Dublin City University School of Education Studies.

Shivaun O’Brien

Shivaun O’Brien is Associate Professor of Education at Dublin City University and a member of the Centre for Evaluation, Quality and Inspection. She is Associate Dean for Professional Development and Partnerships in the DCU Institute of Education.

Notes

1. With the exception of SDT1 (three years of teaching experience), these actors had vast amounts of teaching experience. For example, SDT3 had 12 years, SET2 had 22 years, and SET3 had 19 years.