Steering in complex education systems. Why similar aims can have dramatically different results.

ABSTRACT

In the wake of neo-liberal informed global trends to set performance standards and intensify accountability, the Dutch government aimed for ‘raising standards for basic skills’. While the implementation of literacy standards was hardly noticed, the introduction of numeracy standards caused a major backlash in secondary schools, which ended in a failed introduction of a high stakes test. How can these major differences be explained? Inspired by Foucault’s governmentality concept a theoretical framework is developed to allow for detailed empirical research on steering processes in complex systems in which many actors are involved in educational decision-making. A mixed-methods multiple embedded case-study was conducted comprising nine school boards and fifteen secondary schools. Analyses unveil processes of responsibilisation, normalisation and emerging dividing practices. Literacy standards reinforced responsibilities of Dutch language teachers; for numeracy, school leadership created entirely new roles and responsibilities for teachers. Literacy standards were incorporated in an already used instrument which made implementation both subtle and inevitable. For numeracy, schools distinguished students by risk of not passing the new test affirming the disciplinary nature of schools in the process. While little changed to address teachers main concerns about students’ literacy skills, the failed introduction of the numeracy test usurped most resources.

KEYWORDS:

• Steering dynamics
• governmentality
• complex education systems
• raising standards
• the Netherlands
• secondary education

Introduction

In 2010, something peculiar happened in Dutch education policy. After almost a century of constitutionally protected comprehensive school autonomy, and with no history of a national curriculum, a law introduced compulsory performance standards for literacy and numeracy (Waslander 2010). The stakes are high. If the Dutch Inspectorate of Education finds schools to be at risk of underperformance on these ‘basic skills’, the government can intervene and withdraw funding as an ultimate resource, which means schools need to close. Although the Council of State (2008) argued that the ‘Good Education, Good Governance’ bill runs counter to the constitution, parliament passed the bill without so much as a blink (see Waslander 2010 for a reconstruction). This peculiar feat testifies to just how much Dutch policy has become drawn into a neo-liberal informed global movement of public sector reform that has also included education in many countries (Verger, Fontdevila, and Zancajo 2017). Features and consequences of this education reform movement in national contexts have been subject of ample scholarly work (e.g. Singh 2015; Sifikakis et al. 2016; Verger and Curran 2014). More recently this line of work is extended to new distributed forms of educational governance involving state, market and private actors (e.g. Hartong 2015). Education systems are now ‘complex’ meaning that a wide variety of actors exercise influence from multiple layers and multiple centers (Ostrom 2010). In public administration this development is referred to by concepts such as ‘New Public Governance’ (Osborne 2010), multilevel governance (Pierre and Peters 2005) or network governance (O’Toole and Meier 2004). Knowing what kind of steering processes emerge in such complex education systems is crucial for our understanding of power relations, but they are as yet largely unknown (Theisens, Hooge, and Waslander 2016).

This study aims to contribute to this debate by developing a theoretical framework based on Foucault’s concept of governmentality to enable empirical analyses of steering processes in complex education systems. The framework is used to analyse in detail what the introduction of performance standards for literacy and numeracy brought about in Dutch secondary schools. As will be shown, despite their common origin the effects of performance standards for literacy and numeracy were completely different. Two questions guide this research:

• (1) What steering processes emerged as a consequence of new performance standards for literacy and numeracy?

• (2) How can these differences be explained?

The structure is as follows. After a short introduction on the context and the policy we develop our theoretical framework. We then describe the methodology of a mixed-methods multiple embedded case-study, including nine carefully selected school boards and a total of fifteen schools. The following section describes the findings and unveils how both overt and covert steering processes emerge that have a real impact on teaching and learning. The concluding section clarifies how – even in a complex system with highly autonomous schools – government policies result in standardization of practices. The findings also point to the production of data trails and new forms of surveillance of teachers.

Policy context

The notion of a global talent competition in which the Netherlands is slowly but surely lagging behind, took center stage when the Parliamentary Inquiry on Education Reform concluded ‘that an alarming downward trend is visible in such basic skills as reading and numeracy’ (POO, 2018: 128). As a response, the government introduced the law ‘Good Education, Good Governance’ stating that students were to meet well-defined minimum levels of proficiency at different points in their education careers. The Minister of Education also came with an Actionplan to ‘raise standards’ (2010). The Actionplan formulated policy aims to ‘systematically improve’ secondary education by defining specific goals on PISA indicators. Additional funding was made available to school boards as part of their block funding: the yearly financial accounts must state how they spend the money and how that contributed to raising standards.

The law and the Actionplan (2010) reflect wide held beliefs behind a much more global development in education which puts the focus on ‘basic skills’ (literacy and numeracy) and adopts a combination of standards and accountability as the main steering mode. These neo-liberal informed beliefs outweighed a long tradition in Dutch education policy, which is well known for its high degree of comprehensive school autonomy (OECD 2016; Hooge 2017). The almost silent introduction of performance standards illustrates just how much the public and political debate on education had become immersed with the all too familiar discourses on global competition based on human capital and the need for rigorous accountability measures to urge change (see also Waslander 2010).

Literacy and numeracy were part of the same law and were addressed in the same Actionplan. But from there on the roads to implementation diverged. Performance standards for literacy found their way into examination guidelines and were incorporated in the standardised national exams for Dutch language. Without much discussion, the policy was implemented in all secondary schools. In contrast, performance standards for numeracy found their way to a special test. The initial goal was that all secondary students would be obliged to take the test from 2013 onwards. During the first phase of implementation grades on the numeracy test were to be stated on the diploma but would have no consequences for passing final examinations. From the school year of 2015–2016 onwards the numeracy test was to be high stakes. For students the test would determine whether or not they pass their exams; for schools the proportion of passing students was to become part of the accountability framework of the Inspectorate.

From the start the new high stakes numeracy test spurred protests, objections and complaints by mathematics teachers and other educational professionals. After tests in pilot schools repeatedly showed disappointing results and signalled that high numbers of students might fail their entire exams because of it, the policy was adapted time after time. During the fieldwork of this study the final phase of implementation was postponed, except for students in the highest track. Students in the highest track needed to meet the proficiency levels for numeracy in order to pass their exams, while students in all other tracks were obliged to take the test but it would not be of any consequence. Other specifics of the policy also came to differ by year and by track. It suffices to state here that lack of clarity, complexity and ever-changing regulations contributed to a major backlash in schools.

At the time of writing the turmoil is far from over. After ongoing resistance by teachers, school principals and members of school boards, and continued disappointing test results, forbearance has turned out to become acquittance. In the fall of 2017 the incoming coalition government agreed that ‘there shall be an alternative for the numeracy test’ without providing further details. Another cycle of policy-making and schools dealing with it all is awaiting.

Theoretical framework

How can the dramatic differences between the introduction of standards – silent and smooth for literacy, noisy and failed for numeracy – be explained? For an answer we need to take into consideration that one of the features of complex systems, like the Dutch education system, is that a wide variety of actors is involved in decision-making (Pierre and Peters 2005; Burns and Köster 2016). To understand steering processes in such systems, it is no longer sufficient to examine how schools respond to policies and incentives coming from central or local governments; actions by other actors such as councils, the inspectorate, public and private organisations supporting schools, educational agencies and boards must also be taken into account. Who the relevant actors are in a specific situation is not a given, but a question that needs answering. Moreover, the answer to the question which actors influence behaviour may be different for different actors. For example: for teachers, principals and boards may be the main actors, while for boards also councils and agencies might be very relevant. The challenge for research is to discern the emerging interaction patterns in the system as a whole. A theoretical framework must provide a lens through which these particular features of complex education systems can be studied empirically.

Our theoretical framework is inspired by Foucault’s work on governmentality, a concept combining government and rationality.¹ Although Foucault used the term in different ways, governmentality is commonly described as leading, directing and managing the conduct of individuals (Foucault 1991; Burchell, Gordon, and Miller 1991). In his historical analysis of modernising societies, Foucault identified fundamental shifts in governance (Foucault 1988, 1995). In very general terms, governmentality evolves from a sovereign or hierarchical form, into a disciplinary form based more on instrumentalisation. In a subsequent pastoral form self-governing is key (Perryman et al. 2017). As societies travel along the path of modernisation, governmentality becomes ever more subtle and covert. Power becomes less visible while the exertion of power is not necessarily less coercive nor does it have less impact.

Foucault illustrated the workings of modern and subtle governance with processes such as normalisation and responsibilisation (Foucault 1988, 1995; Schirato, Danaher, and Webb 2012). Normalisation is a process in which social norms of what is considered ‘normal’ are set and enhanced. Such norms can be enforced in many (unobtrusive) ways. As most people wish to be viewed as normal and feel ashamed if they are not, social norms can be powerful self-regulating guides to conduct behaviour (see also Christie 2006). An implicit norm can have very real and very explicit consequences when it underlies what is called a dividing practice. When people who meet the norm are treated differently than people who do not, compliance is rewarded and non-compliance is disciplined (see also McKay and Garratt 2013). Responsibilisation refers to a process whereby one actor grants another actor more autonomy and also holds this latter actor more accountable. Over time, individual citizens are expected to exert self-control and held increasingly responsible for several areas of life, including educational success (Elchardus 2009; Rose, O’Malley, and Valverde 2006; Torrance 2017).

As a result of processes such as normalisation and responsibilisation, formerly visible symbols of power are replaced by more complex forms of power embedded in specific norms, expectations and regulations. As these norms and regulations act upon many areas of daily life, the exertion of power becomes more widespread and less noticeable at the same time. Such new forms of influence – in which language plays a crucial role – are the essence of governmentality (Rose and Miller 1992).

Foucault further points to the workings of instruments and their disciplinary effects (see also Ball et al. 2011). At the macro level, characteristics and features of a population – such as ‘dropouts’ or ‘special needs students’ – are the starting point of policies. How a population is defined, which characteristics are identified as relevant, and how these characteristics are to be measured, are materialised in instruments. Instruments determine what is to be observed and what is to be acted upon; they communicate what is deemed normal and what is not. At the micro level instruments affect people’s behaviour. A school timetable, for example, communicates where pupils are supposed to be and when. Deviance from this norm can be a motive for punishment. It is exactly this system of punishment and gratification which gives instruments its disciplining effects. This includes mechanisms of self-discipline and self-punishment. As these examples indicate, instruments are the devices through which actors communicate and exercise influence on each other.

The notion of self-discipline is central in Foucault’s governmentality perspective. Actors come to discipline, punish, regulate and promote their own behaviour, i.e. to steer themselves. Such self-steering is a result of the internalisation of exogenous steering by others that is not only manifest in bureaucratic or political control, but also in surveillance, discourse, culture, or habit. Self-discipline can be a conscious and deliberate act, but can also work unconsciously. The flipside of the self-steering coin is that actors can use the same methods to develop and employ agency. Actors need not behave as ‘puppets on a string’ – held by the State or somebody else – but can give their own meaning and make their own choices and decisions. People are therefore subjects and objects of steering at the same time (Ball et al. 2011) implying that people are capable of ‘counter conduct’ (Suspitsyna 2010).

Foucault’s thinking about governmentality, and subsequent work by others such as Rose and Ball, provides a promising base for systematic and empirical research into steering processes in complex education systems. Following this line of thinking our theoretical framework is built on an actor perspective. An actor can steer others, can steer oneself and can be steered by others. Who is involved in steering processes may be very different for different topics under study. For empirical research this means that actors cannot be pre-defined just by means of their formal steering roles. The set of actors to be studied must be extended, while their actual behaviour is to indicate whether and if so how and to what extent actors steer others, steer themselves and are being steered by others. Steering processes are made up of (non-)actions of actors and the interaction between these actors. The network of actors together creates patterns of steering processes which are the object of study.

To identify whether and how actors are involved in steering behaviour, we use three concepts that have already proven their value in studies undertaken from a governmentality perspective. These concepts are 1) thinkable, 2) calculable, and 3) practicable (Rose 1999; Edwards 2002; Rose, O’Malley, and Valverde 2006; Gillies 2008; Suspitsyna 2010).

Thinkable

Any notion of what needs to be steered presupposes a language with which we can speak about a phenomenon as well as the ultimate intention of the steering (‘outcomes’). A notion such as ‘raising standards’, for example, presupposes that we have a language to think about ‘educational standards’ and a terminology to describe these standards. This language and terminology contain premisses about our ability to influence (that is, steer) aspects of educational quality in a goal-oriented way. Steering implies a terminology that gives specific meaning to the intention and the objective of steering. Emanating from what needs steering, come notions about responsibility, about who is to be involved and in what role. Empirical research must therefore not only analyse the language that is used to describe a phenomenon but also focus on what roles are assigned to whom. Thinkable issue refers to notions about what needs steering, while Thinkable roles refers to notions about who is to do what.

Calculable

For steering, some kind of information is required to map the nature and magnitude of a problem, to legitimise the need for policy, and to monitor changes over time. This information can be objective or subjective, hard or soft. Steering objectives are operationalised and required information is collected, analysed and represented in specific ways. In short, what is to be steered is made calculable in one way or another (see, e.g., Suspitsyna 2010). How instruments work out is not defined in advance, nor is it inherent to an instrument itself. For example, a judgement on the quality of a school (‘very weak’) is based on an instrument, but the effects of suchs judgements can be very different (see also Van Twist et al. 2013). Also, the same instrument can be employed at different times for different purposes.

Practicable

The language of steering and the instruments that are used may or may not be translated into actual practices. As actors are both objects and subjects of steering, they can respond to the language and instruments of steering by completely buying into it, adopting it, resisting it, or in a whole range of other ways. This also implies that different actors can interpret and use the same instruments in vastly different ways. In education, steering is ultimately geared towards influencing practices in schools, particularly with regard to teaching and learning. The study of steering processes in complex education systems therefore calls for detailed and systematic empirical analyses in schools in order to reveal whether, how and why steering translates into practices, in what respects school practices may differ from each other and why.

Steering dynamics

The three central concepts together comprise a ‘steering trilogy’. This trilogy can be described for every actor involved in steering. Each actor holds its own notions on a policy area (thinkable issue), on who is responsible for what (thinkable roles), develops and uses instruments in its own particular way (calculable) and translates the former in its own way to behaviour in daily practices (practicable). Systematic analyses of the steering trilogy point to a microphysics of steering by tracing an actor’s steering behaviour in detail. To obtain a complete picture of steering processes, the separate descriptions of the steering trilogy for all actors involved in steering on a particular area, must be combined. All actors together produce a pattern of relations and interactions between the notions on what is to be steered, who is responsible for what, the instruments developed and used, and the translation of these to daily practices in schools. The mutual relations and interactions between the actors can strengthen, transform, neutralise and/or subvert each other. This produces a specific pattern of interaction what we call a steering dynamic. Steering dynamics may differ in all kinds of ways: between boards for the same policy, between policies, between education sectors, in time, and so forth.

Methodology

Our research design is a mixed-methods multiple embedded case-study comprising nine school boards and – depending on size – the selection of one, two or three secondary schools within each of these boards (Yin 2009).² The boards were selected on the basis of four criteria: school board size, tracks offered, location and pedagogical vision/identity. These criteria were considered the most relevant to possible differences in steering dynamics within boards. We made a long-list of possible boards based on the combination of these criteria and asked three experts in the field which combination of boards would best serve our goal of maximum variety. Their advice led to a shortlist of boards with an alternative for each board or combination of boards in case they were not able or willing to participate in the research. Five school boards agreed immediately to participate, for the four other boards a backup was contacted. The nine boards included in this study uphold a wide variety of denominations (e.g. Catholic, Protestant, public), are located in urban, provincial and rural areas, and differ in size (from 1.300 to more than 20.000 students). The selection of schools within the participating boards was also made with maximum variety in mind, considering also willing to participate. In total fifteen schools were included in the study. These schools work with different visions (e.g. Steiner, Montessori) and offer different (combinations of) tracks.

As indicated before, it is an empirical question which actors are involved in steering processes. For each school board we started with an initial selection of potentially relevant actors. The interviews indicated that the pre-selected set of potentially relevant actors covered all actually relevant actors: we did not systematically miss out on a relevant actor. Not all of the pre-selected actors appeared to be equally relevant. Specifics are described in the findings section. At the board level interviews were conducted with the chairperson of the executive board, the chairperson of the supervisory board, and the chairperson of the works council representing employees and parents.³ Informed by these interviews, at the school level interviews were conducted with people at all relevant layers of leadership in the school. In all participating schools we aimed at two group interviews with three teachers each. If schools worked with particular roles for teachers – such as coordinators for numeracy or literacy – these teachers were included in the group interviews.

While all actors in leadership were interviewed, such coverage was unfeasable for teachers. To check whether the interviews with teachers were representative, all teachers of the participating schools were sent a web questionnaire immediately after the school visit. In schools with low replies, we recalled once.

Given the nature of an embedded case-study, the design rests on the participation of all relevant actors. We succeeded to include all but two of the relevant people. In total 116 people were interviewed, 54 of which were teachers. In addition, a total of 320 teachers filled in the web questionnaire.

The topic lists for the interviews followed our theoretical framework, focusing on the meaning respondents attach to raising standards in literacy and numeracy (thinkable issue), considerations about who is or ought to be involved (thinkable roles), whether any particular instruments to collect, analyse, collate or distribute information were used (calculable), and how the policy of raising standards effected their daily work (practicable). To tap into the thinkable element we used a very generally worded vignette so as to avoid as much as possible particular wording or meanings ourselves. Interviewees were then asked with a very open question, what their thinking was about the issue. By nature of their respective roles interviews with different actors varied in the relative focus on each element of the steering trilogy. Interviews with teachers tended to put more emphasis on how policies worked out in daily practice. The web questionnaire consisted of a small number of open ended questions covering the three aspects of the steering trilogy.

The interviews were conducted between September 2015 and May 2016. Transcriptions of the interviews were coded in a number of steps, using MaxQDA. First cycle coding resulted in 1444 coded segments (Miles, Huberman, and Saldaña 2014). As a reflection of the number of teachers that were interviewed, a substantial part of the coded segments (N = 520) refers to the element of practicable. The data reduction steps followed the logic of the theoretical framework. First, coded segments were analysed for each actor and for each element of the steering trilogy separately. Using the summaries feature of MaxQDA, this resulted in a detailed description of the steering trilogy for each actor. In the case of teachers, these descriptions were double checked with the outcomes of the web questionnaire to ensure a representative view. Subsequently, these descriptions were summarised at the level of each participating school, based on the steering trilogies of teachers, team leaders, location leaders and school principals. Separate descriptions were made at the level of participating school boards, based on the steering trilogies of chairpersons of the executive board, supervisory board and works council. Given our design of an embedded multiple case-study, we then conducted within case analyses for each of the schools within the same board. This was followed by a between case analyses across boards (see also George and Bennett 2005). All former steps were conducted separately for literacy and numeracy. During the final step comparisons were made between steering dynamics for literacy and numeracy. All steps and interpretations were double-checked by two researchers. Based on our initial analyses we performed a member check during a day-long meeting with case-study participants in September 2016. Our first interpretations were validated by the participants, while mutual discussions further deepened our understanding. The findings presented below are supported by all data sources; the excerpts are solely from the interviews.

Main findings

In general terms, all actors in schools support the policy aim to improve students literacy and numeracy skills. Teachers in particular find literacy and numeracy very important and in serious need of improvement. The overall findings of the teacher questionnaire can best illustrate this. When asked ‘Do you think, regardless of government policy, there should be minimum standards for literacy and numeracy?’ the vast majority of teachers (84%) answered yes, while an additional 7% wasn’t entirely sure. There are no differences between schools, nor between teachers who teach different subjects or different tracks, nor is there any link with characteristics of the school board.

When it comes to the specifics of the policies and the consequences in schools, the general support crumbles and the findings present a stark contrast for literacy and numeracy. In terms of our framework it all comes down to the different ways in which ‘raising standards’ is made calculable and practicable. The findings for literacy are followed by the findings for numeracy. The description follows the components of the steering trilogy and concludes by characterising the overall steering dynamics.

Literacy

Thinkable issue

Among all the actors interviewed, the importance of literacy is undisputed.

‘ … When you look at how incredibly bad it is [student literacy], I really think, I think it is alarming’ (Teacher)

In fact, teachers consider improving literacy skills even more important than improving numeracy skills.

‘ … Because I notice with chemistry that they have more problems with language than with numeracy. So for this moment, I have a language-concern … they get it alright, but it’s not there on paper …’ (Teacher)

Unlike numeracy, all teachers have to do with literacy regardless of the specific subject they teach. Teachers’ concerns are partly based on their observations that students do not understand the questions on tests and lack the skills to formulate their answers in such a way as to score points.

‘When you look at how sentences are sometimes written down, that they get the answer wrong just because they don’t write it down properly. Or giving one-word answers. Or not understanding the question because they have not read it well for example. That happens an awful lot. And that is the same thing over and over again for all subjects of course. ’ (Teacher).

Attempts to improve literacy skills are welcomed.

Thinkable roles

The new literacy standards were incorporated in national exams for Dutch language. In schools it goes without saying that preparing students for these modified exams falls on Dutch language teachers.

‘About literacy … I don’t know that much about it … Maybe that says enough in itself, it’s not an issue.’ (Team leader)

Many respondents conveyed the desirability of a whole school approach to language acquisition. In their view, student literacy skills could improve much more with a practice acknowledging that ‘every lesson is a language lesson’. Such a whole-school approach requires cooperation and coordination between teachers of all subjects. Some schools have made attempts in the (recent) past to establish such a practice, but it hardly ever comes to fruition.

‘… Well, that’s what I’ve done and eventually that was discussed by all teams and departments: yes, this is how we are going to work. Well, we really don’t work that way. It has been approved and now it’s in the bottom of a drawer again. … It just gets bogged down.’ (Teacher)

When a common practice does get off the ground, it is never sustained for long. Teachers mainly point to practical obstacles, such as insufficient time or not working in the same classroom and forgetting to write the word of the week down. Over time, team leaders lose focus after having put much time in their initial coordination efforts. Asked whether their focus was on common practices for literacy, many team leaders replied along the lines of:

‘No I don’t. No. Why not? There are already so many other goals that occupy my mind.’ (Team leader)

As a consequence practices slacken quickly. Despite an often shared ideal of mutual responsibility of improving language skills, the reality is that literacy is the realm of Dutch language teachers. The introduction of performance standards not just confirmed this responsibility, it reinforced it.

Two Steiner schools in the case studies are a notable exception in this respect. Language is core to the pedagogical vision of these schools. Adhering to this vision is self-evident for all educational professionals, as is a strong focus on language in every learning activity. The introduction of performance standards was a non-issue in these schools.

Calculable

Publishers adapted their teaching methods to adhere to the new standards. Apart from using this adapted material, schools did not introduce any specific or separate instruments to enhance literacy skills in students. Nor did they introduce any specific instrument to monitor student progress in any other way than what they were used to. By using adapted teaching methods the new standards did affect the daily practice of Dutch language teachers somewhat. For most – including teachers of other subjects, team leaders, school principals and board members – nothing much did change.

Practicable

As a consequence of the performance standards for literacy the exams for Dutch language were expected to become more difficult. This put particularly students in lower tracks at risk of not passing. Most schools introduced additional Dutch language lessons for these students. For students in higher tracks, no lessons were added to school timetables. In none of the case study schools was the introduction of extra lessons for lower track students disruptive for the school organisation: they were easily incorporated into existing practices. The additional lessons were taught by Dutch language teachers, given on a class basis and the school timetable was simply extended. Some schools found the resources for these additional lessons within their own budgets, others used the government subsidies to implement the policy of ‘raising standards’.

Steering dynamics: steering by incorporation

The steering pattern between all the steering trilogies of all the actors involved can best be characterised as incorporation. The policy was woven into a discourse of essential basic skills that was widely supported by educational professionals in schools. The new performance standards for literacy were incorporated in national examinations for Dutch language. Implementation was relatively easy for schools. Despite the coercive character of the policy, there was hardly any contestation on the part of educational professionals. Publishers incorporated the new standards into their teaching methods but apart from that no other instruments underwent changes nor were any new instruments introduced to monitor student progress. Schools had no other option than to comply for the very reason that an instrument embedded in their practice was adjusted. Steering was as subtle as it was inevitable.

The policy left roles of educational professionals unchallenged and unchanged. If anything, the policy reinforced ideas about who is primarily responsible for student literacy skills. Members of works councils and supervisory boards do not discuss literacy specifically, while members of executive boards, school principals, team leaders and teachers all point to Dutch language teachers, who in turn embrace their role. Yet, most educational professionals share the belief that language acquisition would improve if all teachers coordinate their efforts in a whole-school approach. The case studies show that Dutch language teachers in several schools initiate such actions, but continued support higher up in the school hierarchy proves to be essential to sustain such efforts. In terms of our theoretical framework, a group of teachers is not in a position to create a steering dynamic that involves the whole school. Clearly, not all actors are created equal.

Numeracy

Thinkable issue

Educational professionals agree on the vital importance of numeracy.

‘It is a basic skill.’ (Teacher)

While the importance of numeracy is emphasised, many respondents voice severe criticism on the policy to enhance such skills.

‘I think it’s just ridiculous that it (the test) was implemented just like that. That’s just silly.’ (Teacher)

The criticism concerns a wide range of policy features, such as the separate test for numeracy; the high stakes nature of the test; the nature of the test itself (‘too wordy’); and the disadvantage for particular groups of students. The most heavily criticised features are the lack of clarity and ever-changing nature of the policy measures, often referred to as a prime example of ‘wandering policy’.

‘The numeracy test in itself is too ridiculous for words … and then to say ‘yes, you have to sit the test, but it doesn’t count’ … you just cannot sell that to students.’ (Teacher)

Content analyses of interview segments show that the standardised test has become synonymous with improving numeracy skills, and numeracy has become the equivalent of passing the test.

Thinkable roles

Unlike literacy, it is not self-evident who is the ‘owner’ of raising numeracy skills.

‘Nobody owns numeracy. We don’t have a department of numeracy. We have a chemistry department. There is a department of biology. There is a department of Dutch language. But a department of numeracy? Nobody owns it.’ (Team leader)

The vast majority of teachers considers numeracy not to be part of mathematics. In some schools the issue is highly disputed by math teachers. One of the fifteen schools integrated the numeracy test into existing school subjects – economics, and management and organisation – but only for one of the tracks. In all other instances effectively a new subject ‘numeracy’ was called into being to prepare students for the test.

‘So we created a kind of numeracy department. That’s what it comes down to. You have a head of department …’ (Team leider)

Many schools hired new teachers and with it came new actors, such as new coordinators and new working groups specifically dedicated to numeracy. In all schools existing actors – such as mentors – were assigned new roles. People in various leadership positions gave a number of reasons for the special status and special treatment of numeracy. One recurring reason was the high stakes nature of the test: failing the test would have great consequences for both students and schools. Another reason was that people in leadership positions tried to design school policies that would bypass teachers and keep disturbance of daily practice to a minimum.

‘We didn’t want to put the burden on the math teachers. We have chosen for a set of measures to ensure that others take care of it.’ (school principal)

The special treatment for numeracy was accompanied by centralisation of decision making to the school level and often the board level. Several chairpersons of executive boards became deeply involved in school policies around numeracy, not only bypassing teachers but also school principals. Confusion about roles and responsibilities was created in the process.

‘Really, a chairperson of the board should not manage teachers directly … the chairperson did talk to them and also send them away with an assignment. And we as school principals …’ (school principal).

In none of the case studies were school policies on numeracy discussed with work councils representing teachers and parents. Several supervisory boards got deeply involved. In one of the case studies, the supervisory board discussed the results not just in general which they would normally do.

‘… We more or less discussed the results by school year and by track, looking at: where is it ok and where is it not ok …’ (Supervisory board)

Calculable

The numeracy test is a national standardised test all students must take. The size of the market and the nature of the test made it worthwhile for commercial publishers to invest in digital learning material. All case study schools use one of these methods which are made up by software generating all kinds of information. Not only is information provided on which student has difficulty with what kind of questions, but also on whether students logged in, how many times they worked on a task and for how long. In a number of schools student counsellors or mentors were given a task in monitoring whether and how students did prepare for the test. They must check whether their students actually do log into the system and are ‘on track’. Mentors use the same software, which then also generates information on them. This allows coordinators and principals to monitor the digital behaviour of teachers and mentors. So, while teachers monitor students, coordinators and principals monitor teachers.

‘We put a coordinator on top of it …he also regularly makes analyses for each class and provides teachers with overviews, saying “look here, your student doesn’t practice”. So that they (the teachers) are managed at the same time … it is a bit … teachers say “oh, it is such a burden”. But in the end, the system shows “yes, dear mentor, but you haven’t looked at it since December. Burden how so!?” … There is also an element of control in it .. ’ (Chairperson executive board)

Practicable

The combination of digital learning material and a focus on students passing the test provokes differentiation in teaching. First, schools make distinctions between students in lower and higher tracks. Students in lower tracks are obliged to take numeracy lessons which are scheduled for whole classes on the timetable. For students in higher tracks the approach relies more heavily on self-study. Two arguments underlie this distinctive approach: students in lower tracks are thought to have more difficulty with self-study, and in lower tracks almost all students are found to lack numeracy skills.

‘We want to have controle over it. … self-study at home for students in lower tracks … we just don’t think that’s really going to work.’ (Team leader)

Secondly, students in higher tracks are not treated as a group, but on an individual basis. Students who show up during monitoring as being ‘on track’ to pass the final test are assumed to study sufficiently by themselves. When monitoring indicates lack of effort or lack of progress, students must take special numeracy lessons. Schools main argument is that resources are used most efficiently this way.

‘Students make progress tests … if they pass they don’t need to come to class, so that remaining students can get more attention to get their numeracy skills up to scratch.’ (Team leader)

Steering dynamics: steering through instruments

One thing ties all the elements of the steering trilogies of all the actors together: the test. Improving numeracy skills became synonymous to passing the test (thinkable issue), the test instigated a new ‘subject’ with new actors and new roles (thinkable roles), the test materialized in digital learning material and software which allowed to measure and visualise progress (calculable), and progress on test scores determined whether and what lessons were offered to students (practicable). The steering dynamics revolves entirely around instruments. This dynamic was launched by the blunt and very visible introduction of a new high stakes test by central government. It caused major resistance in schools and the case studies show several examples of counter-conduct. Mathematics teachers resisted heavily and successfully against inclusion of the test in their lessons. Teachers of other subjects resisted less heavily but equally successfully. Several school principals and executive board members revealed in interviews that they first considered incorporating the test in existing school subjects, such as mathematics or economics. Only one of the fifteen schools who participated in this study was successful in doing so, and only in one of the tracks they offer. Other schools did not follow through, either because school leaders did not want to put more burden on (mathematics) teachers, or because there was a backlash from teachers who felt that enhancing numeracy skills were not part of their responsibility and/or whose lessons were already overloaded. Trying to avoid the disturbance of daily teaching practice, school principals and board members organised bypasses. They introduced new actors (e.g. numeracy teachers, numeracy coordinators), new roles for existing actors (e.g. mentors), new tasks (e.g. monitoring students) and new ways of working (e.g. extra classes for students who are not ‘on track’). These bypasses emphasise the special nature of numeracy and come with more centralised decision making by boards. The high stakes nature of the test sparked an uncharacteristic involvement of executive – and even supervisory boards. The stakes of the test are so high, for students and schools alike, that board members felt that responsibilities cannot be assumed by teachers, team leaders or even school principals.

The basic idea behind setting minimum standards is that all students must meet them. The numeracy test established a norm and passing the test is considered normal. Some students struggle more than others to meet the norm so that some groups of students are more likely to get the message that something is ‘wrong’. Educational professionals are painfully aware of these messages.

‘The difficulty with this is that standards are sometimes imposed on a group who can hardly meet them, and yet again it is the lowest track … and you don’t do them a service with this. Those kids don’t need to hear yet again what they’re not good at … you need to give kids hope and trust.’ (Chairperson executive board)

The process of normalisation resulted in dividing practices within schools, to ensure that as many students as possible would pass the test. Digital learning material enabled and encouraged such practices. The software also allows for close monitoring, ranging from students monitoring their own progress, school principals monitoring students and mentors, to supervisory boards monitoring the school policies of executive boards.

Raising standards in complex education systems

Following many other countries on the neo-liberal road of setting performance standards and holding schools accountable, the Dutch government introduced performance standards to ‘raise standards in basic skills’. Literacy and numeracy were part of the same law and the same Actionplan. Nevertheless, two very different steering dynamics emerged. The case of literacy standards is a prime example of how an existing instrument – the national exam – can be adjusted almost covertly, thereby coercing schools to incorporate new demands. This explains why the implementation in schools occurred silent and seemingly smoothly. The case of numeracy standards is in stark contrast and is all about a new high stakes numeracy test. This test had major impact on daily practices and resulted in the introduction of new actors, new roles, new tasks, new ways of monitoring and new dividing practices in schools. The disruptive nature of the test in schools – aggravated by complex and changing regulations – helps explain why implementation was noisy from the outset and ended up in failure.

In both cases, setting standards translated into processes of normalisation in schools. Efforts in schools became focused on ensuring that as many students as possible would obtain the norm, that is, pass the test or exam. Dividing practices emerged as a result, emphasizing the disciplinary nature of the school. Students in lower tracks were given additional lessons on a whole class basis in both numeracy and literacy, so that they spend more time in school. For students in higher tracks, the regular lessons in Dutch language were considered to be sufficient for literacy, while self-study was the default modus for numeracy. Students in lower tracks are believed to need more discipline – provided by the school – while self-study and fostering independence is considered to be more important for students in higher tracks.

The school boards and schools in this study were selected in such a way that maximum variety in steering dynamics was to be expected. Apart from a few noteworthy exceptions – literacy in Steiner schools, and one track in one school where the numeracy test was incorporated into a school subject – major similarities between schools and boards prevailed. This is the more striking, as Dutch secondary schools are considered to be among the most autonomous in the world (OECD 2016). While in many countries the neoliberal bargain is sold in terms of more school autonomy in exchange for more accountability, the Dutch case exemplifies that even a long tradition of autonomy – protected by the constitution – can be readily sacrificed when international competitive performance is allegedly at stake (Brown and Tannock 2009). The findings also indicate that new distributed forms of educational governance may change the steering modes of central government, but that these new modes are not necessarily less hierarchical nor do they loose enforcing power. Despite the multi-layered nature of a complex education system and despite actors operating from multiple centers of power, government policy can easily result in standardization of practices across schools.

The findings further point to the impact of new technologies and the role of for-profit companies in new modes of governance. Despite it’s failed implementation, the introduction of the nationwide high stakes numeracy test may have rolled out the red carpet for commercial publishers to become ever more closely involved in schools’ daily practices. The introduction of the nationwide high stakes numeracy test created a new market for learning materials. The size of the market (all students) and the nature of the test (learning by rehearsal) made the development of digital learning material worthwhile. When students and teachers use these materials, they leave data trails which publishers can use to develop new products. These digital technologies provide new forms of monitoring and surveillance (Williamson 2015, 2016; Souto-Otero and Beneito-Montagut 2016). Page (2017) points out that new forms of surveillance of teachers are not necessarily focused on teaching itself, but on the simulation of teaching. This is exactly what this study found. Teachers are not monitored for how well they interact with students, but whether they use the software.

Clearly, technology allows for very refined forms of steering. This was particularly visible for students in higher tracks. The software indicates whether these students are ‘on track’, meaning that they follow a ‘normal learning curve’ of progression to be successful for the test, based on averages and algorithms build into the software. Based on these indications given by the software, students can continue their self-study and self-monitoring, or must attend additional lessons.

In all, our study underscores the need for systematic and detailed empirical studies if we are to really understand how steering processes work in contemporary complex education systems, and how these processes work out in schools’ daily practices (Theisens, Hooge, and Waslander 2016).

In closing there is one question left: did the government achieve its aim with the introduction of performance standards for literacy and numeracy? From a perspective of policy implementation, the introduction of literacy standards was an unequivocal success while the introduction of standards for numeracy turned into a fiasco. National research monitoring student progress tells another story (College for Tests and Examinations 2017). Due to the nature of incorporation, it cannot be determined whether student literacy skills have improved because the incorporation of the standards in the exam prohobits the new standards to be distinguished. As for numeracy, the overall proportion of students passing the test has increased, but it went up only by one or two percentage points in some tracks and went down with equally small percentage points in other tracks.

This study points to other, more disturbing conclusions. It can be claimed that the introduction of literacy standards was a failure because very little changed in schools, while the numeracy test exemplifies a success because resistance by educational professionals contributed to the full implementation of the high stakes test. Educational professionals experience many forms of policy alienation (Tummers, Bekkers and Steijn, 2012). The high stakes numeracy test pressed schools into putting a lot of time, energy and money towards a test they considered of little help to students. Even more ironic is the conclusion that a neo-liberal ideology stressing efficiency through competition resulted in schools spending their scarce resources on organisational measures to acommodate government policy, rather than on students whose interests are (arguably) at the heart of these endeavours.

The practitioners in our study unequivocally agree that long term goals and incremental changes are best to support real school improvement. The conclusions of international research on system reform support their plea (Harris 2011). The challenge for policymakers is to come to grips with the counter intuitive and unpredictable nature of steering complex education systems as illustrated in this study. The famous words of H.L. Mencken ‘For every complex problem there is an answer that is clear, simple, and wrong.’ ring truer than ever before.

Acknowledgments

The authors would like to thank two anonymous reviewers for their valuable comments on an earlier version of this paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Netherlands’ Initiative for Education Research (NRO), which is part of the Netherlands Organisation for Scientific Research (NWO) under Grant [405-14-401].

Notes on contributors

Sietske Waslander

Prof.dr. Sietske Waslander is full professor of Sociology of Education at TIAS School for Business and Society, Tilburg University, the Netherlands. She is also a member of the Education Council of the Netherlands. She has published work on subjects ranging from social mobility and equal opportunities, to governance and market mechanisms, to mass-customisation, innovation, curriculum and leadership. She carried out several analyses of public and political debates about education and is currently involved in a national research project evaluating policies to stimulate inclusive education. Her teaching focuses on education policy, governance, innovation and leadership.

Edith H. Hooge

Prof.dr. Edith H. Hooge is full professor of Boards and Governance in Education at TIAS School for Business and Society, Tilburg University, the Netherlands. She is Academic Director of the Executive Master of Management in Education at TIAS. Her research activities revolve around governance and management in education systems and organisations, drawing on social network theory and the concept of governmentality. She teaches in the TIAS programs for professional development of (non-)executives, regularly presides in monitoring committees of governance codes and advises in different public sectors.

Henno C. Theisens

Dr. Henno C. Theisens is professor of Public Governance at The Hague University of Applied Sciences, the Netherlands. His work focusses on effective governance in today’s complex public systems, like education, healthcare and public safety. He has extensive experience in the area of education, particularly on issues related to governance and long-term strategy. Previously, he was an analyst at OECD’s Center for Educational Research and Innovation, and a Senior Research Associate at the Centre for Higher Education Policy Studies, of the University of Twente in the Netherlands.

Cissy J. Pater

Cissy J. Pater (MSc) is an educational researcher and project manager at Kohnstamm Institute in Amsterdam, and a PhD student at the University of Amsterdam, the Netherlands. Her doctoral research is on the workings of education systems - in particular competition between schools, public and private schools and free choice for students and parents - comparing the Netherlands and Sweden. She developed a methodology for explanatory evaluation for the Dutch Ministry of Education and is on the editorial board of the Dutch Journal on Educational Law and Policy.

Notes

1. We focus on Foucault’s governmentality concept as it best fits our main interest in this paper, which is steering processes in complex systems. Foucauldian inspired work taking a power-knowledge perspective pointing to disciplinary differences between literacy and numeracy (e.g. Kanes, Morgan, and Tsatsaroni 2014; Öhman and Quennerstedt 2008) would add to our understanding, but is beyond the focus of this paper.

2. Dutch schools traditionally organise themselves in boards along religious or ideological lines; Dutch school boards are not geographically bound. About two-thirds of Dutch school boards are independent i.e. privately run but publicly funded, and about one-third are public. There is equal financial footing, and only a small number of pupils in primary and secondary education, approximately seven percent, attend fully private schools. School boards vary in size from just one school to more than fifteen schools spread around the country, and are managed by a board. Executive board members are professionals in the sense that this is fulltime paid job, for which they are recruited and appointed by a supervisory board (a non-executive board) acting as their employer. Executive board members can best be compared with superintendents of school districts in the United States of America.

3. Every school board is obliged to have a works council which represents employees and parents. Works councils have legal rights to be informed and involved in decision making by the executive board, and can veto important decisions.