What’s wrong with ‘deliverology’? Performance measurement, accountability and quality improvement in English secondary education

ABSTRACT

Informed by the ideology of ‘deliverology’, performance measurement has become a core component of how English schools are held accountable for the quality of their provision. A wealth of research conducted in diverse national contexts where this approach has been influential has suggested that the unintended harms it generates – including a widening of inequalities, a test-driven pedagogic culture and a narrowing of the curriculum – may be outweighing its benefits. In an attempt to circumnavigate such perverse effects, the performance measures used in England have become increasingly sophisticated in recent years, giving them a prima facie plausibility that has led to them being welcomed by some progressive educators as a means of increasing access to high-status knowledge for disadvantaged students. This paper uses data from a survey of English school teachers to interrogate this plausibility. The analysis suggests that when we drill down into the daily life of schools and its underlying logic it becomes increasingly difficult to be comfortable with the progressive defence of the performance measures currently in use; and that, far from improving educational quality, the measures themselves, and the wider deliverology framework with which they are associated, are in certain fundamental respects incompatible with quality improvement.

KEYWORDS:

• Accountability
• deliverology
• performance measurement
• Progress 8
• quality improvement
• social justice

Acknowledgments

We are very grateful to all of the teachers who participated in the research. We would also like to thank Alan Cribb for extremely helpful conversations and insights that have fed into the drafting of this paper and Ken Jones for his advice on the design of the research and helpful feedback during the initial stages of data analysis .

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. The code also had deleterious knock-on effects for the quality of teachers entering the profession and the intellectual breadth and rigour of the teacher training curriculum, since drilling students to pass a mechanical test of the ‘3 Rs’ did not require a broad liberal education for teachers (Rapple 1994).

2. Referred to as the ‘IDACI’ (Income Deprivation Affecting Children Index).

3. GCSE (General Certificate of Secondary Education) qualifications are taken by students in schools in England, Wales and Northern Ireland usually at the age of 16.

4. Often referred to as ‘forced academisation’, this process involves schools being moved outside of local authority control through the issue of an ‘academy order’ by the Secretary of State for Education. Academies are independent, state-funded schools, which receive their funding directly from central government, and are run by a government-approved sponsor. This can be an existing educational institution, or a private or voluntary sector sponsor.

5. Key stage 4 is the stage of education in English and Welsh schools when students are aged 14–16. During this stage, most students work towards national qualifications, usually GCSEs.

6. This categorisation applies to schools failing to meet a ‘coasting standard’ for three consecutive years (DfE 2016a, 2019). The policy was designed to address the assumed ‘complacency’ of schools, often located in ‘leafy suburbs’, deemed not to have stretched their pupils sufficiently and likely to have previously ‘fallen beneath the radar’ because, for example, they had high attaining intakes or had not received a negative Ofsted report (DfE 2015).

7. A school is deemed to have fallen below the floor standard when its Progress 8 score is below −0.5 and the upper band of the 95% confidence interval is below zero (DfE 2016b, 2019).

8. Effectively, the subjects that count towards the EBacc are the GCSE equivalent of what the elite group of Russell Group (2017/18) universities used to refer to at A level as ‘facilitating subjects’, i.e. those that facilitate access to higher study. The list of ‘facilitating subjects’ was abandoned by the Russell Group in 2019 following criticisms that it was contributing to a devaluing of creative and technical subjects (Sharratt 2019).

9. Since we conducted the research the National Union of Teachers has merged with the Association of Teachers and Lecturers to form the National Education Union. At the time of the research in 2016 (and the merger a year later), the NUT, with approximately 400,000 members, was the largest teachers’ union in the UK.

10. Key stage 3 covers the period when students are in the first 3 years of secondary school (and aged 11–14).

11. See footnote 5.

12. Defined by Giroux (1988, 126–8) as ‘active, reflective scholars and practitioners [who can] educate students to be active, critical and risk-taking citizens [and] speak out against economic, political, and social injustices both within and outside of schools’.

13. In framing anachronism and ethnocentricity as validity concerns, we are not here using validity in a technical sense to mean that there is a lack of alignment between the revised GCSE examinations and the knowledge and skills they are purporting to measure. The suggestion is rather that the GCSE examinations are invalid measures of what our participants considered to be worthwhile knowledge.

14. Here we are using the concept of validity in the conventional technical sense of meaning the extent to which something properly measures what it purports to measure.

15. Standardised Attainment Tests.

16. CEM (Centre for Evaluation and Monitoring) is a commercial provider of assessment monitoring systems.

17. See also Nichols and Berliner (2007), Klenowski and Wyatt-Smith (2012), Lingard and Sellar (2013), Ohemeng and McCall‐Thomas (2013), Thompson and Cook (2014), Hofflinger and von Hippel (2018), and Johnson (2018) for similar accounts of such practices operating in other countries where high-stakes testing has been introduced.

18. SEND is an acronym used in English schools for students categorised as having special educational needs and disabilities.

19. Special Educational Needs.

20. English as an Additional Language.

21. Special Educational Needs and Disabilities Coordinator.

22. That is, assessments differentiated by level of challenge.

23. This point is supported by our case study data (reported in Maguire et al. 2019) which suggests that ‘resource poor’ schools are more likely to feel under pressure to prioritise the ‘core’ subjects over ‘many of the experiences that do not obviously relate to official indicators of school success but that enrich secondary schooling for young people’.

24. These findings resonate strongly with those reported in research on other national systems with high-stakes testing regimes. See, for example, Meier and Wood (2004), Lambert and McCarthy (2006), Au (2009), Klinger, Maggi, and D’Angiulli (2011), Ravitch (2011), Polesel, Dulfer, and Turnbull (2012), Polesel, Rice, and Dulfer (2014), Cumming, Wyatt-Smith, and Colbert (2015), Hardy (2015), Picower and Mayorga (2015), Pinto (2016) and Stuart Wells (2019).

25. A phenomenon referred to elsewhere as the ‘datafication’ of schooling (Lingard, Sellar, and Savage 2014), about which there is now a burgeoning empirical literature. See, for example, Selwyn, Henderson, and Chao (2015), Finn (2016), Selwyn (2016), Bradbury and Roberts-Holmes (2017), and Lupton and Williamson (2017).

Additional information

Funding

The research reported in this paper was supported by funding from the National Union of Teachers.

Notes on contributors

Sharon Gewirtz

Sharon Gewirtz is Professor of Education in the School of Education, Communication and Society at King’s College London where she also co-directs the Centre for Public Policy Research. Her research is in the sociology of education and education policy with a particular focus on issues of equality and social justice, teachers’ work, and the changing culture and values of schooling and higher education in the context of managerial reform.

Meg Maguire

Meg Maguire is Professor of Sociology of Education in the School of Education, Communication and Society, King’s College London. She has a long-standing interest in education policy and practice, social justice, the life and work of school teachers, teacher education, and the challenges of inner-city schooling.

Eszter Neumann

Eszter Neumann obtained her PhD in the sociology of education from King’s College London in 2018 and is currently based in the Institute of Advanced Study, Central European University. Her research, which draws on policy sociology, ethnography of education, and action research, concerns education policymaking in England and Hungary, interethnic relations, the production of social inequalities, and normalization processes in schools.

Emma Towers

Emma Towers is a Teaching Fellow in Education Policy in the School of Education, Communication and Society at King’s College London. Before joining King’s, Emma worked for 10 years as a primary school teacher in London schools. Her research interests include teachers’ lives and career trajectories, urban education, and teacher identity.