Comparing effects and side effects of different school inspection systems across Europe

Abstract

In this article, different inspection models are compared in terms of their impact on school improvement and the mechanisms each of these models generates to have such an impact. Our theoretical framework was drawn from the programme theories of six countries’ school inspection systems (i.e. the Netherlands, England, Sweden, Ireland, the province of Styria in Austria and the Czech Republic). We describe how inspection models differ in the scheduling and frequency of visits (using a differentiated or cyclical approach), the evaluation of process and/or output standards, and the consequences of visits, and how these models lead to school improvement through the setting of expectations, the use of performance feedback and actions of the school's stakeholders. These assumptions were tested by means of a survey of principals in primary and secondary schools in these countries (n = 2239). The data analysis followed a three-step approach: (1) confirmatory factor analyses, (2) path modelling and (3) fitting of multiple-indicator multiple-cause models. The results indicate that Inspectorates of Education that use a differentiated model (in addition to regular visits), in which they evaluate both educational practices and outcomes of schools and publicly report inspection findings of individual schools, are the most effective. These changes seem to be mediated by improvements in the schools’ self-evaluations and the schools' stakeholders’ awareness of the findings in the public inspection reports. However, differentiated inspections also lead to unintended consequences as principals report on narrowing the curriculum and on discouraging teachers from experimenting with new teaching methods.

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes on contributors

Melanie Ehren is a senior lecturer at the Institute of Education, working on research into effects and unintended consequences of high stakes testing, accountability systems, and school inspections on school improvement and teaching and learning in schools. She has studied the effectiveness of accountability systems and external evaluations on school improvement in many different countries and contexts, for example, as part of the Organisation for Economic Co-operation and Development and UNICEF country review teams and as a project coordinator of a number of EU-funded comparative studies.

Jan-Eric Gustafsson has since 1986 been Professor of Education at the University of Gothenburg. His substantively oriented research focuses on individual differences in cognitive abilities, and on determinants of educational outcomes at individual and system levels. His methodologically oriented research focuses on conceptual and technical issues of measurement within classical and modern test theory, and on the application of latent variable models to both non-nested and nested data.

Herbert Altrichter is Full Professor of Education and Educational Psychology at Johannes Kepler University, Linz, Austria, and member of the executive committee of the European Educational Research Association (EERA). His research interests include educational governance studies, school development and system reform, evaluation, teacher education and qualitative research methodology.

Guri Skedsmo is working part time as Associate Professor at the Department of Teacher Education and School Research, University of Oslo in Norway and is a Senior Researcher at the Institute of the Management and Economics of Education at the University of Teacher Education Zug in Switzerland. Her research interests include school governance, evaluation, and school leadership. The last 10 years she has been actively involved in developing professional development programmes for school leaders and she teaches at the Master Programme in Educational Leadership and the National Programme for Principals at the University of Oslo. She is also co-editor of the book series Policy Implications of Research in Education (Springer).

Stephan Huber is the Head of the Institute for Management and Economics of Education (IBB) at the University of Teacher Education of Switzerland Zug (PHZG). His research is on organisational education, system counselling, education management, school effectiveness, school improvement, school management, professionalisation of teachers and educational leadership personnel (school leaders), and international comparative research on education.

David Kemethofer studied Sociology at Linz University, Austria, and currently works at the Department of Education and Educational Psychology at Linz University, Austria. His main research interests are educational improvement and quality assurance and evaluation with a focus on school inspections in an international perspective.

Notes

1. Details of sampling design, participation rates and technical details of the CFA and path modelling are presented by Gustafsson, submitted and can also be found on www.schoolinspections.eu.

2. Direct effects include the estimates of the standardized regression coefficients for the dummy variable (representing each of the four inspection models) with respect to each of the latent variables in the model. Total effects are effects of the dummy on each latent variable, which is the sum of the direct effect, and all the indirect effects via other variables in the model.

3. http://www.theguardian.com/education/2014/oct/17/ofsted-tells-teachers-what-not-to-do-in-effort-to-dispel-inspection-myths