‘Measurable but not quantifiable’: The Swedish Schools Inspectorate on construing “quality” as “auditable”

ABSTRACT

‘Quality’ in education has become a global phenomenon, and is frequently discussed in transnational and national policy. In such discussions, ‘quality’ seems to be both worthwhile and demanded, but at the same time, the concept has also been criticised in previous research as it seems to have attained a ‘common sense’ status. In this context, the role of the Swedish Schools Inspectorate (SSI) is raised, as the SSI is commissioned by the Swedish Government to audit and review ‘quality’ in education. However, as the concept of ‘quality’ is left undefined in policy, the SSI has to enact national policy to review ‘quality’ in its quality audits. Hence, the SSI plays an active part in construing ‘quality’, as it makes it auditable through the indicators and systems it uses when conducting quality audits. Thus, this paper has critically scrutinised underlying assumptions and the discursive effects of how the SSI construes ‘quality’ in different policy proposals. To access and analyse such constructs, this article draws on Bacchi’s (2009) post-structural, Foucault-influenced ‘What’s the problem represented to be?’ (WPR) approach. The results indicate that ‘quality’ is presupposed to be both measurable and elusive at the same time. Hence, the SSI operates in a field of tension, targeting something more elusive than just figures and numbers when it comes to ‘quality’. The discursive effects of these presuppositions have implications for what can be said and thought about ‘quality’ at both national and local levels, as well as for students and school staff who are involved in the educational system.

KEYWORDS:

• Swedish Schools Inspectorate
• quality
• quality audit
• WPR approach

Introduction

Recently, there have been many discussions about quality in education on a global scale (OECD, 2019;; World Bank, 2018). In these discussions, there seems to be general agreement that ‘quality’ is something necessary and worthwhile. However, it is not always clear what ‘quality’ in education actually is, or how it should be achieved (Bagnall, 1994; Ball, 2003; Ozga, Dahler-Larsen et al., 2011). In transnational policy, quality seems to be defined in terms of what it is not (OECD, 2019; World Bank, 2018). In such documents, expressions like ‘lack of quality’ frequently appear in relation to different school forms. The same narrative about quality also exists in Sweden, where quality in adult education policy is defined as something that needs to be improved, hence as a ‘quality problem’ (Mufic & Fejes, 2022). However, even though the concept is left undefined in policy, it is often framed as something that needs to be followed up on and audited (Carlbaum, 2016), due to decentralisation and an increased market orientation within the adult education system (Fejes & Holmqvist, 2018; Fejes et al., 2016). The same shift towards an increased focus on governing by objectives, monitoring mechanisms, and external and internal self-regulation can be recognised in other education systems worldwide (Ball, 2007). Thus, the Swedish approach to the ‘quality problem’ can provide insights that may also be relevant in other settings.

In Sweden, it is the Swedish Schools Inspectorate (SSI) that audits education by order of the Swedish Government. In the Education Act (SSI, 2010:800), which provides instructions for the SSI, it is stipulated that: ‘The SSI shall examine the quality of such education and other activities that are under its supervision or under the supervision of a municipality … ’ (SFS, 2010:800, Chapters 26, §19). Moreover, ‘[t]he review shall refer to the quality of the education or activity audited in relation to objectives and other guidelines’ (SFS, 2010:800, Chapters 26, §20). However, in line with the global quality discourse, a definition of quality seems to be missing as the concept is only related to objectives and guidelines. Thus, the SSI has to enact national policy in order to carry out quality audits, and thereby also plays an active role in construing the concept of quality. Delaney (2018) stresses that the meaning of quality is defined in the choice of measure, an activity that the SSI is engaged in as it reviews and audits ‘quality’ on a regular basis. Consequently, this article provides a contribution by critically scrutinising underlying assumptions and the discursive effects of the SSI’s attempts to construe ‘quality’ during a quality audit process where the SSI seems to be caught between two opposing conceptualisations but only has one solution: to keep on auditing. In other words, the SSI has been tasked with auditing and reviewing ‘quality’, a phenomenon that is often left undefined in adult education policy (Mufic & Fejes, 2022). Throughout the audit process, the SSI plays an active role in the process of defining the concept as it seemingly struggles to work out how to make it auditable.

Previous research on school inspections and quality

In contemporary Swedish adult education policy, the ‘solution’ to the ‘quality problem’ in education seems to be an increased focus on accountability and audit practices (Mufic & Fejes, 2022). However, even though there is a wealth of literature that critically examines discourses of accountability in relation to school inspections and audit processes (Kauko & Salokangas, 2015; Perryman, 2009), little attention has been paid to contextual aspects of audits (Bergh, 2015; Braun et al., 2011; Carlbaum, 2016). Moreover, research on school inspections in Sweden has not engaged with the concept of quality, as it has largely focused on regular supervision¹ (Carlbaum, 2016; Lindgren, 2015; Novak & Carlaum, 2017; Rönnberg, 2014, 2019; Segerholm & Hult, 2018). Thus, this article provides a contribution as it scrutinises how ‘quality’ is construed during a quality audit process that focuses on adult education.

Quality seems to be a concept that is both demanded and stressed in transnational adult education policy (Mufic & Fejes, 2022). However, the quality discourse and the efforts to ensure quality are both questioned and criticised by many scholars (Alderman, 2009; Ozga, Dahler-Larsen et al., 2011; Roberts-Holmes & Bradbury, 2016). Previous research on quality in education – as elaborated in policy – emphasises that quality has attained a ‘common sense’ status (Cottle & Alexander, 2012, p. 632) as an ‘objective reality that can be defined, measured, evaluated and assured’ (Tanner et al., 2006, p. 6). Moreover, the risks of the quality discourse are raised, as talk about ‘not quality’ includes the notion that staff can never deliver enough quality (Hunkin, 2019) and that ‘paradoxical expectations land on a local level’ (Bergh, 2015, p. 590). Hence, previous research (see, Alderman, 2009; Andersen et al., 2009; Dahler-Larsen, 2007; Rudoe, 2020) raises awareness of the concern that complex educational issues, such as quality, cannot be solved or improved by measurement and audit performances. Consequently, as the construing of ‘quality’ during a quality audit process seems to be under-researched, and yet crucial to the ongoing national and international debate on educational quality, this article provides a contribution to the research field, for practitioners and policymakers.

Introducing the ‘What’s the problem represented to be?’ approach

To pursue the aim of scrutinising how quality is construed by the SSI, this study draws on Bacchi’s (2009) ‘What’s the problem represented to be?’ (WPR) approach. This approach has post-structural roots, and targets how policy can be analysed by focusing on problematisations. From this viewpoint, policies are seen as contingent constructs by humans that produce effects (Shore, 2012). The focus on problematisations in the WPR approach underpins the assumption that ‘policy cannot get to work without first problematizing its territory’ (Osborne, 1997, p. 174). In other words, by questioning how ‘problems’ are represented in policy, it becomes possible to scrutinise underlying assumptions and rationalities that influence what it is possible to say and do (Bacchi & Goodwin, 2016).

From this standpoint, policy is also understood as something broader than just texts, as all that is said and done in a discursive practice can be understood as ‘a form of proposal and a guide to conduct’ (Bacchi & Goodwin, 2016, p. 18). In this study, what is said and done by the SSI is considered as such. This view of policy relates to the post-structural notion of government as something that includes more than just legislative institutions. Rather, Foucault used the term ‘conduct of conduct’ (Gordon, 1991, p. 2) to describe this extended conception of government that includes ‘any form of activity that aims to shape, guide, or affect the conduct of people’ (Bacchi & Goodwin, 2016, p. 19). From this point of view, government is seen as a ‘problematizing activity’ (Rose & Miller, 1992, p. 181). Instead of seeing the government as a ‘problem solver’, the WPR approach takes a step back in order to challenge this assumption (Bacchi & Goodwin, 2016). Thus, policy is seen to produce ‘problems’ as specific kinds of ‘problems’ rather than just addressing existing problems. Hence, the way that ‘problems’ are shaped in policy affects what can be said and thought about an issue. This does not mean that the WPR approach disregards existing problems, such as drug use or obesity, that policies may focus on. However, due to its post-structural underpinnings, the scope is shifted to question the way that ‘problems’ are construed in policy to disrupt the certainties around what is seen as a ‘problem’ and how this specific problematisation came to be (Bacchi, 2009).

Identifying, reconstructing and interrogating discursive practices

Grounded in Bacchi’s (2009) WPR approach, the focus of the analysis is on critically scrutinising underlying assumptions and the discursive effects of how the SSI construes of ‘quality’ in different policy proposals. To accomplish this, the analysis drew on the question battery that forms part of the WPR approach (Bacchi & Goodwin, 2016). This approach consists of six different questions that all aim to make it possible to identify, reconstruct and interrogate problem representations in policy proposals. This involves taking policy proposals as a point of departure and then working backwards to identify what the problem is represented to be (p. 20). Even though a policy proposal can contain several different problem representations, the analysis focused on the ‘quality problem’ that was stressed several times in policy documents (Ministry of Education, 2018:71, 2019:06; SSI, 2019), and thereby provided a springboard for the rest of the analysis. Thus, the SSI’s construing of ‘quality’ became the focus point, as the increased demand of audit practices is formulated in policy as a ‘solution’ to the ‘quality problem’.

The next step of the analysis asked the question ‘What deep-seated presuppositions or assumptions underlie this representation of the “problem”?’ (Bacchi, 2009). Here, the focus was to identify underpinning conceptualisations of the problem formulation in the selected material. This involved examining how the ‘problem’ is construed in policy proposals by identifying and reflecting upon ‘unexamined ways of thinking’ that could signal the operation of a ‘particular governmental rationality’ (Bacchi & Goodwin, 2016, p. 21). To achieve this, the analysis focused on identifying key concepts, such as ‘quality’, binaries, such as ‘recipes for success’ and ‘development areas’, and categories, such as ‘measurable’ and ‘elusive’. The second question was: ‘What effects are produced by this representation of the problem?’ (Bacchi, 2009). Here, the analysis took the underlying assumptions of the ‘problem’ as its starting point and then focused on scrutinising the discursive effects. In other words, ‘effects which follow from the limited impose on what can be thought and said’ (p. 15).

As the aim of this study is to scrutinise how ‘quality’ is construed by the SSI, the focus of the analysis has been to identify, interrogate and reconstruct ‘truths’ about the concept of ‘quality’ (Bacchi & Goodwin, 2016). More specifically, a practice that produces such ‘truths’ has been observed, namely an SSI conference (the SSI Day). The SSI Day is an annual conference that the SSI organises to disseminate the results of its audits to school staff in Sweden. In 2019, the theme for the conference was ‘Quality work in upper secondary education and adult education’, and the programme included school inspectors, the Director General and researchers, among others, presenting their work, research and results from recent audits. Some of these presentations were ‘The SSI’s picture of the quality and equivalence of Swedish schools’, ‘How to create high quality upper secondary education and adult education for everyone’ and ‘What can be done for more strategic management?’. The audience consisted of school staff from different municipalities in Sweden. The observations were recorded as field notes taken during the presentations.

Written policy was also included in the analysis, such as a programme sheet and PowerPoint presentations that were used during the SSI Day. Documents that stipulate the SSI’s task, such as the Education Act (SFS, 2010:800), a terminology handbook for quality audits (SSI, 2010) and the SSI’s own formulation of its commission on its website (SSI, 2020a, 2020b), were also included in the analysis. These documents were selected because they provide the foundation for the SSI’s task and contain information about how the concept of ‘quality’ is currently construed by the SSI. Additionally, documents from the quality audit that focused on flexibility and individualisation in municipal adult education and was carried out in 2018–2019, before the SSI Day, were also included in the analysis. The selected documents from the quality audit were one project plan (SSI, 2018a), one directive basis (SSI, 2018b) and one quality audit report (SSI, 2019). This written policy was selected because it was the result of the quality audit presented during the SSI Day. The selected material consisted of approximately 750 pages.

Swedish municipal adult education

The scope of this article is polices and audits pertaining to Swedish municipal adult education (MAE). MAE was introduced in 1968, and has since developed into a school form that encompasses more students than upper secondary school (Fejes & Holmqvist, 2018). In policy, MAE is often introduced as a second chance, thus targeting individuals who need to complete their school diploma or reskill to get a job. The school form is regulated by the Education Act and is a part of the national education system. It is organised by the municipality, is free of charge and has a unified curriculum. However, municipalities can choose to organise adult education in different ways by either outsourcing it, most commonly through procurement to private for-profit or non-profit providers. MAE underwent an organisational shift in the 1990s to meet policy demands for a more flexible and individualised educational system (Henning Loeb, 2007). MAE therefore differs significant from the rest of the school system, as students select the courses they need to reach their goal instead of engaging in entire programmes, and admission is continuous. This means that new students can enter a course every week or month, throughout the year. Moreover, the courses are also offered onsite or remotely, during the daytime or evenings.

All these changes in the organisation of MAE were connected to the decentralisation of the entire Swedish education system and the introduction of governing by objectives in the 1990s (Ozga, Simola et al., 2011). The State stipulated educational goals and municipalities could choose to organise the education as they saw fit, as long as they reached the goals. Therefore, the State focused on auditing and following up on whether municipalities achieved the goals (Bergh, 2010).

Regular supervision and quality audits

The SSI was introduced in 2008 as a new way to improve national equivalence (Rönnberg, 2014), namely ‘every student’s right to quality education and equal opportunities to reach the goals of education’ (Carlbaum, 2016, p. 137). To carry out this commission, the SSI inspects education in Sweden by order from the State. However, the way that the SSI audits adult education differs from how the rest of the school system is inspected. The main difference is that it is the responsible authority for adult education – the municipality – that is audited, and not the individual adult education institution. Hence, the State does not have any detailed knowledge about the schools, compared to the regular school system. Issues of quality are therefore handled in different ways by the State, depending on which educational form it concerns. The SSI carries out two different kinds of audits for adult education. First, there is regular supervision that aims to ensure that all schools comply with the applicable laws, rules and curricula (Segerholm & Hult, 2018; SSI, 2020a). These supervisions have a legal character and greater opportunities for sanctions. Second, there is the quality audit based on national goals and guidelines, supported by research results and proven experience, with a focus on different quality aspects (SSI, 2020b). As the SSI puts it, it is ‘tasked with reviewing the quality of such education and educational activities that are under the authority of the authority’ (SSI, 2019, p. 3). The main purposes of the quality audit are to contribute to development, to describe well-functioning elements and to show success factors. In that respect, the quality audit provides an illustrative case, as it makes it possible to scrutinise underlying assumptions of the SSI’s construal of ‘quality’, as well as its discursive effects. When it comes to the concept of quality, this was described as the most central concept in a terminology handbook for quality audits (SSI, 2010). Here, it is concluded that quality is an ‘empty’ and ‘troublesome’ concept (p. 36). In the report, questions are raised regarding whether the SSI audits quality or goal attainment.

Results

The analysis will begin by highlighting deep-seated assumptions about the concept of ‘quality’, after which it will shed light on the discursive effects of such problem representations.

‘Some say that quality is impossible to measure. We don’t think so!’

The first part of the analysis focuses on deep-seated presuppositions or assumptions that underlie the representation of the ‘problem’. First, the analysis illustrates a presupposition about ‘quality’ as something measurable during the SSI Day. Second, ‘quality’ seems to be construed in binaries, hence as either examples of success or challenges. Third, it seems to be presupposed that ‘quality’ needs to be audited and followed up on in order to be improved.

The first assumption about ‘quality’ as something measurable can be illustrated with the following quotation during one of the presentations at the SSI Day: ‘Some say that quality is impossible to measure. We don’t think so! We are working with it. We will give some examples of what it might be.’ (School inspector, SSI Day.) This quotation indicates that measuring ‘quality’ is something the SSI does on a regular basis, as it is part of its job. However, it is also acknowledged in the quotation that the measurement of quality is questioned by ‘some’. Hence, the activity might not be regarded as unproblematic, even though it is done frequently.

The second assumption about ‘quality’, as something that is often referred to in binary terms, will now be exemplified with quotations from the SSI Day. The first example of binary thinking comes from the programme sheet for the SSI Day, where the different lectures are presented with a brief introduction:

Last year, the Swedish Schools Inspectorate made decisions concerning 3,400 schools. As a result of its high presence in Swedish schools, the SSI has extensive knowledge and experience of how quality can be developed. We see many good examples, but also schools with developmental needs. (Programme sheet, SSI Day)

In this short text, the key concept of ‘quality’ is referred to in terms of either good examples or something that needs to be developed. Hence, to talk about ‘quality’ in this way, some kind of reduction must take place as adult education schools are presented as either being good examples or having developmental needs. The second example of binary thinking is when two representatives from two different municipalities are invited to the SSI Day to talk about educational ‘quality’ challenges and examples of success. Hence, the complexity of the concept of ‘quality’ seems to be omitted when it is referred to in terms of either challenges or successes. As the use of binaries suggests a hierarchy and also provides information about what is seen as desirable, the frequent use of phrases like ‘quality driving factors’ and ‘recipes for success’ (School inspector, SSI Day) seems to construe ‘quality’ as something universal and easily attainable. The recipes for success therefore appear to be contextless as they can be used by anyone, and moreover, they seem to guarantee ‘quality’ success.

The third assumption about ‘quality’ as something that needs to be followed up on and audited in order to exist and be improved can be exemplified with a quotation from one of the PowerPoint presentations used during the SSI Day. One of the slides stated that ‘a clear problem picture and follow-ups lead to quality’ (PowerPoint slide, SSI Day). Or, as the Director General put it: ‘When the agreements are loosely formulated, it becomes more difficult to follow up on the quality.’ To get to the ‘quality’, it seems to be presupposed that it needs to be adjusted from the beginning, otherwise it becomes difficult to audit. Consequently, in order for ‘quality’ to be construed as measurable, clear and explicit categories are needed in order to audit it. Hence, some kind of reduction or simplification must take place to make ‘quality’ measurable, as it cannot be audited in relation to every complex aspect of the adult school’s work. And when ‘quality’ is not easy to measure, due to ‘loose formulations’, it seems to be hard for the SSI (and perhaps also the municipality) to follow up on it and audit it. Put differently, the assumption that ‘quality’ is measurable seems to be underpinned by a presupposition that ‘quality’ can be followed up on and audited if it is formulated the ‘right way’.

‘Quality is more than finances, key figures and numbers!’

The second part of the analysis also focuses on deep-seated presuppositions or assumptions that underlie the representation of the ‘problem’. First, the analysis highlights an assumption about ‘quality’ as something elusive and something more than figures and numbers. Hence, ‘quality’ is construed as both elusive and quantifiable during the SSI Day. Second, it seems to be presupposed that is hard to get to the elusive ‘quality’ behind all the figures and metrics.

The first example of the presupposition of ‘quality’ as ‘something more’ is the statement that ‘Quality work is more than finances, key figures and numbers’ (Director General, SSI Day) and the encouragement to ‘Follow up on more than just knowledge results and key figures! It is important that it is quality in the information that comes up to the board!’ In this quotation, there seems to be an ambition to get to something more than just the things that are measurable and quantifiable, such as numbers. This assumption is in line with a formulation in the directive basis that constitutes the foundation of the quality audit, namely that: ‘The questions should not be answered in the form of quantifiable answers with ratings.’ (SSI, 2018b, p. 14.) Hence, ‘quality’ is construed as something non-quantifiable, as the concept – even though it should be measurable – also seems to contain non-measurable conditions. The second example of this presupposition provides a little more information about what quality could be:

When it comes to quality, it’s hard. I’m more optimistic when it comes to finding quality metrics, it’s a key for us to make a choice. We are building on research and we are trying to find broad quality metrics, not just key metrics but processor-based metrics on students’ needs. (Director General, SSI Day)

In this quotation, it is no longer about the binary situation of ‘quality’ or ‘not quality’. Instead, the construing of ‘quality’ turns out to be more nuanced. It thus becomes more context-based, compared to the universal recipes for success. Moreover, even though ‘quality’ is referred to as measurable, it is also acknowledged to be hard to work with. The SSI is trying to find broad quality metrics that do not seem to exist or are not being used yet. However, even though the concept of ‘quality’ is acknowledged as difficult, the concept seems to be left out or even avoided. As it is hard to work with quality, broad quality metrics are introduced in the SSI’s directive basis as a ‘solution’ to the ‘problem’ of auditing (e.g. measuring) ‘quality’, as the concept makes it possible to measure something ‘to an extent of something’ (SSI, 2018b, p. 14). Hence, this is done without the use of quantifiable figures and key metrics, which the SSI seems to want to avoid. In other words, ‘quality’ is assumed to be measurable, but not quantifiable, as it also contains aspects that are hard to quantify. As a result, the SSI seems to struggle to find definitions and tools for audit.

During the SSI Day, the Director General announced: ‘As a small teaser, the Schools Inspectorate will soon offer an audit of education on contract. Principals are good at playing with numbers, but the qualitative aspects are lacking.’ (Director General, SSI Day.) Hence, it is hinted in this quotation that it is the qualitative part of ‘quality’ that is in focus here, as it is pointed out as lacking. Moreover, ‘playing with numbers’ (e.g. quantitative ‘quality’) seems to be construed as insufficient, as it is not what the SSI wants to see. Put differently, the quantitative ‘quality’ seems to be part of a façade, as the SSI is now interested in seeing the qualitative shortcomings behind it. Hence, in this new audit, there will be a redirection of the SSI’s focus, from easily measurable numbers to the qualitative aspects, hence the elusive ‘quality’.

‘We will now ask what you are actually doing?’

The third and final part of the analysis focuses on the discursive effects that are produced by the representation of the ‘problem’. First, the analysis highlights the effects of the assumption about ‘quality’ as something that can be followed up on as either an outcome or a process. Second, the analysis illustrates the effect of the unchallenged presupposition that ‘quality’ needs to be audited and followed up on. Third, the analysis sheds light on the effects of the ‘quality vocabulary’.

During the SSI Day, ‘quality’ was construed in different ways when it comes to when ‘quality’ can be accessed and audited. When the quantitative aspects of ‘quality’ are in focus – figures and numbers – the importance of the build-up and the afterwork of doing, being and making ‘quality’ is stressed. In other words, the SSI encourages both itself and the municipalities to have a clear problem picture and explicit agreements in order to be able to audit and follow up on ‘quality’. This is why ‘loosely formulated agreements’ and ‘unclear problem pictures’ (quotations from the SSI Day) become a ‘problem’, as they are not easily measured and are consequently hard to follow up on. Hence, ‘quality’ is construed as something that can only be accessed by someone before (e.g. problem picture) and after (e.g. success or shortcoming) it is performed and documented. In effect, the concept of ‘quality’ seems to be limited to audits, documentation and follow-ups of explicit and clear categories. Later on, these follow-ups are transformed and regenerated into universal recipes for success that affect what can be said and thought about ‘quality’. Moreover, the concept of ‘quality’ also has implications for how adult education becomes a practice. Much of the ‘quality’ work that is carried out within the inner organisation is therefore not visible via this way of auditing, as it is carried out between the formulation of the problem picture and the final follow-up.

However, when the elusive aspects of ‘quality’ are in focus, it is the inner workings and processes of the adult education school that are targeted by the SSI and not the outcome. The effects of such an assumption can be exemplified with the following quotation, which is a school inspector’s advice to principals:

Dare to ask questions instead of just focusing on problem areas. The principal shows confidence by asking sharp questions to teachers and following up on the answers they receive, creating a robust foundation for the organisation. (School inspector, SSI Day)

In this quotation, ‘quality’ is construed in a more process-oriented way, compared to the outcome focus on figures and numbers. Hence, ‘quality’ is said to be useful for the internal workings of an organisation, not just as an outcome to be followed up on. However, the advice here to focus on asking questions differs from the previous focus on formulating sharp and explicit problem representations. Instead, it is the questions that the principal asks the teachers that should be sharp. In effect, ‘quality’ is construed as something more internalised and practice-based, as it is the principal and not the SSI who is asking the questions. Consequently, it may no longer be as easy to generate recipes for success, as teachers’ answers to principals’ sharp questions will probably vary between adult education schools. The focus of the process seems to be on making the internal workings of adult education organisations more visible. However, the goal still seems to be to get access to ‘quality’, even though it is believed to be located in some kind of dialogue instead of being documented in PowerPoint presentations.

We now turn to the discursive effects of the assumption that ‘quality’ needs to be measurable, audited and followed up on in order to be accessible. The first example of this unchallenged presupposition comes from the SSI Day, at which the Director General said:

There is a large number of orientations in following up on both upper secondary education and adult education. It should not be such a big step to also focus on the qualitative factors, such as students’ feelings of safety.² If we know that students’ success depends on this factor, then we should also follow up on it!

In this quotation, the need for a more elusive approach to ‘quality’ is stressed in relation to students’ need for safety. According to the quotation, this factor is not part of the extensive ongoing follow-ups, supposedly because it is not as easily measurable as figures and numbers. However, even though the SSI is advocating for a more extended and broader approach to ‘quality’, an external follow-up seems to be presupposed as the best way to develop and improve such factors as students’ safety. Put differently, it seems to be assumed that auditing activities are the only way to approach and access ‘quality’, regardless of whether it is easy or difficult to measure ‘quality’. The second example of this assumption is another quotation from the SSI Day, at which a new method to access the more elusive aspects of ‘quality’ was presented:

This is the SSI’s most common area to criticise. Many municipalities have good systems and a lot of figures are collected, but there is no intermediate link. This is the analysis we want to arrive at in what needs to be prioritised. When we have audited systematic quality work, we are overwhelmed with fancy PowerPoint presentations, but it is something that does not feel right. We will now ask what you are actually doing. How did you arrive at your prioritisations? For example, if you prioritise student health, then we will ask how you came to that conclusion. Can you show us the analysis? It’s a new way for us to work!

Here, the SSI is attempting to get at something more than just numbers and figures with this new way of working. Moreover, the question about what the municipality is actually doing seems to indicate that the old way of working did not get to the point, as it is something that ‘does not feel right’. Nevertheless, the assumption that ‘quality’ needs to be audited remains unproblematised, even though this new way of working seems to target the more elusive aspects of ‘quality’. As the SSI’s focus seems to be shifting from the numbers and metrics to broad quality metrics such as students’ need for security and the inner workings of the MAE, it could be assumed that the control and the need for documentation and follow-ups would decrease. However, as there appears to be no easy way to document the elusive ‘quality’ factors that the SSI wants to audit, there seems to be a risk that the demand for documentation will increase even more. Showing and consequently documenting the inner workings and dialogues of an adult education school seem to be even more extensive tasks. In effect, the assumption that ‘quality’ is non-quantifiable could in fact lead to even more paperwork for school staff, as the SSI becomes more and more involved in what is going on ‘behind the façade’. However, this approach might also have unintended effects, as it might interfere with the autonomy of teachers since this approach to ‘quality’ seems to focus more on micromanagement.

We now turn to the discursive effects of the ‘quality’ vocabulary or language of the different assumptions about ‘quality’. As pointed out before, ‘quality’ seemed to be construed in binaries during the SSI Day. However, when the labels of either being successful or having developmental needs are attached to a specific municipality or adult education school, the local context and pre-conditions of each of Sweden’s 290 municipalities seems to be neglected. On the one hand, the success of these recipes is questionable as the municipalities’ preconditions vary. On the other hand, as the focus is on figures, numbers and examples, resourceful municipalities with advanced systems for presenting data could benefit from such an approach (e.g. the ‘fancy PowerPoint presentations’ that the SSI referred to). As practice becomes divided, a more nuanced picture seems to be left out. Moreover, as these universal recipes for success are disseminated, local knowledge about successes and developmental needs also appears to be omitted. Hence, it seems to be presupposed that it is relatively easy to disseminate the results of a quality audit from one municipality to another. This can be exemplified with a quotation from the SSI Day’s programme sheet:

Where are the shortcomings, and what can the representatives of municipal authorities and schools do to ensure good, high quality education? The SSI Inspectorate highlights what we have seen in our audits and provides examples of successful working methods. (Programme sheet, SSI Day)

As the quotation indicates, the party that does the investigating, auditing and following up also knows what is going on in the adult education school. Another example of this is the commonly recurring phrase ‘No scouting, no clue’ (School inspector, SSI Day). As we have seen before, it was frequently pointed out during the SSI Day that the provided successful working methods are based on what the SSI has seen in its audits and the decisions it made concerning 3,400 schools. If we scrutinise this presupposition further, the successful working methods and recipes for success seem to be introduced as a ‘solution’ to the schools’ ‘quality problems’. Moreover, these shortcomings (e.g. ‘problems’) constitute the foundation of the SSI’s quality audit, where phrases like ‘problem picture’ (SSI, 2018b, p. 1) and ‘risk area’ (ibid., p. 12) are common. Consequently, the question about who has the ‘solutions’ to the school’s shortcomings is closely linked to who owns or has the right to formulate MAE’s ‘quality problems’. As it is the SSI that does the auditing and points out these ‘success’ factors and ‘shortcomings’ based on its understanding of the ‘problem’ and its ‘solution’, it is difficult to think differently about ‘quality’.

Discussion

This paper has critically scrutinised underlying assumptions and the discursive effects of the way the SSI construes ‘quality’ in different policy proposals. It has also given examples of how it is construed in organisational practices such as a conference. First, the analysis has highlighted that ‘quality’ is presupposed to be both measurable and elusive at the same time. Second, the analysis has illustrated the discursive effects of the assumptions and how they affect the way that ‘quality’ is construed. The findings also illustrate how the SSI operates in a field of tension between the demand for more and better ‘quality’ in policy and the critical voices in research that question the very same demands – especially since it is commissioned to audit ‘quality’, a concept that is construed as both ‘elusive’ and ‘hard to get to’ during the SSI day. Hence, the construing of ‘quality’ is affected by these different presuppositions. On the one hand, the assumption about ‘quality’ as something quantifiable seems to be in line with the transnational policy demands for easily measurable quality. On the other hand, the assumption of ‘quality’ as something elusive seems to target something behind the easily measurable figures or numbers, in line with the critique of the quality concept formulated in contemporary research.

However, what seems to be left unproblematised in both assumptions is the aim to measure, and thus audit, ‘quality’. At this point, questions remain about what it is that the SSI wants to access behind the ‘fancy PowerPoint presentations’. Hence, as it is the municipality and not the school that is audited by the SSI, this new strategy seems to be an attempt to get access to what is going on beneath the surface. However, is it possible to capture these complex ‘quality processes’ in the inner organisation of an adult education school in the snapshot that a quality audit provides?

The different assumptions about ‘quality’ in adult education can also be linked to the two different audits that the SSI carries out. Even though regular supervision and quality audit are described as different activities in policy, they appear to merge in practice as there seems to be a lack of language when it comes to educational ‘quality’ in MAE. Thus, questions emerge about what the difference between right or wrong, and between good or less good, really is. There is also the question of how ‘quality’ can be measured in degrees in relation to national goals and guidelines, which the SSI mentions as a part of its task (SSI, 2010, 2018a, 2018b, 2019). As Wilcox and Gray (1996) and Thrupp (1998) point out, audits can function as a technology of government, where audit results are reported in terms of judgements such as ‘failure’ or ‘success’. The way that ‘quality’ is construed affects how it can be talked about and viewed, not only during an audit, but also in other practices.

Wilcox and Gray (1996, p. 65) refer to a belief that ‘inspection provides a “true” picture of a school’. In this context, the SSI’s imperative prerogative is problematised. In its attempts to audit ‘quality’, it sets the tone for what can be thought and said about the concept in educational contexts (Osborne, 1997). Or, as Bacchi (2017, p. 6) puts it, it becomes part of how governing takes place, as the ‘problems’ – in this case the ‘lack of quality’ or the ‘development areas’ – are construed as a part of ‘the real’. In line with this, Lundahl et al. (2015) point out that the actor that controls auditing and reviewing also controls what is being done in teaching. Questions about themes and issues that are not included in the interview guides are thereby raised. Lundström (2017) draws our attention to the fact that variables such as personal development, social inclusion, learning and citizenship are not included in the SSI’s SIRIS database. Instead, most of the indicators focus on ‘a narrow subject discourse’, and thus seemingly target something that is conceptualised as easily measurable (p. 43). In MAE, the focus of the ‘quality discourse’ seems to be to prevent students from dropping out and reaching the stipulated goals by implementing flexibility and individualisation (Mufic & Fejes, 2022). Even though these concepts are commonly used in other school forms as well, they seem to be construed in a different way in adult education in relation to ‘quality’, both because the organisation of MAE differs from the regular school system and because it is audited in a different way.

During the SSI Day, it was said that the broad quality metrics should draw on best practice and science. However, most scholars seem to agree that it is difficult to measure concepts like quality, equality and equity (Andersen et al., 2009; Bagnall, 1994; Carlbaum, 2016; Dahlberg et al., 2007; Hunkin, 2019; Ozga, Simola et al., 2011). Many studies also indicate that there is a risk of the focus shifting and becoming about measuring what is easy to measure, hence ‘missing the target’ (Green, 2011). Even though the SSI, which operates in between the contrasting discourses of encouraging and criticising the quest for the ‘quality grail’ (Mufic & Fejes, 2022), partially seems to want to access something more, the procedures of an audit appear to lack the right tools or methods to achieve its objectives (SSI, 2010, 2018a, 2018b, 2019). Hence, quantifiable ‘quality’ and elusive ‘quality’ seem to be construed differently during the SSI day. It is therefore questionable whether the SSI can achieve the qualitative aspects using the same method it uses to access figures and numbers.

A key solution for maintaining quality in policy seems to be to evaluate quality. But what are the problems that the measurements are a solution to? What also seems to be left unproblematised is how the SSI can be sure that the anticipated effects will appear in both local and national arenas (cf.; Carlbaum, 2016). Questions about whether we want to develop education based on pointing out shortcomings also arise. Is there no other way to develop adult education schools and teaching? And where are the local voices on these topics? What are perceived as ‘problems’ at local schools? Further research could perhaps shed light on how quality ‘problems’ are construed at local schools, and on how quality is enacted between school actors and school inspectors during quality audits.

Acknowledgments

I am grateful to Andreas Fejes and Song Ee Ahnfor their support and constructive comments. I would also like to express my gratitude to Ronny Högberg, Lina Rahm, Helena Colliander, Henrik Nordvall, Sofia Österborg-Wiklund and Johanna Köpsén for their helpful and valuable suggestions on previous drafts of this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Johanna Mufic

Johanna Mufic is a doctoral student in education specialising in adult learning at Linköping University, Sweden. Her doctoral work concerns issues of quality in relation to Swedish adult education and school inspection.

Notes

1. The differences between the two kinds of audits will be explained in more detail later on in this paper.

2. ‘Students’ feelings of safety’ refers to how safe students feel in their learning environment. The quotation thus exemplifies how more qualitative factors (in comparison to the extensive focus on numbers and figures) could also be followed up on within the framework of a quality audit.