What is the impact of a principle-based inspection approach?

Figures & data

Table 1. Group model building workshop participants.

	W1	W2	W4	W5	W6	W7	W8	W9	W10	W11	Total
School board member	1	1	1	1	1	2	1	1	1	1	11
Board secretary					1	1		1			3
Director		3			2	3	1	3	3	1	16
Team leader				2	2	1		3	1		9
Teacher	6	4	4	2	4		3	2	1		26
Quality manager		1		1		3	1	3	1	2	12
Other				1				1			2
Total	7	9	5	7	10	10	6	14	7	4	79

Note: The numbering skips from W2 to W4 as Case W3 took place in a school for vocational education and training and had to be excluded for methodological reasons. W = workshop.

Figure 1. Summarizing causal relations model.

Note: R = reinforcing loop; B = balancing loop.

Table 2. Effects of principle-based (PB) inspection on perceptions of the inspectorate’s impact.

	Effective feedback	Bureaucracy	Use of inspectorate’s directions
	β (SE)	β (SE)	β (SE)
Intercept	3.94 (0.09)	2.82 (0.11)	3.24 (0.12)
PB inspection	0.24 (0.04)**	−0.23 (0.05)**	0.15 (0.05)**
Job type	0.08 (0.09)	0.01 (0.11)	0.27 (0.12)*
Sector	−0.02 (0.09)	0.19 (0.12)+	0.02 (0.12)
Round	0.07 (0.09)	−0.22 (0.11)+	0.13 (0.12)
School performance	−0.00 (0.05)	−0.03 (0.06)	−0.15 (0.07)*
Quadratic model	χ^2 = 1.66	χ^2 = 0.43	χ^2 = 1.57
Cubic model	χ^2 = 1.03	χ^2 = 0.15	χ^2 = 1.41
Number of respondents	346	264	304
Number of schools	253	218	232
Variance intercept-only model:
School level	0.06	0.38	0.34
Residual	0.61	0.39	0.57
ICC intercept only	0.09	0.49	0.37

Note: ICC = intra-class correlation.

+p < 0.10. *p < 0.05. **p < 0.01.

Table 3. Effects of principle-based (PB) inspection on outcomes related to educational quality.

	Attention for quality	Ambition to improve quality	Urgency to improve quality	Investments in quality
	β (SE)	β (SE)	β (SE)	β (SE)
Intercept	3.55 (0.13)	4.33 (0.07)	3.18 (0.12)	4.09 (0.09)
PB inspection	−0.03 (0.06)	0.11 (0.03)**	−0.03 (0.06)	0.12 (0.04)**
Job type	0.10 (0.13)	−0.03 (0.07)	0.20 (0.12)+	0.10 (0.09)
Sector	−0.25 (0.13)+	−0.24 (0.07)**	−0.00 (0.12)	−0.12 (0.09)
Round	0.16 (0.13)	0.20 (0.07)**	0.05 (0.12)	0.05 (0.09)
School performance	−0.15 (0.08)+	0.04 (0.04)	−0.16 (0.07)*	−0.14 (0.05)**
Quadratic model	χ^2 = 0.22	χ^2 = 2.69	χ^2 = 0.25	χ^2 = 0.15
Cubic model	χ^2 = 0.88	χ^2 = 15.89**	χ^2 = 1.61	χ^2 = 5.57*
Number of respondents	310	310	307	307
Number of schools	228	228	234	233
Variance intercept-only model:
School level	0.05	0.07	0.30	0.08
Residual	1.07	0.26	0.62	0.44
ICC intercept only	0.04	0.20	0.20	0.15

Note: ICC = intra-class correlation.

+p < 0.10. *p < 0.05. **p < 0.01.

Table 4. Effects of principle-based (PB) inspection on outcomes related to school culture.

	Culture of continuous improvement	Openness	Pride	Professionalization
	β (SE)	β (SE)	β (SE)	β (SE)
Intercept	4.12 (0.07)**	4.12 (0.08)**	4.64 (0.07)**	4.08 (0.08)
PB inspection	0.08 (0.03)*	0.08 (0.04)*	0.14 (0.03)**	0.13 (0.04)**
Job type	−0.04 (0.07)	−0.06 (0.08)	−0.02 (0.07)	0.03 (0.08)
Sector	−0.63 (0.07)**	−0.45 (0.09)**	−0.29 (0.07)**	−0.43 (0.08)**
Round	0.23 (0.07)**	0.27 (0.08)**	0.08 (0.07)	0.17 (0.08)*
School performance	−0.02 (0.04)	−0.02 (0.05)	0.07 (0.04)+	0.01 (0.04)
Quadratic model	χ^2 = 0.40	χ^2 = 0.70	χ^2 = 0.52	χ^2 = 1.51
Cubic model	χ^2 = 3.80+	χ^2 = 3.53+	χ^2 = 5.18*	χ^2 = 1.91
Number of respondents	310	310	310	310
Number of schools	228	228	228	228
Variance intercept-only model:
School level	0.15	0.19	0.11	0.09
Residual	0.31	0.32	0.26	0.37
ICC intercept only	0.32	0.37	0.29	0.20

Note: ICC = intra-class correlation.

+p < 0.10. *p < 0.05. **p < 0.01.

Figure 2. Curvilinear effects of principle-based inspection on outcomes related to educational quality.

Figure 3. Curvilinear effects of principle-based inspection on outcomes related to school culture.

Table A1. Factor loadings for items related to perceptions of the inspectorate’s impact.

Item	Factor 1 – Effective feedback	Factor 2 – Bureaucracy	Factor 3 – Use of inspectorate’s directions
The feedback we receive from the inspectorate helps us to evaluate our work more critically.	0.82	−0.06	0.03
The feedback we receive from the inspectorate helps us to gain insight into our strong and weak points.	0.84	−0.07	−0.02
The feedback we receive from the inspectorate helps us to find solutions when things are not going well.	0.78	0.08	0.03
The feedback we receive from the inspectorate helps us to shape our responsibility for educational quality more explicitly.	0.79	0.08	0.00
The current way of inspection leads to issues being discussed on the school board level while they should be discussed on the school level.	0.16	0.53	−0.10
The current way of inspection leads to less autonomy for the school to make decisions on education-related issues.	−0.03	0.82	0.04
The current way of inspection appears to lead to uniformity among schools operating under the same school board.	0.02	0.65	0.03
The current way of inspection leads to bureaucratization.	−0.14	0.48	−0.10
Improvements in education are largely driven by the latest report of the inspectorate.	−0.02	0.08	0.82
Improvements in education are largely driven by feedback from the inspectorate from the latest inspection.	0.02	−0.04	0.94

Note: Bold values refer to the items that are associated with the factor indicated at the top.

Table A2. Factor loadings for items related to educational quality outcomes.

Item	Factor 1 – Ambition to improve quality	Factor 2 – Urgency to improve quality	Factor 3 – Investment in quality
This school is characterized by an ambition to continuously improve. (ambitions towards performance)	0.65	0.04	0.07
This school is characterized by an ambition to realize high performance for all students. (ambitions towards performance)	0.57	−0.05	0.09
This school is characterized by an ambition to work as a team. (ambitions towards performance)	0.84	0.04	−0.04
This school is characterized by an ambition to stimulate personal development of students. (ambitions towards broad development)	0.83	−0.04	−0.03
This school is characterized by an ambition to educate students to become responsible citizens. (ambitions towards broad development)	0.77	−0.01	−0.02
This school is characterized by an ambition to develop students in a broad sense. (ambitions towards broad development)	0.56	−0.03	0.01
This school is characterized by an ambition to offer opportunities to students outside the regular programme. (ambitions towards broad development)	0.68	−0.02	0.09
In this school, an urgency exists to improve student performance. (urgency to improve school climate)	−0.01	0.61	0.05
In this school, an urgency exists to improve the pedagogical and didactic quality of teachers. (urgency to improve school climate)	−0.06	0.70	0.03
In this school, an urgency exists to improve the safety of the school climate. (urgency to improve school climate)	0.01	0.64	0.02
In this school, an urgency exists to improve the way in which teachers collaborate. (urgency to improve school climate)	−0.08	0.68	0.07
In this school, an urgency exists to improve the culture of quality assurance. (urgency to improve quality assurance)	0.04	0.87	−0.02
In this school, an urgency exists to improve the system of quality assurance. (urgency to improve quality assurance)	0.03	0.89	−0.04
In this school, an urgency exists to improve the reports and documents about educational quality. (urgency to improve quality assurance)	−0.04	0.68	0.02
In this school, we are currently working to improve student performance. (investments to improve school climate)	0.20	0.05	0.59
In this school, we are currently working to improve the pedagogical and didactic quality of teachers. (investments to improve school climate)	0.08	0.04	0.66
In this school, we are currently working to improve the safety of the school climate. (investments to improve school climate)	0.10	0.09	0.57
In this school, we are currently working to improve the way in which teachers collaborate. (investments to improve school climate)	0.05	0.02	0.69
In this school, we are currently working to improve the culture of quality assurance. (investment to improve quality assurance)	−0.06	−0.02	0.91
In this school, we are currently working to improve the system of quality assurance. (investment to improve quality assurance)	−0.06	−0.02	0.92
In this school, we are currently working to improve the reports and documents about educational quality. (investment to improve quality assurance)	0.03	0.03	0.64

Note: Bold values refer to the items that are associated with the factor indicated at the top.

Table A3. Factor loadings for items related to school culture outcomes.

Item	Factor 1 – Culture of continuous improvement	Factor 2 – Openness	Factor 3 – Pride	Factor 4 – Professionalization
This school is characterized by regularly reflecting on how to improve education.	0.63	0.16	0.07	0.02
This school is characterized by a systematic way of evaluating performance (through test score analysis, student surveys).	0.86	−0.04	−0.01	0.00
In this school, teachers discuss the quality of each other’s classes.	0.58	0.27	0.04	0.25
In this school, the school principal gives teachers feedback on the quality of their classes.	0.52	0.12	0.22	0.10
In this school, mistakes are accepted.	0.13	0.50	0.36	0.23
In this school, teachers are watching each other’s classes.	−0.03	0.87	0.00	−0.03
In this school, teachers are involved in each other’s work.	0.17	0.56	0.04	0.15
The team is proud of this school.	−0.03	−0.01	1.00	−0.01
The team is proud of the way students develop in this school.	0.23	0.02	0.49	0.04
Teachers and the school principal possess up-to-date knowledge about “learning” and didactic models.	0.20	0.10	0.11	0.41
Teachers are committed towards self-development.	0.01	−0.04	−0.02	0.92
Teachers regularly try out new ways of instruction.	0.08	0.18	0.05	0.58

Note: Bold values refer to the items that are associated with the factor indicated at the top.