The sustainability of a teacher professional development programme for beginning urban teachers

Abstract

This study investigated the long-term effects of a professional development intervention for beginning urban teachers and explored which characteristics and activities in school organisations contributed to the sustainability of these effects. A quasi-experimental study (n = 72) investigated whether the positive effects of the programme were observed one year after the programme ended. Interviews with teachers and principals (n = 19) were used to explore which characteristics and activities in schools contributed to those positive effects. A significant long-term effect of the programme was found on teachers’ competences and professional orientation. An open culture in the schools was experienced as an important factor for the sustainability of the programme’s effects.

Keywords:

• Teacher retention
• teacher training
• teacher professional development
• professional learning community
• organisational transfer climate

Introduction

Professional development programmes can contribute to the competences and job motivation of teachers (Borko, 2004; Gilles, Davis, & McGlamery, 2009); however, it is not self-evident that the effects of professional development programmes will be maintained after their completion. Research has shown that the effectiveness of professional development programmes is dependent on characteristics of the programmes, such as ‘collectivity’ and ‘connection of the course content with classroom practice’. However, sustainability of the programmes’ effects is also, and perhaps mainly, related to characteristics of the school organisation (Snoek, Enthoven, Kessels, & Volman, 2015; Van Veen, Zwart, & Meirink, 2012).

According to several studies, the creation of a professional learning community (PLC) in schools, in which the focus is on teacher learning and collaboration, is a promising way to promote the continuous professional development of teachers (Little, 2006). It is also possible that PLCs contribute to the long-term effects of professional development programmes.

However, little is known about the characteristics of and activities in PLCs that contribute to the sustainability of professional development interventions. The present study investigated whether the positive effects of a professional development programme for beginning urban teachers (‘Mastery’) were observed one year after the programme ended, and which PLC characteristics and activities in the programme participants’ schools contributed to those positive effects.

Professional development programmes for teachers

Many studies have focused on the effects of professional development interventions for teachers (see for instance Borko, 2004; Morge, Toczek, & Chakroun, 2010). Several studies showed that professional development interventions positively affected the quality of teachers in terms of their knowledge and competences for teaching. For instance, professional development programmes can help teachers develop rich and flexible knowledge regarding their subjects (Borko, 2004). Vogt and Rogalla (2009) observed that a professional development intervention which was based on content-focused coaching contributed positively to adaptive teaching competency.

Another important criterion for teacher quality is teacher professional orientation (Mahieu, Dietvorst, & Peene, 1999). Teachers are expected not only to perform well within the classroom but also to demonstrate professionalism that extends beyond the classroom. Hoyle (1980) distinguished ‘restricted professionality’, in which teachers focus primarily on their own classroom and base their actions on experience rather than theory, from ‘extended professionality’, in which teachers are involved in the school organisation and have an interest beyond the classroom. In the study of Gaikhorst, Beishuizen, Zijlstra, & Volman (2015), teachers perceived a positive impact of a professional development programme for beginning urban teachers on teachers’ professional orientation. Teachers who participated in the programme indicated for instance that they were more involved in the process of school development, more interested in theory and educational development and collaborated more with colleagues after participation in the programme than before.

Furthermore, an important factor contributing to teacher quality is self-efficacy. Self-efficacy is defined as ‘the teacher’s belief in her or his ability to organise and execute the course of actions required to successfully accomplish a specific task in a particular context’ (Tschannen-Moran, Woolfolk Hoy, & Hoy, 1998, p. 233). Hofman & Dijkstra found that teacher professionalisation in a teacher network had a positive effect on the self-efficacy of teachers (Hofman & Dijkstra, 2010).

Professional development programmes can also contribute to the retention of teachers (Gilles et al., 2009; Hofman & Dijkstra, 2010). Research indicates that teachers with low self-efficacy are more likely to experience burnout than teachers with high self-efficacy (Evers, Brouwers, & Tomic, 2002). Such teachers may be at risk of leaving the teaching profession. Professional development programmes can contribute to teachers’ self-efficacy, which in turn could lead to a reduced chance that teachers leave the teaching profession because of burnout. Gilles et al. (2009) observed positive effects of a professional development intervention for beginning urban teachers on the job motivation and retention of these teachers.

Although several studies have shown positive effects of professional development programmes on the quality and retention of teachers, it also appeared that it is not self-evident that the effects of professional development programmes will be maintained after completion of the programmes. Some authors claim that interventions should be permanent to become and remain effective (Yoon, Duncan, Lee, Scarloss, & Shapley, 2007). For instance, this could be realised by follow-up interventions and encouraging the ongoing learning and collaboration of teachers.

Although research has shown that embedding professional development interventions in the school organisation is important for the success and sustainability of the programmes (Snoek et al., 2015), this aspect is often neglected because many studies focus mainly on the effects of professional development interventions as such. Therefore, it is important to investigate how professional development activities can be embedded in the school organisation to maintain and/or enhance the effects of the interventions in the long-term.

School organisation and professional development

In school organisational literature and workplace learning and learning in organisations literature, school organisational activities are described that might contribute to the continuous professional development of teachers and the lasting effects of professional development interventions.

Literature on learning in organisations shows that it is important that school organisations facilitate the transfer of learning outcomes from professional development interventions to the workplace (Hatala & Fleming, 2007). Blume, Ford, Baldwin, and Huang (2010) define transfer as a dynamic and complex process, which consists of two major dimensions: (a) generalisation – the extent to which knowledge and skills acquired in a learning setting are applied to other settings, individuals and/or situations, and (b) maintenance – the extent to which changes from learning experiences persist over time. To successfully transfer learning outcomes from professional development programmes to the workplace, it is important that teachers have opportunities to practise newly gained skills in schools, that teacher learning is appreciated, for instance, by rewards and incentives (Baldwin & Ford, 1988) and that teachers receive sufficient support and feedback (Baldwin & Ford, 1988). Supervisors and colleagues play a crucial role in the transfer process by providing support and feedback and showing their involvement (Baldwin & Ford, 1988). According to Burke and Hutchins (2007), the alignment between the policy of the organisation and goals of the professional development programmes is important for the transfer of the programme to the workplace. The transfer process also appears to be influenced by individual attributes of the participants and the curriculum of the professional development programmes (Baldwin & Ford, 1988; Burke & Hutchins, 2007).

The notion of transfer has been criticised in the literature as being too simplistic, based on replicative conceptions of learning, and as being disconnected from contextual factors that influence the transition between learning and work contexts (Hager & Hodkinson, 2009). We adopt an approach to transfer, however, which acknowledges that the transition between learning context and work context is dynamic and complex and that learned knowledge is always adapted to fit in the work setting, thereby changing that setting as well (Tuomi-Gröhn & Engeström, 2003).

Furthermore, it appears that it is important that school organisations create a culture for teacher learning for successful professional development (Assunção Flores, 2004; Van Veen et al., 2012). A culture for learning refers to an open organisational climate, in which the employees collaborate and trust one another, feel safe to make mistakes and where open communication is the norm (Tschannen-Moran & Hoy, 2000; Van Woerkom, 2004). Literature on learning in the workplace has shown that it is important that organisations provide sufficient support and feedback, opportunities to learn and access to learning resources (Ashton, 2004). Additionally, involved colleagues who collaborate and support one another appeared to be important (Assunção Flores, 2004; Meirink, Imants, Meijer, & Verloop, 2010; Hobson & Ashby, 2012). The research of Geijsel, Sleegers, Stoel, and Krüger (2009) showed that school leaders play an important role: school leaders who provided teachers with the security required to experiment, make mistakes and exchange expertise and experiences appeared to positively affect professional development.

Several activities in school organisations that could foster the transfer of professional development interventions to the workplace and promote the ongoing professional development of teachers, such as collaboration between teachers and opportunities for support and feedback, are typical for what has recently been referred to as professional learning communities (PLCs) (Little, 2006).

Although there are variations in how researchers define professional communities, most definitions encompass practices that are supportive for teacher learning, including observation, problem solving, mutual support and advice (Grodsky & Gamoran, 2003; Little, 2006). PLCs refer to close relationships between teachers, typically with the implication that these relationships are oriented toward teacher professional development (Little, 2006). Strong PLCs are characterised by an overall vision, in which teacher learning is considered relevant by both school leaders and teachers (Little, 2006). There is a collective focus on and shared responsibility for student learning, collective control over important decisions and collaboration between teachers (Grodsky & Gamoran, 2003; Little, 2006; Stoll, Bolam, McMahon, Wallace, & Thomas, 2006). Teachers in strong PLCs have access to new knowledge about teaching and learning and to the expertise of colleagues from in- and outside their school. Teachers provide one another with feedback on individual performance and aspects of classroom and school practice (Little, 2006). Preconditions for the creation of strong PLCs are sufficient time, materials and space and access to the expertise of colleagues (Little, 2006). Furthermore, school leaders play an important role, which is to cultivate PLCs (Stoll et al., 2006).

Several studies have shown positive relationships between PLCs in schools and the ongoing/continuous professional development of teachers (Little, 2006; Wilson & Berne, 1999). There are also indications that the success of professional development programmes is dependent on the quality of PLCs in schools (Little, 2006). Simultaneously, there are indications that when a school supports teachers’ participation in high-quality professional development programmes, PLCs are strengthened (Little, 2006).

Although many studies have stressed the importance of PLCs in schools for the (ongoing) professional development of teachers and success of professional development programmes, it is not known which activities and characteristics contribute to the long-term effects of professional development programmes.

Contextual background

In this study, the sustainability of the effects of a professional development programme (which was called the ‘Mastery’ programme) for beginning teachers in an urban educational context was investigated. Research has shown that teaching in an urban context is, for a variety of reasons, such as having to deal with cultural diversity, unsafe atmospheres, and language differences, challenging and difficult for teachers (Groulx, 2001; Kooy, 2006; Smith & Smith, 2006).

Teachers are often not well prepared for the challenges of urban teaching (Çelik & Amaç, 2012), which can lead to an outflow of teachers from urban schools and education in general. Because teaching in urban areas is challenging, it would be expected for beginning teachers to be provided with additional support in the beginning phase of their career, for instance in the form of a professional development programme that focuses on urban teaching. In this study, the sustainability of the effects of the ‘Mastery’ programme was investigated. The long-term effects were investigated one year after the teachers had completed the ‘Mastery’ programme. The ‘Mastery’ programme aimed to support teachers in the first phase of their teaching in urban primary schools.

Characteristics of the ‘Mastery’ programme

The ‘Mastery’ programme was a professional development programme for beginning primary school teachers working in Amsterdam (the capital of the Netherlands). The programme was called ‘Mastery’ (in Dutch ‘Meesterschap’) for two reasons. On the one hand, the name refers to the Dutch word ‘Meester’ (which is ‘Teacher’ in English). On the other hand, the name refers to ‘mastery’ in the sense of ‘competent for urban teaching’.

The programme, which was developed collaboratively by the teacher education institutes in Amsterdam, had a twofold purpose: to increase the quality of teaching and contribute to the retention of beginning teachers in the urban educational context.

The programme focused on the core competences required for urban teaching, including dealing with cultural diversity and language differences, cooperating within the school environment and ensuring safety. The intention was that participants would increase their expertise in these four areas as a result of their participation in the programme. The programme consisted of four modules: ‘School and environment’, ‘Safety’, ‘Language’ and ‘Cultural diversity’.

The content of the programme was focused on the acquisition of skills necessary to meet the challenges of teaching in a complex urban environment, such as collaborating with professionals both in- and outside the school environment, addressing aggressive behaviour and language deficiencies of children, communicating with parents of different cultural backgrounds - and on developing an extended professional orientation.

The programme lasted one year and consisted of the following three components: group meetings (these involved theoretical input from experts, opportunities for sharing experiences and group assignments), classroom application (participants apply new insights to their teaching practices) and lectures (in which experts explored urban themes). Additionally, supervision was organised, offering a context for beginning teachers to share experiences and expertise.

The participants were divided into groups of approximately 15 members and the groups met once every two weeks at a teacher training institute. The groups were accompanied within each theme by experts (i.e. professor, lecturer or teacher educator) in the field concerned. All participants were required to invest an average of four hours every two weeks over a period of 12 months (for a more extensive discussion about ‘Mastery’, see Gaikhorst, Beishuizen, Zijlstra, & Volman, 2015).

Previous research has shown positive effects of this programme on the competences and self-efficacy of teachers (Gaikhorst, Beishuizen, Zijlstra, & Volman, 2015). The results of this study showed that the competences for urban teaching and self-efficacy of the teachers who followed the ‘Mastery’ programme improved relatively more than those of the teachers who did not follow the ‘Mastery’ programme.

The present study: purpose and research questions

This study examined the long-term effects of the ‘Mastery’ programme on the quality and retention of beginning urban teachers. The effects of the programme were investigated one year after the teachers completed the programme. Another focus of study was which characteristics and activities were typical of school organisations where teachers showed positive long-term effects of the ‘Mastery’ programme, with a special focus on characteristics and activities that are considered typical of professional learning communities.

The research questions of this study were formulated as follows:

1.	What are the long-term effects of participation in the ‘Mastery’ programme on the quality (competences, professional orientation and self-efficacy) and retention (job motivation and career choices) of teachers?
2.	Which school characteristics and activities in the schools contributed to long-term effects of the ‘Mastery’ programme?

Method

Research design

The first research question was investigated using a quasi-experimental design with an experimental and control group. The long-term effects of participation in the ‘Mastery’ programme on the quality (competences, professional orientation, self-efficacy) and retention (job motivation and career choices) of teachers were measured using a knowledge test and questionnaires. Pre-measures (administered before the beginning of the programme), post-measures (conducted directly after completing the programme) and retention measures (administered one year after completion) were used.

The second research question was investigated through interviews. This qualitative method was chosen because we wanted to obtain a detailed understanding of the school characteristics and activities that were meaningful for the teachers such that the activities helped the teachers maintain and/or enhance the long-term effects of the ‘Mastery’ programme.

Participants

For the quantitative component of the present study, all 133 teachers who participated in the post-measurement (see Gaikhorst, Beishuizen, Zijlstra, & Volman, 2015) were approached and asked whether they would like to participate in the retention measurement. All participants were teachers who taught at a primary school in the capital of the Netherlands (Amsterdam). These teachers joined in the study voluntarily. The experimental group comprised teachers who did participate in the ‘Mastery’ programme, whereas the control group teachers did not. Teachers from the control group participated in professional development programmes that were not focused on urban themes. These programmes addressed topics such as ‘mathematics’ and ‘ICT’.

In total, 44 of the 67 teachers from the control group and 28 of the 66 teachers from the experimental group completed the retention measurement. Whether the group of teachers who completed the retention measurement was comparable to the group who did not complete the measurement was determined for both the experimental and control groups by comparing the post-measurement scores. There were no significant differences between the teachers who completed the retention measurement and those who did not (p-values were all > 0.05).

To derive obvious conclusions regarding the effects of the intervention, a matching procedure was developed whereby the participants of both conditions were matched on several potentially interfering variables. The conditions appeared to be comparable for all characteristics (see Table 1).

Table 1. Comparison of the conditions in terms of the general characteristics of the teachers.

	‘Mastery’ condition			Control condition
	M	SD	n	M	SD	n
Teaching experience^a	6.37	1.81	27	6.86	1.74	44
Gender^b	0.93	0.26	28	0.98	0.12	44
School population (SES)^c	2.23	1.24	26	2.50	1.15	44
School population (ethnic background of parents)^d	2.54	1.27	26	2.66	1.22	44
Highest achieved level of education^e	2.29	1.05	28	2.66	1.14	44
Place of teacher training^f	0.30	0.47	28	0.30	0.46	44

^a 1 = 0.5 year, 1 = 1 year, 2 = 2 years, 3 = 3 years, 4 = 4 years, 6 = 5 years, 6 = 6 years, 7 = 7 years.

^b 0 = male, 1 = female.

^c 1 = 0–30% pupils with lower SES, 2 = 30–50% pupils with lower SES, 3 = 50–70% pupils with lower SES, 4 = 80–100% pupils with lower SES.

^d 1 = 0–30% non-Dutch pupils, 2 = 30–50% non-Dutch pupils, 3 = 50–70% non-Dutch pupils, 4 = 80–100% non-Dutch pupils.

^e 1  =  lower secondary education, 2  =  higher secondary education, 3 = pre-university education, 4 = other.

^f 0  =  Inside a large city, 1  =  outside a large city.

For the qualitative study, only those teachers who showed positive long-term effects of the ‘Mastery’ programme on teachers’ competences and self-efficacy were selected. In total, 10 of the 15 teachers who showed positive long-term effects of ‘Mastery’ wanted to participate in the study. These 10 teachers and nine of their school principals were interviewed. In total, nine schools were included in the qualitative study. All participants joined in the study voluntarily. Before the start of the interviews, the teachers and principals were informed about the interview procedure, the anonymous processing of the data and the opportunity to discontinue their involvement in the interviews at any time.

Instruments

A questionnaire and test were used to measure the different dependent variables. The questionnaires were administered before, immediately after and one year after completion of the programme (pre-, post- and retention test). The questionnaires that were used for the retention test were identical to those administered for the pre- and post-test. The period between the measurements was identical for both the experimental and control groups (namely, one year).

Competences for teaching in an urban environment

Competences for teaching in an urban environment were operationalised as knowledge of methods of coping with language differences, threats to safety, cultural diversity and various actors in the school. To measure these competences (in terms of knowledge), a knowledge test was developed by the educators of the programme in collaboration with the researchers. We realise that knowledge is a poor operationalisation of competence, but in the context of the programme this was the most feasible method of measurement. The knowledge test consisted of 54 multiple-choice questions that addressed four urban themes. The teachers were asked to choose the best answer from four alternatives. The variable ‘knowledge score’ was calculated by computing the amount of correct answers. Cronbach’s alpha was 0.70 for the pre-test, 0.71 for the post-test, and 0.70 for the retention test. The following is an example of a question:

‘The three pillars of language-oriented vocational education are as follows:

(1)	ask questions, generate answers and give instruction;
(2)	provide context-rich work, provide interactive work and provide language support;
(3)	give feedback, provide structure and use understandable language;
(4)	explicate concepts, explain thought processes and formulate course objectives.’

(The right answer is 2)

Professional orientation

Professional orientation was measured by the questionnaire that was developed by Jongmans, Biemans, and Beijaard (1998), based on Hoyle’s (1980) characterisation of teachers with a restricted or an extended professional orientation. The questionnaire consists of 13 items. The teachers were asked to indicate to what extent they agreed with each of the 13 statements using a five-point scale that ranged from ‘totally disagree’ to ‘totally agree’. Negative responses were interpreted to indicate a restricted orientation, whereas positive responses were interpreted to indicate an extended professional orientation. The variable ‘professional orientation’ was calculated by obtaining the mean of the 13 items. Cronbach’s alpha was 0.76 for the pre-test, 0.80 for the post-test, and 0.76 for the retention test.

Teacher self-efficacy

Teacher self-efficacy was measured using the ‘Attitude towards the teaching profession’ questionnaire (Meijer & van Eck, 2008). This questionnaire was translated and adapted from Kyriacou and Kunc (2007). The scale consisted of nine items, and the teachers were asked to indicate the extent to which they agreed with each of the statements using a five-point scale that ranged from ‘totally disagree’ to ‘totally agree’. Negative responses were interpreted to indicate low self-efficacy, and positive responses were interpreted to indicate high self-efficacy. The variable ‘self-efficacy’ was calculated using the means of the nine items. Cronbach’s alphas were 0.86, 0.88, 0.87 for the pre-, post- and retention tests, respectively.

Motivation for teaching

The job motivation of the teachers was measured using the ‘Vision of teaching and job satisfaction’ questionnaire (Meijer & van Eck, 2008). The teachers were asked to indicate the extent to which they agreed with each of the 10 statements using a five-point scale that ranged from totally disagree to totally agree. Negative responses were interpreted to indicate low motivation (for teaching), and positive responses were interpreted to indicate high motivation. The variable ‘motivation’ was calculated using the means of the 10 items. Cronbach’s alpha was 0.71 for the pre-test, 0.78 for the post-test and 0.78 for the retention test.

Career choices

The career choices of the teachers were measured using several questions regarding actual and planned career choices. The questions addressed the intention of the teacher to remain in education (in this profession, at this school, in an urban environment), the steps taken to orient into other sectors and the actual actions taken to leave the education profession.

Characteristics and activities that contributed to the sustainability of the ‘Mastery’ programme

For a detailed understanding of the school characteristics and activities that contributed to the sustainability of ‘Mastery’, semi-structured telephone interviews were conducted with 10 programme participants, who reported positive long-term effects, and nine principals of these participants.

The interviews consisted of two sections. In the first section, the interviewer asked about the perceived influence of the programme on the dependent variables of the quantitative study: competences, professional orientation, self-efficacy, job motivation and career choices. The second section of the interviews focused on the characteristics and activities considered typical of a PLC that were performed in the schools where teachers reported positive long-term effects of the programme. The respondents were first asked to spontaneously report on these school activities and characteristics. Thereafter, they were asked to respond to a number of school characteristics and activities that are considered important for the (ongoing) success of professional development in the literature (see Table 2). They were invited to elaborate on how these characteristics contributed to the sustainability of the programme’s effects.

Table 2. Checklist of school characteristics and activities that were used in the second section of the interviews and times that they were recognised by teachers and principals.

School organisational characteristics and activities from the literature	Teachers (n = 10)	Principals (n = 9)
Opportunities in the school to share newly gained expertise from the professional development programme with colleagues	10	8
Appreciation in the school for teachers’ input from the professional development programme	10	7
Opportunities in the school to practise newly gained expertise from the professional development programme	8	9
Involved colleagues in the school who show their interest in the professional development programme	7	5
Alignment between the policy of the school and goals of the professional development programme	5	7
Support and feedback from colleagues in the school regarding the professional development programme	4	5
A culture for teacher learning in the school:
• Teachers have access to new knowledge about teaching and learning	9	9
• Teachers collaborate with one another in the school	8	7
• Teacher learning is considered to be relevant by both school principals and teachers	7	7
• There is collective control over important decisions	6	9
• There is a safe atmosphere in the school (in which teachers feel free to make mistakes)	6	6
• There is a collective focus on and shared responsibility for student learning	6	7
• The school organisation is focused on teacher learning	5	6
• Teachers learn from one another in the school	4	6

Data analysis

Quantitative data

Multilevel modelling was used to determine the effects of the ‘Mastery’ programme on the different dependent variables. Measurement occasions (level 1) were treated as nested in teachers (level 2). The independent variables in the analyses were condition (experimental or control condition) and measurement occasion, whereas the dependent variables were competences, professional orientation, self-efficacy, job motivation, and career choices. The assumptions for multilevel modelling were checked, and no violations were observed.

The effect sizes were calculated using the formula of Raudenbush and Liu (in Feingold, 2009, p. 7), whereby we divided the coefficient for the interaction effect between the retention test and experimental condition by the pooled standard deviation of the dependent variable on the two measurement occasions (namely, pre- and retention test).

Qualitative data

A content analysis was employed to analyse the data from the interviews (Miles & Huberman, 1994). The responses to the interview questions were examined and coded by the first author. The codes referred to the perceived effects of the ‘Mastery’ programme on the dependent variables (for example, an increase in extended professional orientation) and the PLC activities and elements that were present in the school organisations where teachers reported positive effects of the programme (for example, the opportunity for sharing experiences with colleagues). It was difficult to determine inter-rater reliability because of the interpretative and iterative nature of the data analysis (Akkerman, Admiraal, Brekelmans, & Oost, 2008). To enhance the trustworthiness of the analysis, the following procedures were followed:

1.

All fragments that were difficult to code, in the perception of the coder (first author), were discussed with another experienced researcher (second author). These fragments and codes were discussed until a consensus was reached and the coding was adjusted to the outcome of this discussion.

2.

The outcome of the interpretation of the meaning was audited by a procedure whereby the codes of two (randomly chosen) scored interviews (10%) were checked and discussed in a peer review (Kvale, 2007). The codes from the coder (first author) were verified by an individual who did not participate in the study. Coding for ‘competences’, ‘self-efficacy’, ‘job motivation’, ‘career choices’ and ‘PLC activities and elements’ demonstrated a 100% concurrence rate. However, coding ‘professional orientation’ was less uniform, with a 91% concurrence rate. The codes that were less uniform were discussed until agreement was reached, and the coding was adjusted to the outcome of the discussion.

Results

Descriptive statistics of the main variables

The descriptive statistics of the dependent variables included in the study are presented in Table 3.

Table 3. The descriptive statistics for the dependent variables.

Condition	Pre-test				Post-test				Retention test
	M	SD	Min	Max	M	SD	Min	Max	M	SD	Min	Max
‘Mastery’ condition:
Knowledge	27.31	5.39	14.00	39.00	34.56	5.19	21.00	44.00	32.29	5.34	23.00	47.00
Professional orientation	4.19	0.33	3.54	4.92	4.19	0.40	2.85	5.00	4.39	0.30	3.77	4.85
Self-efficacy	3.81	0.37	2.67	4.89	3.92	0.42	2.89	4.89	3.98	0.41	3.11	4.78
Job motivation	4.10	0.36	3.44	4.78	3.83	0.33	2.89	4.44	4.12	0.42	3.44	4.78
Career choices	3.41	0.96	2.00	5.00	3.13	0.94	2.00	5.00	3.07	1.02	1.00	5.00
Control condition:
Knowledge	28.85	6.65	15.00	42.00	29.30	5.76	18.00	43.00	28.56	6.04	15.00	44.00
Professional orientation	4.37	0.34	3.54	4.92	4.42	0.34	3.69	4.92	4.34	0.38	3.54	5.00
Self-efficacy	3.98	0.53	2.44	2.89	3.90	0.54	2.56	5.00	4.02	0.56	2.89	4.89
Job motivation	4.06	0.45	2.89	4.89	3.67	0.49	2.67	4.67	3.93	0.51	2.78	4.89
Career choices	3.06	1.01	1.00	5.00	2.84	1.00	1.00	5.00	2.77	1.10	1.00	5.00

Evaluation of the long-term effects of the ‘Mastery’ programme

Teachers’ competences

The outcomes of the multilevel analysis are presented in Table 4. There was a significant interaction effect between the retention test and experimental condition, which indicated that, given the initial difference between the conditions in the pre-test (in which the experimental condition scored lower than the control condition, see Table 3), teachers’ knowledge increased relatively more in the ‘Mastery’ condition from the pre-test to the retention measurement. The effect size was 0.72, which can be considered large (Cohen, 1992).

Table 4. Parameter estimates for the multilevel models of teachers’ competences predicted by measurement occasion and condition.

	Knowledge
	Coeff.	SE	p
Measurement occasion 1 (pre-test)	28.74	0.85	0.00
Measurement occasion 2 (post-test)	29.25	0.81	0.00
Measurement occasion 3 (retention test)	28.75	0.92	0.00
Condition (experimental)	−1.44	1.16	0.22
Post-test ^* experimental condition	6.86	0.99	0.00
Retention test ^* experimental condition	4.13	1.30	0.00

^* = ‘in combination with’

The results of the qualitative analysis confirmed the conclusion of the quantitative analysis. The majority of the respondents indicated that the teachers developed several competences for urban teaching, such as dealing with parents (both highly educated parents and parents with culturally diverse backgrounds) and language deficiencies.

Teachers’ professional orientation

The interaction effect between the experimental condition and retention test was significant (Table 5). This result indicates that, given the initial difference between the conditions on the pre-test (in which the experimental groups scored lower on the professional orientation scale than the control group, see Table 3), teachers’ professional orientation increased relatively more in the ‘Mastery’ condition from the pre-test to the retention measurement. The effect size appeared to be 0.58, which can be considered medium (Cohen, 1992).

Table 5. Parameter estimates for the multilevel models of teachers’ professional orientation predicted by measurement occasion and condition.

	Professional orientation
	Coeff.	SE	p
Measurement occasion 1 (pre-test)	4.37	0.04	0.00
Measurement occasion 2 (post-test)	4.41	0.05	0.00
Measurement occasion 3 (retention test)	4.32	0.05	0.00
Condition (experimental)	−0.18	0.06	0.00
Post-test ^* experimental condition	−0.04	0.06	0.57
Retention test ^* experimental condition	0.20	0.07	0.01

^* = ‘in combination with’

The results of the qualitative analysis confirmed the conclusion of the quantitative analysis. In the opinion of the respondents, the ‘Mastery’ programme contributed to the development of a broader view on teaching. Several respondents mentioned that the teachers were more interested and involved in the process of school development (for instance, they became members of project groups) after participation in the programme.

One teacher declared the following:

I focus more on things outside my classroom. I also became more interested in teaching in Amsterdam, in what happens on other schools. I also joined the participation council, and by following ‘Mastery’, I became more interested in what we can change at the policy level.

Teachers’ self-efficacy

The interaction effect between condition and the retention test was not significant (Table 6), which indicated that teachers’ self-efficacy did not increase more in the ‘Mastery’ condition from the pre- to retention measurement.

Table 6. Parameter estimates for the multilevel models of teachers’ self-efficacy predicted by measurement occasion and condition.

	Self-efficacy
	Coeff.	SE	p
Measurement occasion 1 (pre-test)	3.98	0.06	0.00
Measurement occasion 2 (post-test)	3.89	0.06	0.00
Measurement occasion 3 (retention test)	4.02	0.07	0.00
Condition (experimental)	−0.17	0.08	0.04
Post-test ^* experimental condition	0.20	0.07	0.00
Retention test ^* experimental condition	0.09	0.09	0.35

^* = 'in combination with'

However, the results of the interviews showed that, in the opinion of the respondents, the ‘Mastery’ programme affected the self-efficacy of the teachers positively. For instance, the teachers felt more confident in their contact and communication with parents and colleagues and in providing language education.

One teacher stated the following:

I feel more confident. I can make more easily contact with parents … that was something that we have learned … how you can handle that. Additionally, that you can more easily give advice to parents, that you know better how the process of language learning works with second language learners.

Teachers’ job motivation

The interaction effect between the experimental condition and retention test was not significant (Table 7). This result indicated that teachers’ job motivation did not increase more in the ‘Mastery’ condition than in the non-‘Mastery’ condition from the pre-test to the retention measurement.

Table 7. Parameter estimates for the multilevel models of teachers’ job motivations predicted by measurement occasion and condition.

	Job motivation
	Coeff.	SE	p
Measurement occasion 1 (pre-test)	4.06	0.05	0.00
Measurement occasion 2 (post-test)	3.67	0.05	0.00
Measurement occasion 3 (retention test)	3.95	0.06	0.00
Condition (experimental)	0.04	0.07	0.53
Post-test ^* experimental condition	0.11	0.07	0.14
Retention test ^* experimental condition	0.07	0.09	0.42

^* = ‘in combination with’

Although the quantitative results showed no influence of the programme on the job motivation of the teachers, several respondents to the interviews mentioned that the programme affected the motivation of the teachers positively. These respondents explained that the programme provided teachers with (new) inspiration or energy for teaching and/or insights into the attractive aspects of the education profession.

One teacher stated:

Yes, I have to say that it is a very hard job, but I experienced that … the programme gave me new energy … it gave me the energy to do things better, I became motivated through that.

Teachers’ career choices

There were no significant interaction effects regarding teachers’ career choices. The p-values of the interaction effects of the different items were all > 0.05, thus indicating that the teachers’ career choices did not increase more in the ‘Mastery’ condition than in the non-’Mastery’ condition from the pre-test to the retention measurement.

However, several respondents to the interviews experienced a positive contribution of the programme to teachers’ career choices. According to these respondents, the programme stimulated the teachers to think more about their professional development and encouraged them to develop themselves in a certain direction (for instance, using follow-up courses).

School characteristics and activities

The majority of the teachers and principals mentioned that it was possible in their schools to develop and apply what teachers had learned in the ‘Mastery’ programme in their schools. These teachers and principals underlined the importance of opportunities to practise the newly gained expertise from the programme in their schools. This was organised in the schools through new tasks and roles for the teachers.

Furthermore, the majority of the respondents mentioned that in their schools it was possible to share the expertise from the ‘Mastery’ programme. All teachers and nearly all principals emphasised the value of opportunities to share newly gained expertise from the programme with colleagues. At some schools, sharing was mainly informal, such as during lunch or coffee breaks, at other schools, sharing also occurred during formal occasions, such as group meetings and seminars.

The involvement of principals in the ‘Mastery’ programme was also mentioned by several respondents. According to these respondents, transferring expertise from the ‘Mastery’ programme to the workplace could be accomplished if the principals were involved in the programme. Four teachers and five principals underlined the importance of ‘support and feedback from colleagues regarding the programme’. These respondents indicated, for instance, that colleagues and principals helped teachers perform the assignments from the ‘Mastery’ programme in the school. Through their involvement in the assignments, principals became aware of the themes that were discussed in the “Mastery’ programme and obtained insight into the kinds of expertise that the teachers had developed. The principals considered the teachers as ‘experts’ on these themes and provided the teachers with opportunities to further develop their expertise after completing the programme. However, although the schools were the ‘good’ examples, many teachers did not experience sufficient support and feedback at their schools. These respondents indicated, for instance, that the expertise from the ‘Mastery’ programme could have been further developed if the school principals were to have adopted a more active and stimulating role.

Furthermore, the respondents considered the involvement of teachers and their newly acquired expertise in important school organisational developments as important for the further development of the acquired expertise from the ‘Mastery’ programme. With the involvement of teachers in important school organisational development, teachers felt that their participation in the programme was not only something that they had performed for themselves but also had value for the school organisation. Consistent with this result, five teachers and seven principals stressed the importance of alignment between the policy of the school and goals of the programme, which could be considered helpful for the involvement of teachers and their expertise in important school organisational developments.

The majority of the teachers and principals responded to the open question regarding the school activities and characteristics that an open culture, in which teachers could share their ideas and expertise from the ‘Mastery’ programme with colleagues, was most important for the development of the acquired expertise from the programme. The majority of the teachers and principals emphasised the value of involved colleagues in the school who showed interest in the programme. Involvement was mainly expressed in what several respondents called ‘small’ things, such as asking what the teachers had performed during meetings of the ‘Mastery’ programme or the presence of principals at the graduation ceremony.

Additionally, effective communication with colleagues was mentioned by several respondents. According to these respondents, it was important that teachers and school principals provide one another with information, advice and feedback in an effective manner..

All teachers and the majority of the principals underlined the importance of appreciation in their school for teachers’ input from the programme, in particular because teachers could substantiate their input from theory they acquired from the ‘Mastery’ programme. Appreciation for the teachers’ input was mainly expressed in compliments to the teachers and in seriously listening to the input of the teachers.

The majority of the principals emphasised the importance of all the different aspects of a ‘culture for teacher learning’ in their schools. The teachers also stressed the value of these aspects. The schools in this study were the ‘good examples’ in the sense that teachers in these schools showed positive long-term effects of the ‘Mastery’ programme. However, regarding two aspects of the culture for teacher learning, namely ‘the school organisation is focused on the learning of teachers’ and ‘teachers learn from each other’, fewer teachers were positive. Teachers who did not recognise the first aspect indicated that they had the opportunity to follow professional development programmes in their schools, but there was no plan for how the acquired expertise would be used in the school. The other aspect of the learning culture was recognised by only four teachers. These teachers mentioned that their schools had plans to allow teachers to learn from one another, but these plans were often not realised for practical reasons.

Furthermore, it was remarkable that nearly all principals recognised the aspect ‘There is collective control over important decisions’ in their schools, whereas fewer teachers experienced this collective control. Teachers who did not recognise this aspect mentioned that they had the opportunity to provide input regarding important decisions, but their input was not taken seriously by the school management because management makes the final decisions and these decisions are occasionally different from those of the teachers. However, many respondents mentioned that a culture for teacher learning was realised in their schools, which helped teachers develop, apply and share their expertise from the ‘Mastery’ programme.

Conclusion

Long-term effects of the ‘Mastery’ programme

The results of the quasi-experimental study showed that the ‘Mastery’ programme had a positive long-term effect on the competences and professional orientation of the teachers. A positive quantitative effect of the ‘Mastery’ programme on the professional orientation of the teachers was not observed in our previous study in which only a short-term measurement was conducted (Gaikhorst, Beishuizen, Zijlstra, & Volman, 2015). This result might indicate that the development of a professional orientation requires more time and can therefore only be measured after a longer period.

The quasi-experimental study showed no positive long-term effects of the programme on the self-efficacy of the teachers, which contrasted with the outcomes of our previous study (Gaikhorst, Beishuizen, Zijlstra, & Volman, 2015). Furthermore, the quantitative results showed that participation in the ‘Mastery’ programme had no effect on the teachers’ job motivation or career choices. The absence of an effect could be because the teachers who participated in the study were all motivated before they began the programme, as evidenced by the fact that the teachers had high scores on the motivation scale in the pre-test (ceiling effect). This high motivation was also the case with the teachers from the control condition who followed different professional development programmes from the ‘Mastery’ programme.

Although the quantitative results of this study showed no long-term effect of the programme on self-efficacy, job motivation and career choices of the teachers, the interviews revealed that, according to several participants in the interviews, the programme did contribute in the longer term to the self-efficacy, job motivation and career choices of the teachers.

School activities and characteristics

The results of the interview study showed that in schools where teachers reported positive long-term effects of the ‘Mastery’ programme, teachers experienced sufficient possibilities to apply, share and further develop the acquired expertise from the ‘Mastery’ programme within their schools. The school organisational characteristic that was considered most important for the sustainability of the programme’s effects was an open culture. Other school organisational characteristics and activities that were considered important by the respondents included the involvement of principals in the programme, the involvement of teachers and their acquired expertise in school organisation developments, and effective communication among team members.

The study showed that in schools where teachers reported positive long-term effects of the ‘Mastery’ programme, the majority of the school organisational conditions and activities that were identified in the literature as important for the (ongoing) success of teacher professional development were underlined as important by the teachers and principals. There were possibilities to practise and share expertise, and there was appreciation for teachers’ input from the programme and involved colleagues who showed their interest in the programme. The majority of the respondents believed that a culture for teacher learning was realised in their schools and, in their opinion, this culture helped the teachers to develop, apply and share their expertise from the ‘Mastery’ programme within the schools.

Discussion

This research aimed to contribute to our knowledge of teacher professionalisation. The study showed the long-term contribution of a professional development programme to the quality and retention of beginning urban teachers and identified several school characteristics and activities that were considered valuable for the transfer of learning and sustainability of the effects of the professional development intervention.

Although many studies have stressed the importance of PLCs in schools for the (continuous) professional development of teachers (e.g. Little, 2006; Stegall, 2011; Wilson & Berne, 1999), which activities and characteristics in PLCs contribute to the transfer and sustainability of professional development interventions has not previously been investigated. In this study an open culture, the involvement of principals in the programme, the involvement of teachers and their acquired expertise in school organisation developments, and effective communication among team members appeared to contribute to the long-term effects of the ‘Mastery’ programme.

The respondents considered an open learning culture, in which the teachers could share their expertise with colleagues and teachers and principals seriously considered their expertise, the most important. This outcome was consistent with the results of previous research, in which the value of an open learning culture for the transfer and success of (ongoing) teacher professional development was emphasised (see, for instance, Assunção Flores, 2004; Little, 2006; Tschannen-Moran & Hoy, 2000; Van Veen et al., 2012).

The study also emphasised the role of good leadership in the sustainability of professional development interventions. This result was consistent with the outcomes of the study of Snoek (2013), who concluded that in schools where the transfer of a professional development programme occurred successfully, the teachers felt that their expertise was acknowledged and used by their principals in the development of the school organisation.

Furthermore, this study showed how schools can contribute to the sustainability of the effects of professional development programmes. For instance, new tasks and roles for the teachers were arranged in the schools where long-term effects occurred, which was – according to the teachers and principals – important for the further development of the newly gained expertise from the ‘Mastery’ professional development programme.

Another important result of this study was that the long- and short-term effects of the ‘Mastery’ programme were different. Several studies on professional development interventions include only pre- and post-measurements and no long-term measurements (see for instance, Vogt & Rogalla, 2009). We observed a significant positive, long-term effect of the programme on the professional orientation of the teachers, whereas this was not found as a short-term effect. This might indicate that the development of a professional orientation requires more time and can therefore only be measured after a longer period. This emphasises the importance of not only focusing on short-term effects of professional development interventions, but also doing research on long-term effects. The effects of professional development interventions on variables that require more time to develop may not have been measured in other studies.

Another difference that we found between the short- and long-term measurements was that there were no positive long-term effects of the programme on the self-efficacy of the teachers, whereas we did find positive short-term effects (Gaikhorst, Beishuizen, Zijlstra, & Volman, 2015). One reason for the differences might be that in the longer term teachers developed a broader professional orientation, and participated in new roles and tasks in their schools, which could have made them more aware of aspects of the teacher role that they could still improve on.

Besides differences in the outcomes of the long- and short-term measurements, differences were also found between the results of the quantitative and qualitative studies. The quantitative study showed no long-term effect of the programme on self-efficacy, job motivation and career choices of the teachers, whereas the interviews revealed that, according to several participants in the interviews, the programme did contribute in the longer term to the self-efficacy, job motivation and career choices of the teachers. One reason for the differences between the quantitative and qualitative results could be that there was a ceiling effect in the quantitative analysis; both conditions had high scores on the pre- and retention test scales. Another reason for the differences may be that only teachers who showed positive long-term effects of the ‘Mastery’ programme were interviewed, whereas the questionnaires were also filled out by teachers who showed no positive long-term effects. Furthermore, the differences could be caused by the fact that in the interviews the teachers were asked to reflect on changes in their own functioning, which entails the risk that they overestimated these changes.

This study has some limitations, and further research on teacher professionalisation is necessary. First, the interview study was on a small scale, and the results cannot be generalised. The small-scale design made it possible to obtain in-depth information regarding the specific activities and conditions in schools where teachers reported positive long-term effects of the ‘Mastery’ programme and the perceived value of these activities and conditions. However, it would be worthwhile to determine whether the outcomes of this study could be verified by a larger (quantitative) study.

Another limitation of the study is that we focused on the activities and characteristics that were present in the teachers’ schools, and paid no explicit attention to individual teacher attributes that might have had an impact on the transfer process and sustainability of the ‘Mastery’ programme’s effects. Theory on transfer shows that besides workplace characteristics individual learner characteristics too, such as ability, personality and motivation to use the learned competences in daily work settings, have an impact on the transfer of learning (Burke & Hutchins, 2007). Therefore, it is important to focus in follow-up research not only on activities in and characteristics of the school setting, but also on the role of individual teachers in the transfer process.

Furthermore, we focused only on the long-term effects of a professional development programme on the quality and retention of teachers, but it would also be interesting to investigate whether the programme has a long-term effect on student performance.

Despite these limitations, this study provided insight into the long-term effects of a professional development programme and the school organisational activities and characteristics that were perceived as valuable for the sustainability of the professional development programme. The results can contribute to the knowledge of teachers, educators, educational support services, schools, school boards, and researchers concerning the organisation of sustainable teacher professionalisation.

Disclosure statement

No potential conflict of interest was reported by the authors.