Improving kindergarten teachers’ differentiation practices to better anticipate student differences

Abstract

This article presents the findings from a teacher intervention in Dutch kindergartens aimed at improving teachers’ differentiation practices (DP) to better anticipate student differences. The intervention was designed to improve the match between student levels and curricular activities, in particular for high-ability students and consists of three components. A pretest–posttest cluster randomised design was used with three conditions: control (n = 34), pilot intervention (n = 32) and improved intervention (n = 34). Quantitative results demonstrate that the intervention fidelity was relatively high in the improved intervention. Correlations confirmed that teacher-reported DP were higher in schools where the intervention was more completely implemented. DP were enhanced in both intervention conditions, but showed greater improvement in the improved intervention. Qualitative data revealed process characteristics that reflect problems schools encounter with this intervention. Teachers can be supported in improving DP, but this requires school-wide intensive and long support.

Keywords:

• Screening
• instructional differentiation
• curriculum differentiation
• kindergarten teachers
• school policy

Introduction

Children entering early education differ greatly in terms of cognitive and socioemotional development, socio-economic and cultural background, and so forth, all of which can be highly relevant for their future learning. Unfortunately, these differences are not always taken into account in their formal education. A large-scale study (Engel, Claessens, and Finch 2013) in the United States, for example, showed a misalignment between student skills and content coverage in kindergarten with children often exposed to mathematics content that they have already mastered. Moreover, teachers often provide the same activities to all children rather than using available assessment data to provide different types of activities to children with different ability levels or skills (Al Otaiba et al. 2011). In northern European countries, academic activities in kindergarten may be even less adapted to student cognitive levels, since the emphasis in kindergarten is traditionally on socialisation and play and not “learning” (Oberon 2013; Organisation for Economic Cooperation and Development 2006; Tazouti et al. 2011).

A lack of fit between needs, abilities and prevalent teaching practices often results in children not functioning optimally; that is, they do not function at a level concomitant with their abilities. This is the case for high-ability children who underachieve (cf. Colangelo, Assouline, and Gross 2004) as well as children who lag behind their age peers (cf. Mulder, Roeleveld, and Vierke 2007). Both groups were found to have diverse academic and socioemotional problems due to this mismatch (Claessens, Engel, and Curran 2013; Csikszentmihalyi et al. 1993; Schmitz and Winskel 2008; Tomlinson et al. 2003).

In the Netherlands, differentiation in instruction and curricula traditionally aims to improve the achievement of lower scoring children and enhance the general education level (De Boer, Minnaert, and Kamphof 2013; Doolaard and Harms 2013), hereby neglecting high-ability children. This neglect is not limited to the Netherlands. For example, in the US, Brighton et al. (2005) showed that teachers believed their gifted children did not “need” differentiation; when teachers differentiated, they tended to focus on struggling children. However, optimal learning requires effective instruction interactions (Cabell et al. 2013), in which abilities and learning activities match (Bronfenbrenner and Morris 2006). Instruction and curricula therefore need to be adapted to the levels and needs of individual children – including high-ability children – in a classroom. Unfortunately, within the current educational context, teachers often lack a clear picture of the levels at which their students function (Al Otaiba et al. 2011; Doolaard and Harms 2013; Mooij 2000; National Association for the Education of Young Children 2009). Furthermore, teachers are often unaware of the cognitive levels required for specific curricular learning activities (Mooij 2007; Oberon 2013; Tomlinson et al. 2003). Even if teachers are aware of student levels, they are often unable to adjust curricular learning activities to this.

This article presents an intervention aimed at solving these problems by improving teachers’ differentiation practices (DP) in Dutch kindergartens. The intervention was designed to improve the match between student levels and curricular activities, in particular for high-ability students. The ultimate goal was to enhance the learning and development of all children, including the high-ability children. But before checking student outcomes, it is important to assess the intervention fidelity and actual changes in teacher DP (O’Donnell 2008; Swanson et al. 2013). Whether the intervention succeeded in improving teachers’ DP by using a design-based research approach (Anderson and Shattuck 2012; Barab and Squire 2004) is the focus of this article.

Child characteristics X instruction interaction

Learning is the result of dynamic interactions among innate abilities, environmental characteristics and personal characteristics (Bronfenbrenner and Ceci 1994; Bronfenbrenner and Morris 2006; Gagné 2011; Heller 1999; Heller et al. 2000; Magnusson and Allen 1983; Reis and McCoach 2000). According to Bronfenbrenner and Morris (2006), children develop through everyday interactions with the environment. Socio-cognitive theorists such as Vygotsky (1978) assign a crucial role to the support from the social environment, such as guided participation. The role of education, particularly of teachers, is to allow children to have experiences that are in the zone of proximal development ((ZPD); Vygotsky 1978). ZPD constitutes the difference between what the child is able to achieve without help and with help. Experiences in this zone are essential for learning. As the ZPD differs according to each child’s abilities, effects of particular instructional strategies will vary (Al Otaiba et al. 2011; Connor, Morrison, Fishman, et al. 2011; Connor, Morrison, Schatschneider, et al. 2011; Tomlinson 2005). In this respect, Kalyuga et al. (2003) refer to the “expertise reversal shift”, meaning that instructional designs which are effective with inexperienced or less-knowledgeable learners have negative effects when used with experienced or more knowledgeable learners. Teachers should take these so-called child characteristics by instruction interactions (Connor, Morrison, Fishman, et al. 2011) or aptitude-treatment interactions (ATI, Cronbach and Snow 1977) into account.

Adjusting the curriculum to the learning of individual children and thus meeting children’s developmental needs (National Association for the Education of Young Children 2009) and adequately anticipating student differences involves effective differentiation in content, process, product and/or learning environments, based on variances in students’ readiness levels, interests and learning profile preferences (Tomlinson 2005; Tomlinson et al. 2003). The ultimate goal is to “maximise the potential of all learners by proactively designing learning experiences in response to individual needs” (Santangelo and Tomlinson 2012, 310). This means minimising whole-group interaction and increasing working in flexible small groups and with individual children. Flexible pacing is one way of addressing existing learner variance (see Vygotsky 1978). For example, in a little store set-up in the classroom, one child can be challenged by the teacher to select a certain number of groceries while another calculates the total price of the groceries, and a third by taking in play money and giving back change; all are challenged by different learning goals (i.e. counting, addition and subtraction). By providing additional support for children who need it, and enrichment activities for children who are ready to move ahead more quickly, teachers can maximise learning for all children.

Differentiated instruction in small groups can be beneficial for children of all ability levels with regard to achievement and learning behaviour in various contexts such as literacy skills (Al Otaiba et al. 2011; Connor, Morrison, Fishman, et al. 2011), oral reading fluency (Firmender, Reis, and Sweeny 2013; Reis et al. 2011), mathematics and science content (Simpkins, Mastropieri, and Scruggs 2009; Ysseldyke and Bolt 2007), and study habits, social interaction, cooperation, attitude towards school and general mental health (Gayfer 1991).

Improving kindergarten teachers’ differentiation practices (DP)

Although, in theory, adequately anticipating student differences is possible by providing differential learning activities, many teachers acknowledge that putting differentiation into practice is difficult (Tomlinson and Imbeau 2012). The complex skill of effective differentiation has not been mastered by all teachers (Santangelo and Tomlinson 2012; Van de Grift 2010). Teachers need pedagogical content knowledge (PCK) to guide their actions (Shulman 1986). PCK combines the “how” of teaching, or pedagogical knowledge, with the “what” of teaching, or content knowledge. Several diagnostic, instructional and system aspects can support teachers to optimally combine the pedagogical knowledge with the content knowledge in their DP (Mooij 2007).

First, learning must start with some types of diagnostics of an individual student’s level of competence within a specific domain (Mooij 2007; Shulman 1986). Determining students “entry level and then regularly monitoring their progress is necessary in order to continually modify free play and instruction and vary grouping patterns for meeting students” changing characteristics and needs (Purcell et al. 2002). Teachers often lack the necessary knowledge of both student levels and learning activities to be able to match activities to students’ needs. For instance, Doolaard and Harms (2013) found that Dutch kindergarten and elementary teachers were unaware of their students’ skill levels. An overload of assessment instruments and lack of knowledge on how to effectively use them for monitoring progress may be responsible for this problem (Doolaard and Harms 2013; Tomlinson et al. 2003).

Second, kindergarten teachers need to be comfortable with and gain proficiency in the curriculum they are teaching (Tomlinson et al. 2003) to be able to make appropriate instructional decisions. According to Shulman (1986), teachers need to know how to structure the learning content, what common conceptions and misconceptions students may have concerning this content, and how they can use different teaching strategies to facilitate students’ learning of the content. This requires a solid understanding of a subjects’ (or skills’) learning goals and developmental progression as well as instructional strategies for using a differentially implemented curriculum (Mooij et al. 2014; Tomlinson et al. 2003). Learning processes can be evaluated and managed in order to provide for further learning steps. However, the levels, use and effects of learning materials are often not clear to teachers, in particular the materials for high-ability children (Doolaard and Harms 2013).

Third, beliefs and practices related to systemic issues in the kindergarten and elementary school need to fit differentiated instruction in a flexible system (Tomlinson et al. 2003, 125), as systemic factors such as school climate and resources, as well as teacher attitudes can impede the use of differentiated instruction (Maier, Greenfield, and Bulotsky-Shearer 2013; Roy, Guay, and Valois 2013). A strong focus on covering prescribed curricula, limited space in the classroom, standardised schedules and class time, inflexible routines and management strategies and/or perceptions that the teachers differentiate already complicates differentiation (Engel, Claessens, and Finch 2013; Maier, Greenfield, and Bulotsky-Shearer 2013; Tomlinson et al. 2003).

Intervention

Professional development interventions can help teachers and schools overcome the problems mentioned and improve DP in classrooms (Fullan 2007; Sheridan et al. 2009; Tomlinson et al. 2003). The intervention in this study was designed based on extensive literature review and earlier pilots (Mooij 2008, 2013a, 2013b; see “The Current Study”). It aims at improving teachers’ use of differentiated instruction for all children, with a particular focus on how it could support high-ability children. A detailed description can be found in Mooij et al. (2014). We will discuss the three critical components that are at the heart of the intervention (e.g. O’Donnell 2008).

Three components

The first critical component in the intervention is the screening of entry characteristics of all incoming four-year-old children (Mooij 2000). This is based on the assumption that teachers need adequate knowledge of individual students’ cognitive and social needs and levels to appropriately differentiate in the curriculum. The screening uses a 29-item questionnaire for parents and teachers which is psychometrically based (Cronbach’s alpha range from .65 to .92). It estimates the relative child development in seven areas: social interaction/communication, general cognition, language proficiency, preliminary arithmetic, sensomotor level, emotional-expressive level and expected educational behaviour/motivation of the child. The response options estimate whether a child is less (1), slightly less (2), about the same (3), slightly more (4) or more (5) developed compared with his or her peers. In this way, teachers and parents cooperate and inform each other about the child’s levels and needs when they start in kindergarten so that teachers can make informed decisions.

Second, teachers and researchers collaborated on the development of a differentiation framework .This component is based on the assumption that teachers need knowledge of the structure and levels of the curriculum in order to match children’s levels and needs with appropriate curricular learning activities (see Mooij et al. 2014). This framework – for (preparatory) mathematics and language separately – consisted of a table in which the centrally defined learning goals (Stichting Leerplan Ontwikkeling 2010a, 2010b) were connected with the levels, assessment methods and the learning materials corresponding to these goals. Teachers were asked to arrange learning materials in the cabinets in the classroom according to the levels in the framework. Generally, Dutch kindergartens keep most of the children’s playing and learning materials on shelves distributed throughout the classroom, along the walls, or in the corridors. Children work with these materials on their own initiative, at specific times, or because of an assignment by the teacher who may want to promote some pedagogical or instructional criterion. The way these materials are organised is generally not according to the specific domain or ability level. In this intervention, the materials need to be ordered with respect to content such as mathematics and language, and difficulty level. Teachers use different colours and icons or other symbols on each of the shelves to indicate the ordering of contents and levels. Such prepared playing–learning situations enable small groups of students or individual students to use the materials and instructions independent of their age. Of course, students will always need the teacher, but for different students this support is needed for different types of activities and to varying degrees.

The third component was the development of a policy protocol per school with information about how student assessment and differentiated instruction took place, with a special focus on high-ability children. This component was based on the assumption that a structured, preventive approach provides guidance within the team and for parents. The protocol clarifies for both teachers and parents how the school handles the assessment of students’ levels and what the school’s differentiation approach is for high-ability students in particular.

Support sessions

In school, support or training sessions were organised to help implement the intervention. Presence of kindergarten teachers, the internal counsellor and the principal was mandatory. All principals had received information on the three components of the intervention necessary curriculum changes and how to realise them. The principals were asked to distribute the information to their teachers and regularly discuss the intervention in their team. In the support sessions, the implementation characteristics and changes were discussed. The researchers clarified the meaning of the successive components and promoted the correct development or implementation of the components in each school. Kindergarten teachers within each school were expected to collaborate and develop or implement the components stepwise in their own practice. An overview of all the intervention activities is presented in Table 1.

Table 1. Overview of intervention activities.

Provided by researchers	Content of intervention activity	Hours (Per school)
Intervention (overarching)*
Presentations	- theoretical introduction about learning and instruction, gifted pupils, differentation practices	2
	- introduction to the intervention components
	- examples of components (photographs)
	- description of tasks for teachers over the year
	- supposed outcomes
Manual for teachers	- helps schools with setting up and realising the three components	n.a.
Background literature	- two articles about successful cases of differentiation practices	n.a.
	- models of self-regulated learning projects
	- brochures about learning materials for gifted pupils
Mail or phone support	- whenever teachers needed support	2–20
Component 1: Screening of Entry Characteristics**
Presentations	- underlying theory	3
	- how teachers should use the screening of entry characteristics
	- how teachers could inform and involve parents
	- how teachers could use tests to more precisely determine the competence levels of pupils
Worked Examples	- discussing (subgroups and plenary) a case of screening outcomes with high parent scores and low teacher scores	1
Assignments	- describing the level determination of pupils before the intervention and discussing possible improvements	2–10
	- screening 4-year-old pupils who start in school
	- describing and discussing initial outcomes of screening
	- discussing advantages and disadvantages of different types of diagnostic questionnaires, materials, and tests
Manual	- describing the screening procedure	n.a.
	- describing the usage of the digital portal
	- appendices with information for parents
	- appendices with the questionnaire “on paper”
Feedback	- checking if teachers start using the questionnaire	1–3
	- if not, providing additional information and support
Digital portal	- to support the administration of the online questionnaire and to facilitate the comparison of the questionnaire between parents and teacher	n.a.
Component 2: Differentiation Math/Language Curricula***
Presentations	- underlying theory	4
	- providing examples of structured learning materials in cabinets (photographs)
Worked Examples	- teachers visiting to other schools to see an example of structured differentiated math and language curricula	0–4
Assignments	- introduction (subgroups): describe the initial structure of the curricula and discuss problems and possibilities	4–40
	- use learning materials for number sense to develop a structure for a part of the mathematics curriculum
	- development of differentiated math and language curricula in the school
	- teacher presentation of their developed curricula to other teachers and receiving feedback
Visit by researchers to the classrooms	- to provide feedback on the initial developed curricula	2–4
	- to advise on improvements on the curricula
Feedback	- ongoing support during support sessions	2–4
	- feedback by email/phone on sent-in photographs with explanations by teachers
Component 3: Differentiation Policy Protocol***
Presentations	- underlying theory	2
	- providing checklist of key elements
Assignments	- comparing three protocols	6
	- discussing usage and important elements of the protocol
	- writing a protocol
	- providing a case of gifted pupil showing how the protocol was applied
Feedback	- feedback by email on sent-in protocols	1–2

^*  = Supposed contribution to DP: to enhance teachers’ knowledge about DP.

^**  = Supposed contribution to DP: to enhance teachers’ knowledge and skills concerning diagnostics and assessment DP.

^***  = Supposed contribution to DP: to enhance teachers’ knowledge and skills concerning instruction and management DP and activities for high-ability children DP.

The current study

In this study, a design-based research approach is used to design and evaluate the intervention. Design-based research is a very broad concept (McKenney and Reeves 2012), but is always situated in the educational context, and happens in collaboration between researchers and practitioners (Anderson and Shattuck 2012; Barab and Squire 2004). Design-based research involves an iterative process of (1) investigation or analysis of the problem under study, (2) design and prototyping of an intervention in practice and (3) evaluation of the design and reflection (McKenney and Reeves 2012). By using multiple research cycles, researchers can refine their intervention in order to improve the usage and effectiveness of the intervention. Extensive literature reviews and earlier pilots were described and studied by Mooij (2008, 2013a, 2013b), and formed the basis for the “pilot intervention” described in the current study.

In the first year, a one-year pilot intervention was carried out as described above. At the end of the year, the intervention, more specifically, the intervention fidelity and implementation process were evaluated (see Dijkstra et al., forthcoming). It appeared that not all teachers had an accurate picture of the expected activities in the intervention beforehand. Teachers reported unclear intervention’s means and goal and difficult integration of the complex intervention in the school organisation.

Design-based research allows researchers to refine interventions involving iterations (Anderson and Shattuck 2012), as there is always room for improvements in the design and delivery. Based on the evaluation, the support sessions were adjusted to better and more clearly communicate the intervention in the following year. In the so-called “improved intervention”, the necessary adaptations in teachers’ practices were structured more around the three components and teachers received criteria. The necessary concrete changes in practice were further clarified by using more representations (i.e. images of structured learning material from other schools, etc.), worked examples (i.e. framework of a level-based curriculum, protocols, etc.) and assignments during the sessions. For example, teachers were asked to describe in detail the subsequent steps they took with a child in their class whose entry characteristics were scored highly by parents and/or by themselves. While discussing these cases during support sessions, teachers could adjust their handling and teaching of high-ability children using appropriate differentiation. Setting agreements at each meeting to implement a specific part of the intervention before the next meeting enhanced sustainability. In addition, schools presented and explained to other schools how they tried to implement the components. This enhanced intervention-directed interactions between schools to help schools to learn more from each other.

The main question for the current research was formulated as: Does the intervention for kindergarten teachers positively affect teacher differentiation practices? Teacher-reported DP will be used to answer this question.

Method

A mixed methods approach employing qualitative and quantitative methods in the same study was used as recommended in design-based research (Anderson and Shattuck 2012).

Research design

The study uses a randomised design with “multiple treatments and control with pretest” (Shadish, Cook, and Campbell 2002, 258). Schools were randomly assigned to either the pilot intervention or the control condition. Randomisation ensured that the conditions were similar to each other before the intervention. Teachers in the control condition did not participate in the training and continued with their regular teaching. Teachers in the pilot intervention condition participated in a one-year training. The improved intervention was implemented in the original control group the following year. In this way, all teachers were trained while the research benefits a control group. Table 2 presents the design. Teacher-reported DP were measured at the start (pretest) and end (posttest) of the school year. The posttest in the control condition is used as pretest in the improved intervention condition.

Table 2. Research design.

Year	Pretest	Condition	Posttest
1	DP	Control	DP
	DP	Pilot intervention	DP & Intervention fidelity
2	DP	Improved intervention	DP & Intervention fidelity

The delivery of an intervention as intended or designed (O’Donnell 2008) can influence the effect of the intervention greatly. Therefore, to answer the main question, intervention fidelity was determined. In the current research, we hypothesise that intervention fidelity is higher in the improved than in the pilot intervention condition, which in turn would result in more improved teacher-reported DP. In the control condition, we expect no improvement of teacher-reported DP.

Sample and procedure

Schools were recruited from the school network of the researchers and ranged in size from 42 to 392 children. 19 schools (n = 34 teachers) were initially control schools, 18 schools (n = 32 teachers) participated in the pilot intervention and 19 (originally control) schools (n = 34 teachers) participated in the improved intervention. All schools received a financial compensation to purchase learning material and for allotting teachers the necessary time for intervention activities, where relevant. Kindergarten teachers, internal counsellors and principals participated in about four (range: three to five) regionally support sessions per year, with one to six schools per region/session.

Instrumentation

Intervention fidelity

The extent to which the critical components of the intervention were implemented in the schools was scored at the end of the school year. Each component was scored separately, that is: (1) screening of entry characteristics, (2) using a differentiation framework for the math (a) and language (b) curriculum and corresponding organisation of materials in the cabinets and (3) using a policy protocol for DP and high-ability children in the school. The 3- and 4-point scoring categories (see Table 3) were collaboratively established and defined by the researchers based on 10 randomly chosen schools and then applied to all schools. For the screening, category “fully implemented” means that over 90% of new four-year-old children in the school were screened by parents and teacher, category “partially implemented” means that only a portion of these children was screened and category “not implemented” means 10% or fewer of these children were screened. The differentiation frameworks for math and language and corresponding organisation of materials were scored using categories “developed” which means the framework corresponds with the materials in the cabinets, “advanced development” which means that the school largely developed this, but needed to fine-tune the correspondence between framework and materials in the cabinets, “started development” which means that the school developed a first version of the framework and “not developed”. Categories for the policy protocol were “fully developed” for complete protocols, “partially developed” for a partially developed protocol and “not developed”. The second and third components were collaboratively scored by the researchers.

Table 3. Intervention fidelity (n = 37 schools).

Component and scoring categories	Pilot intervention		Improved intervention
	N (schools)	%	N (schools)	%
Screening of entry characteristics
Screening not implemented	4	22	–
Screening partially implemented	5	28	2	11
Screening fully implemented	9	50	17	89
Differentiation math curriculum
Framework not developed	2	11	1	5
Started structuring in framework and materials	8	44	–
Advanced structuring and organising framework	6	33	5	26
Framework and organisation of materials developed	2	11	13	68
Differentiation language curriculum
Framework not developed	10	56	2	11
Started structuring in framework and materials	3	17	1	5
Advanced structuring and organising framework	3	17	4	21
Framework and organisation of materials developed	2	11	12	63
Differentiation policy protocol
Policy not developed	16	89	1	5
Policy partially developed	2	11	9	47
Policy fully developed	–		9	47

Teacher-reported DP

DP were measured with a self-report questionnaire in which teachers indicated the extent to which 11 statements applied to their DP (range: 0–100; cf. Mooij 2007). This questionnaire was theoretically validated by the research of Mooij (2007, 2008) and empirically used in a number of studies (Mooij 2013b; Mooij, Paas, and Fettelaar 2012). In the study of 2013, for example, Mooij implemented a similar intervention in 8 primary schools for gifted education, and used the questionnaire to assess the DP in the schools.

Face validity was established by experts in the field of instructional strategies and gifted education. Further, the questionnaire was pilot tested on a subset of primary school teachers of mixed-ability schools. After psychometric analysis, a number of bad quality items were deleted from the final questionnaire. The final statements concerned the teachers’ use of specific diagnostics and assessment methods (4 items), instruction and management strategies (3 items) and activities for high-ability children (4 items, see Table 4). For example: “For each subject area, students are grouped in homogeneous groups based on their abilities” and “For high-ability children, additional content or projects are taught”.

Table 4. Spearman ρ correlations between the intervention fidelity and teacher-reported DP at posttest (n = 66).

Differentiation practices	Intervention fidelity
	Screening entry characteristics	Differentiation math curriculum	Differentiation language curriculum	Differentiation policy protocol
Diagnostics and assessment
Assess students’ entry characteristics	.07	.22	.26*	.22
Discuss student levels with parents	.14	.41^**	.46^**	.45^**
Assessment based on observation	.05	.09	.12	.12
Assessment occasions based on students’ readiness	−.05	.27^*	.34^**	.16
Instruction and management
Individual learning pace	.18	.28^*	.27^*	.02
Subject-based ability grouping	−.14	−.09	.02	.14
Independent work in small groups	−.09	−.05	−.09	.05
Activities for high-ability children
Acceleration in subjects	.32^**	.34^**	.27^*	.05
Curriculum compacting	.20	.43^**	.40^**	.31^*
Additional subjects and projects	.36^**	.43^**	.48^**	.28^*
Student choice for extra learning activities	.18	.27^*	.28^*	.20

*  = significant at .05 level.

**  = significant at .01 level.

Field notes

The procedures and consensus reached during the training sessions were recorded by two researchers in field notes. The information was collected per school. In this way, qualitative data concerning the intervention fidelity and the promotion of DP in each school’s practice became available.

Data collection

The DP were measured at the start (pretest, September) and at the end (posttest, June) of the school year. Teachers were invited by email to complete the digital DP questionnaires in NetQ®, a web-based programme for administering questionnaires. If the teachers did not complete the questionnaire within 2 weeks of receiving the first email, the email was sent twice at intervals of 2 weeks. The response rates for the pretest and the posttest were 95 and 100%, respectively.

Intervention fidelity data were collected in several ways. The extent to which the screening of entry characteristics was implemented was tracked for each school in NetQ®. All teachers presented their cabinets with materials to the researchers face-to-face in their classroom. Further, in-between and after the training sessions, teachers sent written frameworks and/or policy protocols to the researchers.

Data analyses

DP data were checked for normal distribution of answers by using the Shapiro–Wilk test (Razali and Wah 2011). None of teacher-reported DP were normally distributed (all p < .05). Therefore, non-parametric tests were used to answer the research question. First, differences in initial teacher-reported DP (pretest) between the three conditions were assessed to confirm that randomisation of schools was successful. The Kruskal–Wallis test revealed that only one DP differed between conditions, namely the screening of entry characteristics (T = 8.50, df = 2, p = .014). Thus, randomisation was largely successful. To test the relation between the schools’ intervention fidelity and final teacher-reported DP, Spearman’s rank order cross-level correlations were used. Teachers’ DP scores at posttest were related to their school’s intervention fidelity as assessed by the researchers. In addition, the Wilcoxon signed-rank test was used to assess pretest–posttest differences in teachers’ DP scores. Finally, the field notes were analysed by ordering and summarising the procedures and outcomes of each intervention meeting per school and across schools. We present a summary of the main issues in the intervention.

Results

Quantitative results

Intervention fidelity

The intervention fidelity in both intervention conditions is shown in Table 3. The screening of new 4-year-old children was met in 50% of the pilot schools and in almost 90% of the improved intervention schools. The differentiation of the language and math curricula (completing a framework of the curricula and organising learning materials based on this framework) was developed in nearly 70% of the improved intervention schools, while in the pilot intervention only two out of 18 schools (11%) fully developed these components. None of the pilot schools and about half of the improved intervention schools fully developed a school-wide differentiation policy protocol. It seemed to be difficult for the schools to establish such a general document, since it must be approved by many persons and could take many discussions and a lot of time. In sum, the intervention fidelity was higher in the improved intervention than in the pilot intervention, which means that the adaptations in the intervention delivery enhanced the intervention fidelity.

Relation between intervention fidelity and final teacher-reported DP

The cross-level relationships between intervention fidelity and final DP were assessed with Spearman’s correlations (see Table 4).

Table 4 shows that 20 out of 44 correlations between the intervention fidelity and the teacher-reported DP are significant. They range from ρ = .26 for the intervention fidelity of differentiated language curriculum with teacher-reported assessment of children’s entry characteristics, to ρ = .48 for intervention fidelity of differentiated language curriculum with additional subjects and projects for high-ability children. Most correlations are found between intervention components and activities for high-ability children. In general, these correlations confirm the relation between the intervention fidelity and the final teacher-reported DP. In other words, teacher-reported DP are higher when the intervention is more completely implemented.

Pretest–posttest differences in teacher-reported DP

Table 5 presents the results for the teacher-reported DP in each condition on the pretest and the posttest. Only one teacher-reported DP (variable testing times (Z = 1.98, p = .048)) was improved in the control condition, which confirms the stability of DP. In the pilot intervention, three teacher-reported DP were improved: teachers more often screened children’s entry characteristics (Z = 4.24, p = .00), assessed using observations (Z = 2.27, p = .02) and grouped by ability per subject (Z = 2.33, p = .02) than before the intervention. In the improved intervention, even more DP were improved. That are screening of entry characteristics (Z = 4.15, p = .00), discussing screening with parents (Z = 3.37, p = .00), assessment by observation (Z = 1.98, p = .048) and curriculum compacting (Z = 1.95, p = .051, borderline significant). Thus, the improved intervention condition was relatively most successful in improving the teacher-reported DP.

Table 5. Medians and ranges of teacher-reported DP in three conditions.

Differentiation practices	Control condition (n = 34)				Pilot intervention (n = 32)				Improved intervention (n = 34)
	Pretest		Posttest		Pretest		Posttest		Pretest		Posttest
	Mdn	Range	Mdn	Range	Mdn	Range	Mdn	Range	Mdn	Range	Mdn	Range
Diagnostics and assessment
Assess students’ entry characteristics	75	100	70	100	50	100	98**	90	70	100	100**	90
Discuss student levels with parents	75	100	75	100	50	100	50	100	75	100	100**	100
Assessment based on observation	75	75	75	100	75	100	100*	50	75	100	100*	50
Assessment occasions based on students’ readiness	50	100	50*	100	50	100	50	100	50	100	50	100
Instruction and management
Individual learning pace	75	90	50	100	75	100	63	75	50	100	55	100
Subject-based ability grouping	58	100	65	100	50	100	75*	100	65	100	75	75
Independent work in small groups	80	50	80	75	78	80	78	75	80	75	80	75
Activities for high-ability children
Acceleration in subjects	50	100	50	100	50	100	75	100	50	100	75	100
Curriculum compacting	60	100	50	100	50	95	50	90	50	100	75(*)	100
Additional subjects and projects	50	95	50	90	38	100	33	100	50	90	55	100
Student choice for extra learning activities	50	95	40	100	33	100	50	100	40	100	55	100

^*  = significant at .05 level.

^**  = significant at .01 level.

Qualitative results

Summary of field notes

This section presents the main issues a majority of schools encountered, as noted by the researchers during the training sessions.

Working along the intervention guidelines was new for kindergarten teachers. The first challenge for the teachers was understanding the pedagogical-psychological approach to systematically improving development and learning for every child, and the aim of realising this approach by providing corresponding systemic play, learning and organisational conditions in each class.

In addition, the activities for increasing student-based differentiation in a traditional age-related school required a lot of comprehension and additional work by the teachers. The researchers observed that some kindergarten teachers showed resistance to systematically educating young children, because the teachers preferred a play-based mentality in kindergarten. Other teachers believed that students at the higher end of the ability spectrum had no need for difficult subject matter.

It became clear during the intervention sessions that some teachers or schools initially resisted screening entry characteristics. For example, some teachers and schools were not keen on using “another” questionnaire for parents, mainly because the schools did not want to “bother” parents or because the schools expected some parents would act in a problematic way. However, after several weeks of using this questionnaire and comparing the parents’ outcomes with the teachers’ outcomes, the kindergarten teachers indicated that the screening questionnaire gave them valuable information about the child’s development. Apart from the initial resistance, implementing the screening in schools was found to be convenient.

The main qualitative finding is that, as expected, matching the outcomes of the screening for each student with the appropriate levels of challenge in learning tasks is essential at the beginning and for further development of each student. The training sessions showed that teachers and schools lacked enough time and expertise to include complete diagnostic information about student levels based on screening and norm-based and criterion-based tests. This was further complicated by the overload of different types of assessment instruments and learning materials.

Findings

It is important to study the implementation of an intervention before studying students’ outcomes, as the first can be more or less optimal and thereby effecting the latter. Our research shows that the intervention with three critical components based on important features for differentiation is not easily adopted by teachers. Schools in the pilot intervention scored non-optimal on intervention fidelity. A higher level of intervention fidelity in the improved intervention condition indicated a higher level of implementation. Spearman’s correlations confirmed the significance of meaningful relationships between the intervention fidelity as assessed by the researchers and teacher-reported DP in schools. Furthermore, there was a difference in effect of the pilot and improved intervention: teacher-reported DP were enhanced in both cases, but showed relatively great improvement in the improved intervention.

Discussion

Optimal differentiation in instruction and curricula between students in a mixed-ability classroom asks a lot from teachers (cf. Hertberg-Davis 2009). In this study, teachers reported doing more DP related to level determination of children, but these practices are only the first step in a continuing process of offering appropriate play and learning tasks. Further transforming a school’s orientation and change for effective DP including diagnostically based variations in free play and curriculum progress turned out to be more difficult (see also Mooij 2007). This requires teachers to have a strong PCK base to be able to identify learning needs of all students and respond to these differences with learning experiences that match these needs (Shulman 1986). Thus, further research should more specifically focus on how teachers can use the estimations of the school-entry characteristics of each incoming child in primary school as starting point to realise more appropriate placement of the child in the main instructional lines of the differentiated curricula, including appropriate content complexity and instruction.

In this respect, ICT has the potential to support the educational differentiation needed for students (Mooij 2008). The current ICT-based student-monitoring systems for schools are organised according to students’ age and grade (Nieveen and Kuiper 2012) and therefore not 1:1 suitable for more differentiated learning, independent of age and grade. Further research could focus on the best ways to use and integrate ICT programmes in everyday teaching practice.

Some teachers in this study perceived that high-ability children had no need for difficult subject matter (cf. Doolaard and Harms 2013; Reis and Westberg 1994). This noncompetitive mentality has traditionally been present in Dutch society (Hofstede 1997). In this respect, Mooij and Fettelaar (2010) clarified that equal opportunities in education are often confused with uniform teaching. Equality is then psychologically mistakenly interpreted as equal education for every student, regardless of level. As one of the main aspects of pedagogigical content knowledge, teachers have to understand that the curriculum, instruction and learning pace can be different for different students in the same classroom (Shulman 1986). This study stresses the importance of taking teacher beliefs into account when schools want to implement differentiated curricula in their teaching. More research into adapting teacher beliefs for optimal differentiated instruction for all students including high-ability students is recommended.

For a successful implementation of differentiated instruction, the teachers themselves must be motivated to change existing practices, feel ownership for the educational change, work in a professional team culture and require adequate support from the principal (Fullan 2007; Oberon 2013; Sheridan et al. 2009; Smit and Humpert 2012). These factors were not present in all schools (Dijkstra et al., forthcoming). For example, the principal was invisible in some schools, and negative teacher attitudes turned out to be persistent, resulting in continuing resistance to facilitating high-ability children at their own level. Other studies have also found that teachers’ attitudes and beliefs were an obstacle (Hertberg-Davis 2009; Smit and Humpert 2012). The stronger intervention fidelity and DP improvements in the improved intervention could indicate that teachers’ motivation and ownership were higher compared to the pilot intervention. Still, the qualitative findings show that the intervention is in need of more and stronger support to be fully integrated into the primary schools.

Limitations

This study is based on a sample of schools taken from the researchers’ network of schools, with an interest in curricular differentiation and high ability. The sample is thus not completely representative of Dutch education and may be somewhat biased. However, as shown in the studies cited, comparable processes and experiences as reported in our study have been found by other Dutch and international researchers which supports the present findings and conclusions. However, this limitation also indicates that trying to realise comparable changes in kindergarten and elementary schools involved in a representative, nonself-selected sample will be even more difficult.

A second limitation is that we used DP as reported by teachers, who may be sensitive for socially desired answers (Wen, Elicker, and McMullen 2011). However, Desimone, Smith, and Frisvold (2010) observed that several studies have shown that anonymous teachers’ self-reports on their teaching are highly correlated with classroom observations and that one-time surveys that ask teachers questions about the content and instructional strategies the teachers emphasise are quite valid and reliable. In this study, the concrete actions, learning material and descriptions with students presented during the training provide some tangible evidence of teachers DP. However, in future studies classroom observations should be included in the research design to gain more information about differentiation and relevant applied practices in classes, and to further increase and check the validity of the assessments.

Implications

As the study shows, implementing differentiated instruction, including instructional adaptations and academic progress monitoring (Roy, Guay, and Valois 2013), is not a minor transition (Hertberg-Davis 2009). Focusing on support for and ownership of the teacher is important (Bergen and Van Veen 2004). A change in mentality is required for some teachers before they will teach students at the lower and higher ends of the spectrum at their own level. It is important that these beliefs are identified before participation and are anticipated by the deliverers of a training or intervention as much as possible, because if the teachers do not want to change, the change will not succeed.

A next research step, thus, could focus on how teachers could best be taught (i.e. in pre-service) and trained (i.e. in in-service) in improving their differentiation skills. Here, pre-service and in-service teacher education can play an important role by providing teachers with a stronger PCK base, including knowledge about the characteristics and needs of high-ability students and by more intensive training in the acquisition of differentiation skills.

Systemic change requires a much longer period than one year to be completed (Duffy et al. 2006). For sustainable implementation, long-term professional development is also important (Fullan 2007). After the intervention ended, some schools in this study decided to collectively continue improving their DP. Ongoing support would give these schools the opportunity to further develop teaching practices for diverse children. Here, a community of practice approach (Lave and Wenger 1991) is recommended, wherein participants can share experiences with colleagues. Causal analyses of longitudinal student data should then also be used to indicate whether the education and differentiation improvement at school and class level improved the students’ cognitive and socioemotional functioning.

Notes on contributors

Elma M Dijkstra (1987) is a PhD candidate at the Welten Institute, Research Centre for Learning, Teaching and Technology at the Open University of the Netherlands. Her research is centred around the optimalisation of education for gifted pupils, especially in Kindergarten. Her PhD project involves the training of teachers to anticipate pupil differences and apply adequate DP. She studies the effect of teachers’ insight into children’s levels of cognitive and social developments, teacher subject and curriculum knowledge, and school-wide policy for integrating differentiation on the cognitive and socioemotional development of (initially) excellent and nonexcellent kindergarten children.

Amber Walraven (1980) is a senior researcher at the Institute for Applied Social Sciences (ITS), Radboud University. She is an expert in the field of ICT in education. Integrating ict in education, and what that means for teachers, students and organisation are the main focus of her research. She is also the project leader of two projects around optimal education for excellent students. Both projects involve the implementation of new approaches, materials and procedures in everyday education. Very similar to ICT in education this affects teachers, students and organisation. The ways teachers go about these (ict and education for excellent students) changes, effects on student learning and what others can learn from it, are discussed in various publications.

Ton Mooij holds a position as a professor by special appointment for educational technology at Welten Institute (Research Centre for Learning, Teaching and Technology), Open Universiteit in the Netherlands. He also served as a manager and senior researcher at ITS (Institute for Applied Social Sciences) of Radboud University Nijmegen, the Netherlands. His research combines ICT-supported assessment and improvement of school processes including cognitive and social learning effects with pupils in primary and secondary education. His main topics of interest concern the high ability of pupils, development of differentiated education and support for optimal cognitive and social learning in school practice, school safety, and prevention of bullying and violence for both pupils and teachers in and around schools. He has been awarded several prizes for his publications focusing on theory and experimental research on cognitive excellence in elementary school practice. He has conducted six national surveys into pupils’ and teachers’ social safety, bullying and violence, in both primary and secondary education. Ton Mooij also investigated various school-based intervention practices to effectively prevent cognitive and social problem behaviour of pupils.

Paul A Kirschner (1951) is Professor of Educational Psychology and Programme Director of the Fostering Effective, Efficient and Enjoyable Learning environments (FEEEL) programme at the Welten Institute, Research Centre for Learning, Teaching and Technology at the Open University of the Netherlands as well as Visiting Professor of Education with a special emphasis on Learning and Interaction in Teacher Education at the University of Oulu, Finland. He is an internationally recognised expert in his field. He is a past President (2010–2011) of the International Society for the Learning Sciences, a member of the Scientific Technical Council of the Foundation for University Computing Facilities (SURF WTR), is the chief editor of Journal of Computer Assisted Learning and an associate editor of Computers in Human Behaviour and chief editor of the Journal of Computer Assisted Learning, is author of the highly successful book Ten steps to complex learning and an editor of two other recent books (Visualising Argumentation and What we know about CSCL).

Disclosure statement

No potential conflict of interest was reported by the authors.

Acknowledgements

This research was supported by Agentschap NL under grant number ODB 10029.