A practical approach to assessment for learning and differentiated instruction

ABSTRACT

Assessment for learning (AfL) and differentiated instruction (DI) both imply a focus on learning processes and affect student learning positively. However, both AfL and DI prove to be difficult to implement for teachers. Two chemistry and two physics teachers were studied when designing and implementing the formative assessment of conceptual understanding (AfL), as well as whole task-first differentiating instruction (WTDI). The teachers were offered design and enactment heuristics that showed them how they could redesign and enact their lessons to implement AfL and WTDI. The heuristic support was based on theories on decision-making in complex practices. Our assumption was that this support would be considered practical by the teachers and contribute to the implementation of the new practices. Teachers redesigned, enacted and evaluated the lessons using the heuristics. They were interviewed pre- and post, additionally, their lessons were videotaped and lesson designs were collected and analysed. Data-analysis shows that all teachers changed their practices permanently and implemented AfL and WTDI. Although, they considered some aspects as unpractical, the study reveals that the heuristic support was overall practical for the teachers involved and therefore contributes to insight in how to improve implementation of change proposals.

KEYWORDS:

• Professional development
• formative
• assessment

Introduction

Assessment for learning (AfL) and differentiated instruction (DI) both imply a focus on learning processes and learning needs and affect student learning positively (Corno, 2008; Ruiz-Primo & Furtak, 2006; Tomlinson et al., 2003; Black & Wiliam, 1998; Yin et al., 2014). Not surprisingly, both practices have been highlighted internationally in policy documents (EC, 2015). However, AfL and DI are still uncommon, as they prove to be difficult to implement for teachers (Furtak et al., 2008; Mills et al., 2014).

The tendency is to attribute low impact of innovative teaching practices to a deficiency in teacher’s knowledge, beliefs and skills, which often leads to the conclusion that more extensive professional development is needed (Kennedy, 2010). Following Kennedy (2010), it is our contention that this assumption places too much on the teacher and that a more fundamental problem needs to be addressed first: research–based change proposals tend to overlook the demands teachers face in the classroom, and the nature of decision-making in such complex settings, and are therefore considered not very practical by teachers. Teachers always need to achieve multiple goals simultaneously that partly follow from classroom demands and are difficult to weigh (Kennedy, 2010; Janssen et al., 2015). Moreover, they have limited time and resources. Therefore, a teacher will evaluate a change proposal only as practical when implementation does not undermine important goals (e.g. work atmosphere), and when efficient procedures are available that show the teacher how he/she can implement the change proposal in his/her regular practice, given that their time and resources are limited (Doyle & Ponder, 1977; Janssen et al., 2015).

In this study we wanted to explore what practical support for implementing AfL and whole task DI (WTDI, see below) entails for teachers that bridges the gap between proposals and practice. With this aim, we developed practical support for implementing AfL and WTDI, based on a theoretical framework and methodology, that we applied in the context of biology education for innovative teaching practices, e.g. learning-by-designing (Janssen et al., 2013) and open inquiry (Janssen et al., 2015). Concretely, we investigated how two chemistry and two physics teachers used and valued the support for AfL and WTDI in the context of a professional development programme. The participants were experienced chemistry and physics teachers. Their motivation for entering the in-service programme was based on a commonly felt need to expand their repertoire in the direction of AfL and WTDI.

Assessment for learning

AfL focuses on monitoring the quality of the learning process and on providing continuous feedback to guide learning and teaching, which can positively influence learning processes. In the long term, AfL can contribute to the development of metacognitive skills and a feeling of ownership (Black & Wiliam, 2009).

Black and William define the essence of effective AfL as follows:

the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited (p. 10)

Goals and success criteria should be made explicit, students should not be afraid to make ‘mistakes’, and feedback has to be immediate, based on a sound analysis of the ‘evidence’ (Cowie et al., 2018; Luna, 2018).

In the context of science education, AfL has focused mostly on ways to monitor conceptual understanding, as scientific concepts are often complex and counterintuitive. Ruiz-Primo and Furtak (2006) distinguish between formal and informal AfL. Informal AfL activities do not involve a set of predesigned assessment questions. The teacher has the challenging task to create opportunities for informal AfL and to adequately diagnose understanding. Cowie et al. (2018) and Luna (2018) emphasise the importance of teacher noticing in this process: the teacher recognises in dialogue with students what their ideas are and responds in order to help them to construct the scientifically accepted ideas. Their studies show, however, that to adequately elicit each student’s responses, and interpret and recognise underlying ideas, in order to provide adequate feedback is rather challenging in itself (Furtak & Heredia, 2014).

Formal AfL, the focus of this study, is more directive in the sense that prompts are predesigned to make explicit conceptual understanding, which can facilitate the analysis of student responses. Furtak et al. (2008), for example, argue that to be useful, assessment prompts should be designed to make explicit different possible levels of conceptual understanding. A range of possible student responses should be anticipated as much as possible, as these serve as evidence for different levels of understanding that might be present in a classroom.

Hence, the challenge for the teacher is shifted more from interpretation of student responses in-the-moment to the design of ‘useful prompts’ and to the anticipation of the range of possible student responses.

Whole task differentiating instruction

DI proposals are based on insights into differences between students and can take many forms (Tomlinson et al., 2003). An approach to DI is to let students work on common, complex ‘whole’ tasks with common goals, but to tailor support – learning routes – for finishing the tasks to the students’ needs. What students ‘need’ varies. Some students will need a thorough explanation of the theory before feeling sufficiently confident, others can manage themselves with a few hints and a textbook as a reference (Corno, 2008). Ideally, support should be offered to each student at the exact level that enables him or her to complete the task. Several assumptions are underlying WTDI: students are challenged and subject matter becomes more meaningful to them, as it is offered in the context of ‘solving the whole task’ (Lazonder & Harmsen, 2016). The whole task can be used to activate relevant prior knowledge in an integrated manner. Moreover, if students are stimulated to choose the support that suits them, this will contribute to their self-knowledge and meta-cognitive skills (Loibl et al., 2017).

The whole task first approach to DI (WTDI) is very demanding for teachers, however. Students have a certain autonomy of choice and are made responsible for part of the teaching-learning process. Instead of the familiar puzzle-like tasks for practising theory, students are offered more ill-defined tasks that form the starting point for developing theory. All this implies more uncertainty for both students and teachers. Teachers are confronted with at least two practical problems: how to ensure that each student can choose the support they need?, and: How to make sure that each student receives the support at the right moment? Moreover, to design whole tasks and different learning routes is a highly challenging and time-consuming endeavour in itself.

Practicality and classroom ecology

Researchers tend to attribute low impact of change proposals to deficiencies in knowledge and skills, teachers point at their lack of practicality. Doyle and Ponder (1977) identified three interrelated criteria that teachers use to evaluate the practicality of proposals (see also Janssen et al., 2015): (1) instrumentality: are there feasible procedures available that show how the proposals can be implemented in practice, given the limited time and resources?; (2) congruence: are the change proposals sufficiently congruent with current practices that teachers developed over the years, and the goals that they connect to those practices?; (3) cost-benefits balance: do the benefits of implementing the change proposal sufficiently outweigh the costs (time, resources)?

Practicality is not an individual teacher characteristic or belief. Rather, practicality considerations arise from repeated experiences of practising teaching in the unique and demanding ecologies of classrooms. A classroom is a habitat – a context with characteristic dimensions, features, and processes that constrain and guide behaviour for its occupants (Schoggen, 1989). We briefly discuss typical common features of ‘the classroom’ to make insightful the demands that shape practices.

In secondary education teachers typically teach groups of 25–30 students in confined spaces with specific settings and limited resources, for a limited time (say 50 min/lesson). Teaching is group work, hence, teachers design events for groups in a way that at least a minimum of the students’ trust and involvement is secured (Doyle, 2015). In the unfolding of lessons, teachers have to monitor and sustain activities. Routinised activities are low risk in comparison with more loosely defined, cognitive high demand activities, in terms of the probability that they will disturb involvement and trust of students, and, subsequently, a productive work atmosphere. Furthermore, teacher and students share a history that shape ‘how things are done’ (Doyle, 2015). Finally, the content is largely fixed by the curriculum and assessment. This determines behaviour to a great extent (Doyle & Ponder, 1977).

Constraints that follow from the classroom ecology typically translate into goals that teachers cannot afford to ignore and that define what is possible: teachers need to facilitate learning, create a productive work atmosphere, build trust, et cetera.

Decision-making in complex practices

Besides external constraints, people also face internal constraints, as a result of being human. People develop heuristic ways of working to deal with the complexity of the work context, instead of unrealistic, optimal ways: they follow cost-effective procedures (heuristics) to achieve multiple goals in a satisfying manner (Gigerenzer & Gaissmaier, 2011). Heuristics allow us to act on limited information. The gaze heuristic, for example, specifies that a teacher should pay attention to behaviours that are likely to have impact on the group level of engagement. In scanning a class, a teacher would at least pause when it appears that a group impact is potentially occurring. This monitoring strategy enables a teacher to ignore a great deal of behaviour in favour of seeing that which is likely to have consequences, pause to make a quick decision about the degree of potential for a group impact, and intervene if necessary. The result is a smooth running lesson with minimal need for public disciplinary actions (Van den Bogert et al., 2014).

The ‘ways of working’ that people develop and how they connect goals to heuristics can be represented in a goal system (GS) (Kruglanski et al., 2013). A GS consists of a hierarchy of goal-means relationships in pursuit of a task. Goals reflect both classroom demands and personal valued goals and provide insights into teachers’ goal-directed practical reasoning. High in a teacher’s goal-hierarchy are goals that reflect the kind of teacher he/she wants to be (Carver, 2012). Lower goals are means to higher goals. The lower in the hierarchy the more concrete goals become (down to concrete actions). Higher goals can be connected to multiple lower goals and vice versa. People tend to value goals with multiple connections more than goals with fewer connections (Kruglanski et al., 2013). For example, a teacher can connect several lower goals such as discuss homework, explain theory, let students practice and apply theory in assignments to the higher goal students develop understanding, in turn, a lower goal can serve multiple higher goals. Discuss homework might serve the goals checking understanding and maintaining good work atmosphere.

In complex settings people cannot develop ways of achieving all their goals in an optimal way (Pollock, 2006). Rather, people develop practices in a stepwise, heuristic manner, through redesign of current practices, and will assess each step in terms of undermining or contributing to their goals (Westbroek et al., 2015). Only if people estimate the expected value (desirability × probability) of an adaptation higher than the expected value of the current practice, they will feel willing and able to take that step (Pollock, 2006). Expected values are based on motivational beliefs and to what we see as advantages and disadvantages of an adaptation (Ajzen & Fishbein, 2008).

Based on this framework, we can now specify that a change proposal can be practical if it meets the following three criteria.

The change proposal:

1. allows gradual adaptation of the existing repertoire. Adaptations should not deviate too much from a teacher’s GS and with what (s)he considers important goals and means (congruence).

2. provides cost-effective heuristics that show how adaptations can be realised through redesign of his/her current teaching practice (instrumentality).

3. convinces a teacher that the expected value of the adaptation is higher than his or her current teaching practice (low cost).

Heuristic support for AfL and WTDI

In the present study we developed heuristic support that the teachers were offered concerning AfL and Whole-task-first Differentiated Instruction (WTDI), based on our theoretical framework. The heuristics show teachers how they can gradually adapt their lessons, through the use of readily available resources (instrumentality and cost), without disrupting regular practice and connected goals too much (congruence). The heuristics for AfL were based on van den Berg (2003) and Emmett et al. (2009). The WTDI heuristics were based on Janssen et al. (2013) and de Graaf et al. (2018).

Assessment for learning

In the first half year of the programme, teachers were offered heuristic support for AfL, which concerns rather defined teacher-centred activities. The offered heuristics for AfL are related to concept checks and immediate feedback, are geared at monitoring conceptual understanding of all students and can be applied regardless of the type of lesson. They are divided in an assistance in the design as well as in the enactment of the design.

Design heuristics

1. Establish typical conceptual mistakes that students make for the topic at hand, and the typical student reasoning patterns behind these mistakes (use teaching experience and abundant literature on alternative conceptions as available resources). The focus on a typical mistake/reasoning pattern provides the teacher a clear focus for in-the-moment analysis of student responses.

   1. typical intuitive idea that students and lay people have, for example, is that if an object moves, a net force must work on the object in the direction of its motion.

2. Formulate two similar conceptual questions that are directly geared to exposing the same typical mistake and reasoning pattern. This focus on an anticipated intuitive idea facilitates analysis of student responses, and, hence, teacher noticing. Design can be converted into a less challenging selection activity by deriving questions from readily available resources regarding the textbook used, available tests and literature on misconceptions.

   • For example: ask students which force(s) work on a stone after it is thrown into the air A) when it goes up, and B) at the top of its trajectory.

3. Use a format for answering that makes student answers directly visible (multiple choice, drawings, etc.). Such formats allow teachers to monitor and analyse relatively quickly a sample of student answers immediately. See Figure 1.

Figure 1. Example of a format for a series of AfL questions.

Enactment heuristics

1. Make sure that all students have their answering-format in place and pose the first question of the series of two.

2. Walk around and scan the answers of 10–15 students, roughly estimate the percentage of ‘typical mistakes/alternative reasoning patterns’. The design of the prompts, and the formats used, are to facilitate teacher noticing in such a way, that step 2 suffices for a rather adequate impression of most students’ ideas. In the example of the stone, the teacher can relatively quickly see whether students drew a force in the direction of the stone’s motion.

3. Decide on this estimation what to do next: (i) most answers are correct: move on; (ii) about half of the students made typical mistakes: peer-teaching; (iii) most answers are wrong: plenary explanation. Based on the evidence collected in step 2, step 3 offers a practical way to adapt instruction. The specific conceptual focus of the prompts also contributes to this step: appropriate feedback can be anticipated.

4. Pose the second question after peer teaching/explanation to check again. Both teachers and students can check understanding, which possibly leads to a success experience.

Whole-task-first differentiated instruction

Next, teachers were offered support for designing and enacting the more complex teaching practice whole task based differentiated instruction (WTDI).

Many typical science lessons in secondary education have four segments in the following sequence (Lyons, 2006):

1. Teacher explains theory

2. Students apply theory in part tasks

3. Teacher introduces a more complex whole task

4. Students integrate theory in solving a more complex whole task

Two heuristics facilitate the design process as they support teachers in redesigning their lessons, using their own lesson segments as building blocks. This way, mandatory content is still covered.

1. The reverse heuristic: Reverse the sequence of lesson segments by moving the introduction of the whole task (3) to the start of the lesson. To facilitate design, the teacher can use a task that he or she normally offers students later on. The new sequence of lesson segments becomes: 3, 1, 2, 4.

2. The remove-and-build heuristic: Remove all lesson segments that you normally would consider support for working on the task, such as ‘explain theory’. Use your lesson segments to design at least two learning routes.

3. Let students choose a learning route for working on the whole task themselves, instead of assigning a learning route to individual students based on an assessment of their learning needs. This allows students to choose which learning route they feel fits their needs best, which is related to the development of self-knowledge and metacognitive skills.

Similar to the AfL heuristics, the WTDI heuristics offer practical rather than optimal solutions. Instruction is more adapted to the learning needs of all students (and not optimal, which is an utopia), all students have more autonomy. At the same time, the limited number of learning routes and the fact that all students work on the same ‘whole task’ reduces the complexity of teaching. Moreover, students can choose to listen to the teacher’s explanation if they feel uncertain.

Figure 2 presents the different optional lesson designs that can emerge, depending on the teachers’ preferences (expected values).

Figure 2. Three possible lesson designs that can emerge from applying the ‘reverse’ and ‘remove-and-build’ heuristics: version A, B and C.

Research questions

The support was offered to the teachers in the context of a professional development programme. Our aim was to investigate the practicality of the heuristic support, from the teacher’s perspective, and how implementation of AfL and WTDI was influenced. Our research questions were:

1. To what extent did the teachers incorporate AfL and WTDI in their lessons?

2. To what extent did the teachers consider the design and enactment of AfL and WTDI practical?

Method

Two chemistry and two physics teachers participated in the study. We considered the four teachers as separate cases (Merriam, 1998). The cases were alike in that they all built on the same theoretical assumptions underlying the usefulness of heuristic support, but at the same time might differ, due to different contexts of the participants (Yin, 1994). The teachers were first treated as four cases (within-case analysis, Miles & Huberman, 1994). The following data were collected: each teacher’s designs of AfL activities and WTDI lessons and their evaluation reports of enactment; video-tapes of the respective lessons and questionnaires and interviews (see below). By comparing and contrasting the cases, common patterns and discrepancies were identified with respect to design, enactment and long-term impact (between-case analysis, Miles & Huberman, 1994). Examples from individual cases nuance and clarify the more general findings across cases. The different data sources were used to triangulate claims with respect to how the four teachers implemented AfL and WTDI (if), and to what extent they perceived the heuristic support as practical and why.

Participants and context

A total of 15 chemistry and physics teachers volunteered to participate. We randomly selected two chemistry teachers A (female) and B (male) and two physics teachers C and D (male) to participate in the study. Part of the data concern the teachers’ own perceptions of practical usefulness. To involve the teachers in the study and avoid as much as possible socially desirable answers, we had explained to them our theoretical framework and the assumptions that we were investigating and had explicitly asked them – as experts of practical usefulness – to take the role of critical friends. Teachers A and B teach A-level chemistry at the same urban school in the Netherlands. Teacher-C teaches A-level physics at a different provincial urban school. Teacher-D teaches lower general secondary physics in a rural school in the North of Holland. The teachers had between 6 and 15 years of teaching experience. All four teachers taught classes of 25–30 students.

The programme consisted of 15 meetings and covered 1.5 years. In the meetings participants practised with AfL (first half year) and WTDI. The use of the design heuristics and the enactment of the designs were modelled with the participants. Participants redesigned their own lessons and shared and discussed design products. Next, participants enacted their designs in their classes, video-taped their lessons and wrote a brief evaluation on their experiences. Before every new meeting, participants uploaded their work to the coaches/researchers (first and third author) who provided individual written feedback. Each meeting started with a retrospect on experiences. A cycle of design, enactment and evaluation typically covered 4–5 weeks. Both AfL and WTDI were at least enacted twice in the context of the programme, and sometimes more often.

Data collection and -analysis

Below we describe how we collected analysed data for each teacher. In the next step, we investigated the general perceived practicality of the heuristics by establishing to what extent we could identify commonalities and differences between the teachers, and how these could be explained.

RQ 1: To what extent did the teachers incorporate AfL and WTDI in their lessons?

The following data were collected and analysed:

1. Two lesson designs of AfL and WTDI were collected produced by each teacher when participating in the programme, together with videotapes of the respective lessons and written evaluations. The framework in Table 1 was used to establish to what extent the designs met a set of criteria for AfL activities and WDTI lessons that were derived from the design heuristics. Analysis results (first author) were discussed with the second and third author until an agreement was reached. The video tapes and evaluation reports were used to check actual enactment, in order to establish possible deviations and reasons for deviating from the design. This led to a first insight into how (if) the heuristics were applied by each of the teachers.

2. The teachers’ own perceptions of long-term impact on their lessons were investigated by measuring perceived impact of their goal system. Each teacher was asked to reconstruct his or her GS-representation, in an interview (1 h), supported by the first or second author. All interviews were conducted at the school of the respective teacher. We used the well-established laddering interview method, derived from Veledo-de Oliviera et al. (2006) (de Graaf et al., 2018). Each teacher built up a GS with post-its on a flip over in response to the following questions:

   1. Picture what would be a representative lesson in a regular class you would teach that lesson to.

   2. Reconstruct the sequence of building blocks or lesson segments that constitutes your representative lesson: What do you do when teaching such a lesson? What do you typically start with? What do you do next? Describe briefly each of the lesson segments on separate post-its.

   3. Describe briefly for each of the lesson segments on separate post-its how you prepare for the respective lesson segment at home, and

   4. Next, for each lesson segment consider why you implemented this lesson segment in this way, why do you consider it important. Write each of the goal statements on a separate post-it and place it above the respective lesson segment. Connect the goal statements with the respective lesson segments by drawing lines. For each of the goals, ask yourself why you consider the goal important (this question was repeated until the teacher could not come up with yet a higher goal). Describe briefly each of these higher goals on separate post-its and link them with lines to the lower goal(s).

Table 1. Analysis framework for lesson designs.

A digital version of each GS was made and submitted to the respective teacher for a member check. All teachers responded per email that the image of their GS correctly reflected a regular lesson of theirs.

After the 1 ½ years, all teachers were interviewed at their schools (45 min, post-interview). They were confronted with their digital GS-representation and were asked if something had changed in the image as a result of participating in the programme and adapt their GS representation accordingly by indicating/writing changes in the image with a pen (new goals, new goal-means relationships, different order or adaptation of lesson segments, goals that were better/less achieved). Based on this a new digital GS-representation was made, with adaptations in bold. The new GSs were presented to the respective teacher for a member check. Again, all teachers responded per email that the new image correctly reflected a regular lesson of theirs after the trajectory.

analysis was directed at differences between the pre- and post-GS-representations, at the level of:

1. lesson segments: incorporation of AfL or WTDI lesson segments, and

2. higher goals:

   1. goal achievement: improvement or decline of goal achievement as a result of implementation of AfL/WTDI lesson segments

   2. new goals emerge or goals disappear as a result of incorporation of AfL/WTDI lesson segments.

This way, changes in lesson segments could be linked to changes in goal achievement and patterns in changed goal-means relations across the four teachers could be identified. The analysis was done by the first author and checked by the co-authors. The concrete changes that the teachers indicated were all rather straightforward. No differences in interpretation occurred.

RQ 2: To what extent did the teachers consider implementation of AfL and WTDI practical?

The teachers were offered a brief questionnaire immediately after their first design of AfL and WTDI respectively, but before enactment. The teachers were offered the same questionnaire individually at their schools after enacting AfL and WTDI, respectively, followed by a brief interview (20 min) in which they were asked to elaborate their answers. To get an indication of how the teachers valued the heuristics they were asked in the questionnaire to score both the desirability and probability of AfL and WTDI on a bipolar 7-point scale (Ajzen & Fishbein, 2008). This is based on the desirability scale ranged from ‘very undesirable’ (−3) to ‘very desirable’ (+3). The probability scale ranged from ‘I will certainly not succeed in that’ (−3) to ‘I will certainly succeed in that’ (+3). The product of the estimated desirability and probability determines the expected value (Pollock, 2006). Additionally, to explain their scores, teachers were asked to make a list of what they considered important advantages and disadvantages of AfL and WTDI, respectively.

We considered shifts in the expected values of each teacher indications for changes in the extent that each teacher feels willing and able to implement AfL and/or WTDI in their lessons. Such shifts can be further explained by desirability and probability scores (e.g. did a teacher estimate AfL after enactment of more or less ‘desirable’ and/ or ‘probable’?), in combination with perceived advantages and disadvantages. For example, a teacher can estimate that WTDI is more desirable after practising, as he/she experiences that certain anticipated disadvantages disappear and vice versa. Together with interviews about the questionnaires, this combination of data provides an image of the extent to which each teacher perceived the heuristic approaches to AfL and WTDI as practical. These findings were used to explain the differences in implementation (RQ1) and to establish common themes across the four teachers.

Results

Table 2 provides an overview of the findings with respect to each of the teachers. In the following paragraphs we will discuss in more detail the patterns that emerged from contrasting the cases.

RQ 1. To what extent did the teachers incorporate AfL and WTDI in their lessons?

Table 2. Overview of each teacher’s results after finishing the PDP.

Assessment for learning

Our findings show that all four teachers were able to design and enact forms of AfL after two rounds, but none of their AfL activities matched with all four criteria (Table 1).

In the first rounds of design and enactment, especially the physics teachers, typically designed activities that were too complex and assessed too many conceptual understandings at once. Teacher C, for example, asked his 16–17-year-old A-level students to draw what would be the movement of air molecules (horizontal, vertical or still) at different places in the tube of his trumpet (mouth piece, at ¼ of the tube; halfway), if he produced a second overtone (which he did with his trumpet in class). The lesson went extraordinary well, but took two lesson periods and resembled more a WTDI lesson than an AfL activity.

In the next rounds of design and enactment, all four teachers developed a series of conceptual questions that were more directed at specific typical learning problems (criterion 1). They all used formats that enabled them to assess results on the spot (criterion 2). They also all provided feedback after assessing results, mostly in the form of additional plenary explanations (criterion 3). However, none of them used a series of questions that assessed the same learning problem (criterion 4). A criterion that would have enabled the teachers to assess their ‘feedback intervention’ and to create a success experience among their students. Instead, they all made series that assessed slightly different things.

Teacher A, for example, used a series of conceptual questions to assess her 16–17-year-old A-level students’ understanding of weak acids, and whether they were able to apply chemical equilibrium theory when they are to think about what happens when weak acids are dissolved in water. To asses such understanding, she designed a question-series that invited her students to think about weak acid solutions at a particle level. However, the second question, in fact, assesses a slightly more complex understanding of the behaviour of a weak acid solution, as it involves the reaction of an acidic acid solution with a OH⁻ solution.

Whole-task-first differentiated instruction

All teachers were able to design and enact WTDI- lessons that met the criteria described in Table 1. All teachers experienced that their students were more motivated and ‘on task’ and that they were more able to monitor learning processes and provide feedback. However, all teachers also encountered different organisational challenges (see next section).

Teacher-C’s experiences are for a great part representative for the teachers’ experiences with WTDI. He used and slightly adapted the following whole task that he normally would offer his students (16–17-year-old A-level) in a later stage. The objective of the lesson was ‘apply Ohm’s law to calculate with power and voltage’ (criterion 1):

A 110 V – 600 W American iron is connected to a 230 V European outlet. The isolation melts down when the power exceeds 2000 W. Calculate the power the iron will use and predict if this will cause a fire.

Teacher-C offered his students three routes to work on the problem (criterion 2, the other teachers typically designed 2 routes). He had written the worked-out answer on the back of the chalk board. Students that followed route 1 (Figure 2 version B) worked on the task in groups of 4; students that chose route 2, got a worked example and were offered a set of part tasks to exercise calculations with Ohm’s law, they could stop and work on the whole task if they felt they were ready for that; Students that chose route 3 were offered a more elaborate explanation in the form of a YouTube clip, including stepwise modelling of an exemplary calculation task. Next, they could practise more with part task until they felt ready to move to the whole task. In his written evaluation Teacher-C reports that the division in three groups went rather smooth. More than half of the students opted for group 3 and only 4 students for group 1. The remaining students chose group 2. The teacher reported that students were motivated and on task. Students were actually happy to solve the task in the end. Moreover, Teacher-C also reported that he was able to address students’ learning needs much more efficiently, partly as this way of working provided him with more insight into student reasoning (about the whole task). An experience was shared by all four teachers. He experienced some practical problems. For example, the classroom was a bit too crowded so he asked some of his students to work outside class in the hall.

Impact on the teachers’ goal systems

All teachers’ initial GS basically had the following overarching pattern ‘explain – practise part task – practise whole task’. All teachers said they used mostly informal AfL practices to assess learning problems during ‘students practise part task’ lesson segments. Teacher-B indicated that he adapted his instruction somewhat to the needs of his students: students who wanted to skip his explanation and move on directly to the part tasks, were allowed to do so.

For example, Figure 3 shows the GS-representation of Teacher-D. His regular sequence of lesson segments basically followed a rather traditional pattern. Teacher-D used demonstrations, applets and thought experiments to make his explanation lesson segment interactive. He connected multiple goals to this lesson segment: making learning processes visible, anchoring content, involving students in the story, the ability to address the whole class at once.

Figure 3. Teacher D’s GS-representation at the start of the programme.

In the post-interviews all teachers indicated changes in their practices that they related to better goal achievement (including new goals they valued).

All teachers implemented AfL activities at the start and end of the lesson and during their explanation. All teachers implemented WTDI, but not all in the same way. Teacher-B and Teacher-A also used other approaches, such as letting students choose if they want to skip their explanation and prefer to work on tasks. Teacher-C used the WTDI approach only at the end of a chapter (and not as a way to introduce new theory). Teacher-D used the whole task first approach in all levels of teaching. They all perceived their lessons as more interactive and more directed at student reasoning.

The teachers indicated that changes led to improved goal-means relations, and sometimes new goals emerged. Teacher-C indicated that he achieved his goal ‘work order’ (classroom management) better, now that he implements AfL and WTDI more often. Teacher-A and -B both mentioned that they are more able to check understanding and that they learn about differences. In general all mentioned goals that pertain to improved learning processes, such as subject matter is assimilated better, lessons are more meaningful, students learn from each other, students become more independent (each goal is mentioned by a different teacher). Sometimes goals that pertain to the teachers’ own learning and motivation emerged, such as ‘my own learning process improved’ (Teacher-A); and ‘more interesting for me’ (Teacher-D). All teachers additionally indicated that they pay much more explicit attention to learning goals in the planning of their lessons, something that none of them did explicitly before.

Figure 4 shows the changes that Teacher-D indicated in his GS. He now implemented AfL in his explanation lesson segment, and instead of letting students work on part tasks he varies with teaching methods, and regularly uses WTDI. He connects new goals to these changes: it is more interesting to me and students become more independent.

RQ 2. To what extent did the teachers consider the design and enactment of AfL and WTDI practical?

Figure 4. Teacher D’s GS-representation after the programme finished.

In this section we present results that reveal additional, more detailed information about how the teachers valued the heuristic support.

Expected values, advantages and disadvantages

To create a more complete image of how teachers assessed the practical usefulness of the heuristic support, we combined the advantages and disadvantages that teachers wrote down in the questionnaires, with what they mentioned as advantages and disadvantages in the additional interviews. Table 3 shows the advantages and disadvantages of AfL that the teachers wrote down in response to the questionnaire. The four teachers scored these new teaching practices high on both desirability and probability before enacting their first designs of the respective practices (in the session), as well as after a period of 6 months with practising designing AfL (each teacher designed and enacted 2–3 AfL activities, respectively). Table 3 shows that most prominent disadvantages that the teachers mention – preparation time/difficulty and lesson time – are not mentioned anymore after practising with AfL. However, this finding conflicts with the post-GS interviews. In these interviews, Teachers A, B and C indicated that they considered designing a ‘good AfL question series rather challenging’. Teachers A and B additionally suggested to compile a tool kit of readily available AfL resources.

Table 3. The three most salient advantages before and after implementing AfL in the perception of the teachers.

The monthly meetings of the following year focussed more on designing and enacting WTDI lessons, although the teachers could also practise AfL. Table 4 shows the advantages and disadvantages of WTDI that the teachers wrote down in response to the questionnaire. All four teachers designed and enacted at least 3 WTDI lessons in this period and scored the WTDI approach similarly high on both desirability and probability.

Table 4. The three most salient advantages before and after implementing WTDI in the perception of the teachers.

Table 5 shows that Teacher-C scored the probability of the WTDI approach 2 points higher after practising a few times. In the post-interview he stated that he decided to implement WTDI not as a way to introduce a new theory, as he thinks this too challenging for both students and himself. Rather, he implements WTDI as a way to let students practise with applying theory they learned.

Table 5. The teachers’ scores on the desirability and probability on a 7-point Likert scale of AfL and WTDI, respectively, before and after practising. The expected value (E) is the product of the estimated desirability (D) and probability (P).

Noteworthy is that Teachers A, B and C feared to lose overview of what their students were doing. Except for teacher-B, this fear disappeared after practising. In the post-GS interviews all teachers (except teacher-B) indicated that they now had a better insight into their students’ learning. In fact, the WTDI approach enabled them to naturally combine AfL with DI: as students choose a support route themselves, Teachers A, C and D could suffice with coaching students who asked for it in a more specific way. This way the teacher had insight into how students estimated themselves and where they got stuck (if) while working on the whole task. Also, two teachers mentioned as a disadvantage before and after practising that designing a whole task lesson costs extra preparation time. In the post-interviews all teachers mention this. However, all indicated in the post-interviews that they still think that implementing WTDI is worthwhile (which is confirmed by the (shifts in) expected values measures and changes in GSs).

Finally, in the post-interview Teachers A, B and C indicated that they planned to start a similar programme around AfL and WTDI in their own schools, with their colleagues (which they did for three years (A and B) and two years (C)). We consider this additional proof of the practicality of the heuristic support.

Conclusion and discussion

Commonly valued change proposals tend to have a low impact on practice (Furtak et al., 2008; Mills et al., 2014). The general tendency is to attribute this to a deficiency in teacher knowledge, beliefs and skills (Kennedy, 2010). In this study we argued that a more fundamental problem is side-stepped this way: change proposals generally lack practicality. They are predominantly geared at enhancing learning processes, and do not sufficiently take into account other goals that emerge from the classroom ecology. Moreover, change proposals generally do not fit the heuristic manner of reasoning that teachers use for designing ways to achieve their goals (Janssen et al., 2015; Westbroek et al., 2016; Doyle & Ponder, 1977; Kennedy, 2010).

In this study we investigated how four teachers used and valued heuristic support for implementing assessment for learning and whole task differentiated instruction. The support made visible how the teachers could incrementally implement AfL and WTDI, by recombining and adapting their lesson building blocks (including ways of enactment), without disrupting important goals that might reflect classroom demands (congruence). By lowering the demands for implementing AfL and WTDI this way, we expected that a condition for implementation would be met: that teachers estimate implementation as feasible and an improvement (Pollock, 2006).

Based on our findings we can conclude that all four teachers who participated in this study were generally able to use the heuristics offered to incorporate AfL and WTDI in their regular lessons. All teachers also considered the heuristic support predominantly practical.

They implemented a series of AfL activities in their practice, which made their lessons more interactive. (see the Results section, for an example). Additionally, all four teachers implemented WTDI, as envisioned in the PDP, except for Teacher-C who implemented WTDI lessons only at the end of a unit.

We can conclude that the criterion congruence was most met: once designed, teachers had not much difficulty in implementing AfL and WTDI as implementation proved to contribute to important goals and did not disrupt important others. An exception is Teacher-C, who implemented WTDI more as an assessment of understanding after learning theory, at the end of a unit, instead of a way to develop a new theory. Teacher-C, in fact, implemented an adapted, a lower demand and, hence, lower risk, version of WTDI. This might be explained by the fact that Teacher-C had the least teaching experience (6 years).

The heuristics showed the teachers sufficiently how they could redesign their lessons and implement AfL and WTDI. In this sense the heuristics also were sufficiently instrumental. However, they did find designing good concept checks and selecting an appropriate whole task challenging and time-consuming. The four teachers all stated in the interviews that thinking explicitly about learning goals, and whether their lesson plans actually met learning goals, was something they learned. However, this also made their lesson preparation more complex. Being very experienced teachers, they simply were not used anymore to think through their lessons in such a detailed manner, which is in line with Furtak and Heredia (2014). Future research should focus on how the heuristics might be adapted to facilitate this aspect of design. Finally, the teachers stated that the WTDI approach contributed to their insight in their students’ learning processes: they had insight in how students estimated themselves and where students got stuck while working on the whole task. Although we did not investigate the relation between WTDI and AfL, this suggests that the WTDI approach facilitates teaching noticing (Cowie et al., 2018; Luna, 2018). In retrospect, we suggest that AfL might also be used to establish appropriate learning routes in a WTDI approach. For example, specific AfL-concept checks could support students to assess themselves, in order to better decide which learning route suits them. Another interesting venue might be to investigate whether ‘whole tasks’ can be designed to incorporate specific alternative conceptions which allows the teacher to anticipate possible learning difficulties. This might further facilitate teacher noticing. Future research should focus on how WTDI and AfL could be combined.

To conclude, as this study is small scale and explorative, we need to be cautious with generalising conclusions. Also, it focused on teacher beliefs and behaviour. Future studies should connect (changes in) teacher beliefs and behaviour to (changes in) student learning. Still we contend that our findings are more broadly applicable, as all teachers face similar practicality issues (Janssen et al., 2013; Kennedy, 2010). Furthermore, earlier work on different change proposals in the context of biology education (Janssen et al., 2013, 2015; de Graaf 2018) show similar positive results. This suggests that change proposals, as outcomes of educational research, need to be transformed into ‘redesign heuristics’ that are ecologically valid in order to increase impact. Additionally, goal system representations make insightful an individual teacher’s perception of the classroom ecology, by showing how he or she connects goals to lesson design. Therefore, goal systems form the starting point for change at an individual level: which change in practice a teacher might evaluate as an improvement. To conclude, we contend that heuristic support, in combination with a teacher’s goal system, form a bridge between research and practice. It is important to focus future research on the development and assessment of such heuristics.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.