Using scaffolding strategies to improve formative assessment practice in higher education

Abstract

Despite the increasing popularity of formative assessment as an instructional tool, teachers find it difficult to implement formative assessment strategies in their practice. To address this we developed a training programme based on formative assessment and scaffolding literature. A quasi-experimental design was utilised to examine the programme’s usefulness; 8 teachers in higher education received the training and eight teachers did not. A questionnaire (N = 260) was administered before and after the training to determine students’ perceptions of their teachers’ adaptive behaviour. The trained teachers also participated in an interview about the usefulness of the training. Although no significant interaction effects between time and group were found, results over time reveal that both trained and untrained teachers showed more adaptive behaviour by providing more challenge when needed. Both groups of teachers also showed more non-adaptive behaviour since they gave more support and challenge than needed and became less supportive when students needed support. Results between groups indicate that trained teachers showed more adaptive behaviour than untrained teachers. Interview results indicate that teachers reviewed the training positively and reported scaffolding theory as a useful addition. Teachers requested more time and support to implement newly learned strategies in daily practice.

Keywords:

• Formative assessment
• teacher training
• adaptive teaching behaviour
• higher education

Introduction

Formative assessment has become a well-known and important concept in education and research (Black and Wiliam 2018; Schildkamp et al. 2020). Formative assessment is defined as a process of constant interaction between students and teacher (Black and Wiliam 2012), and should be viewed as an integral part of teaching and learning (Leenknecht et al. 2021). However, formative assessment approaches are not regularly adopted in classroom practice (Robinson et al. 2014; Boud et al. 2018). Teachers appear to have trouble implementing formative assessment approaches in their daily practice (Schildkamp et al. 2020). They have insufficient knowledge of what formative assessment is (Robinson et al. 2014), and lack practical examples of formative assessment approaches (Box, Skoog, and Dabbs 2015). In this study, we designed a training programme aimed at improving university teachers’ capabilities to use formative assessment approaches in the classroom. To provide the practical examples teachers need, this study makes use of teachers’ support strategies described in scaffolding literature (Van de Pol et al. 2014).

Formative assessment’s cyclical nature

Formative assessment is defined as a process (Black and Wiliam 2012). Most daily educational practices tend to focus on if students have learned, while it seems more logical to also focus on what students have learned to improve both teaching and learning (Pryor and Crossouard 2008). A focus on what students have learned provides insight into what students know or do not know yet, and helps to determine which steps should be taken for future teaching and learning. The process of formative assessment can therefore be seen as ‘a cyclical programme of high and low-stake tasks in which students are actively involved (as assessee and/or assessor)’ (Leenknecht et al. 2021, 237), in order to provide opportunities to apply the obtained insights for future teaching and learning.

Several authors underline the cyclical nature of formative assessment and its subdivision in several iterative, consecutive phases (e.g. Ruiz-Primo and Furtak 2007; Antoniou and James 2014). In most cyclical programmes, eliciting students’ responses and interpreting those responses in relation to the learning objectives plays a central role. For example, Antoniou and James (2014) emphasise the importance of sharing the learning objectives with students at the start of a formative assessment task. At the end of the task, the achievement information gained from the task can be used by the teacher through providing feedback and regulating learning by providing instruction or help.

Gulikers and Baartman (2017) reviewed several formative assessment programmes aimed at developing a more generic formative assessment cycle (see Figure 1). This generic cycle consists of five phases, in which both students and teachers are actively involved. In the first phase, the expectations (e.g. the learning goals) should be stated and discussed. In the second phase, information is gathered on where students stand with respect to accomplishing those learning goals. In phase three, the gathered information is analysed to verify whether it is sufficient to gain insight into what the student has already accomplished and where there is room for improvement. The fourth phase focuses on providing adaptive feedback and discussing this with the student. The fifth phase is about adjusting teaching behaviour in order to properly adapt it to the needs of the student, but also about adjusting student learning behaviour so it fits better with the demands of the learning tasks.

Figure 1. The formative assessment cycle (Gulikers and Baartman 2017).

Whereas in the model of Antoniou and James (2014) providing feedback and teaching adaptively are presented as one-or-the-other options, Gulikers and Baartman (2017) explicitly combine both options in their generic formative assessment cycle. With phase five of the cycle, Gulikers and Baartman place formative assessment in a broader educational context. According to Boud et al. (2018), this is exactly what is needed to stimulate students’ active involvement in and responsibility for their learning. Far too often, the consequences of assessment for teaching and learning are not considered beyond the act of assessment itself (Winstone and Boud 2020). For example, feedback is not transferred to new tasks, or only the instruction of the formative task is adjusted and not the instruction of successive tasks.

Embedding formative assessment in teaching with help of scaffolding

Gulikers and Baartman (2017) indicated that teaching in an adaptive manner is the most difficult phase of the formative assessment cycle for teachers to implement. Most of the reviewed studies reported a lack of follow-up actions and teacher adaptivity, or only stated that teachers implement follow-up actions but not how they do this (e.g. Suurtamm, Koch, and Arden 2010). Teachers’ intentions to implement formative assessment are predicted by several factors. These factors can be contextual factors in the classroom or school environment, but also personal factors like self-efficacy (Sach 2015; Yan and Cheng 2015; Schütze et al. 2017). This self-efficacy can be supported by formal training (Schütze et al. 2017). Unfortunately, current professional development programmes and pre-service education initiatives on formative assessment are often not a remedy for teachers’ feelings of insecurity (Gearhart and Osmundson 2009; Buck et al. 2010). While teachers are looking for practical examples of and guidelines for formative assessment programmes (Box, Skoog, and Dabbs 2015), the examples in the literature on how to take evidence-based follow-up actions appear to be very scarce (Forbes, Sabel, and Biggers 2015).

Where the literature on formative assessment seems to fall short, inspiration can be drawn from the literature on scaffolding. Scaffolding and formative assessment theories appear to be closely related (Shepard 2005) but not well integrated (Van de Pol, Volman, and Beishuizen 2012). Scaffolding is a form of teacher support to help students in reaching their potential performance level (Van de Pol et al. 2014). Both scaffolding and formative assessment stress the importance of active interaction between students and teachers. In contrast to formative assessment, the scaffolding literature does provide concrete suggestions for teachers on how to exhibit adaptive teaching behaviour. That is, it offers a more thorough description of and practical guidelines for adaptive teaching behaviour by describing several scaffolding strategies (e.g. Silliman et al. 2000; Van de Pol, Volman, and Beishuizen 2010). These strategies can provide the practical examples and guidelines teachers need for adjusting their teaching behaviour and provide valuable improvement for current formative assessment training programmes. In the strategies, a distinction is made between what should be scaffolded (i.e. intention) and how the scaffolding should take place (i.e. means). Through distinguishing between means and intentions in a scaffolding strategy, the rationale for a specific type of adaptive teaching behaviour can be substantiated and reflected upon. In the literature on scaffolding (e.g. Many 2002; Silliman et al. 2000; Van de Pol, Volman, and Beishuizen 2010) five scaffolding intentions are identified:

1. recruitment (getting students interested in the task);

2. reduction of the degrees of freedom (simplifying the task);

3. direction maintenance (keeping the learner on track to their specific target);

4. contingency management/frustration control (rewarding and punishing for student performance and keeping students motivated); and

5. student performance and keeping students motivated); and

6. cognitive structuring (providing explanations to organise a learning task).

To support these scaffolding intentions, a teacher can use different scaffolding means. These means can be diverse, e.g. modelling (showing how to do something), instruction (on what to do), explaining (how to do a task in more detail), providing hints (explaining only parts of the task), providing feedback (suggestions for improvement), verification (clarifying or confirming information of the student), verbal participation (i.e. having a student interact in a discussion), and asking a student questions (Pata, Lehtinen, and Sarapuu 2006; Postholm 2006; Van de Pol, Volman, and Beishuizen 2010).

Although the identification of intentions and use of means may be helpful for teachers, this usefulness strongly depends on the teacher’s capability to align the scaffolding strategy properly with a student’s current achievement level (Wittwer and Renkl 2008; Van de Pol et al. 2014). To use scaffolding strategies effectively, it is therefore important for teachers to be aware of their students’ current mastery level. For teachers to achieve this awareness and increase their adaptive classroom behaviour, it is not only important for them to make use of scaffolding strategies but also to make use of all phases of the generic formative assessment cycle.

Research aim

To address teachers’ experienced difficulties when exhibiting adaptive behaviour in their classroom, a teacher training - based on literature on formative assessment and scaffolding - was developed and implemented. In the training, teachers followed the phases of the generic formative assessment cycle and concrete suggestions were provided in phase five to support teachers in exhibiting their scaffolding intentions and associated means in their classrooms. The research aim was twofold, examining the effectiveness of the training in terms of: a) students’ perceptions of their teachers’ (non-) adaptive behaviour and b) teachers’ perceived usefulness of the training.

Prior research on scaffolding has shown that teachers who apply scaffolding strategies are perceived as more adaptive compared to teachers that do not make use of these strategies (Van de Pol et al. 2021). Since the training aimed to support teachers to communicate learning objectives, elicit and interpret students’ ideas, provide feedback on those ideas, and consequently adapt their teaching behaviour based on the previous steps, we hypothesised the training would have a positive effect on students’ perceptions of adaptive teacher behaviour (Hypothesis 1). It was also expected that the training with a special focus on scaffolding strategies would provide concrete guidelines. The lack of these guidelines made teachers experience difficulties with applying formative assessment strategies and exhibiting adaptive teaching behaviour before the intervention (Box, Skoog, and Dabbs 2015, Heitink et al. 2016). It was thus hypothesised that the teachers would perceive the training as useful (Hypothesis 2).

Method

Design

A quasi-experimental pre- post-test design was utilised to examine the perceived usefulness of the training. Teachers and students from two Dutch universities of applied sciences participated in the study. Eight teachers received the training (training group) and eight did not (non-training group). Within each university, allocation to the groups took place based on a representative distribution between groups and teachers’ availability during the fixed training timeslots. The training sessions were organised separately per university, but all associated teachers took part simultaneously within their own university so they could learn from the joint discussions and each other’s experiences. Student perception data regarding their teachers’ exhibited (non-) adaptive behaviour was measured within both groups 2 weeks before the first training session (i.e. pre-test) and 2 weeks after the second training session (i.e. post-test). Teachers’ perceived usefulness of the training was examined through semi-structured interviews within 2 weeks after they completed the training. Teachers in the non-training group received the training after data collection was completed.

Participants

In total, 16 teachers (six female and 10 male), from two Dutch universities of applied sciences voluntarily participated in this study. The subject matter taught by the teachers varied over different kinds of domains, such as maritime technology and social work. The teachers were able to sign up for the training programme through the university website.

Teachers were allocated to either the training (four teachers from each university) or the non-training group (two and six teachers from respectively university 1 and 2). For each teacher, students from one specific class participated in the study. Class sizes were relatively small, averaging around 20 students. In total, 260 students (about 80%) provided data regarding the perceived adaptive teaching behaviour. The training group consisted of 128 students (M_age = 22 years, age range: 17-51 years, including 48 female students), and 132 students were part of the non-training group (M_age = 23 years, age range: 17-55 years, including 40 female students).

Ethics approval for this study was obtained from the first and third authors’ organisation. All teachers and students were informed about the research (data collection, data processing, data storage and reporting). They were ensured that the participation was voluntary and all data would be treated confidentially. All participants provided active consent.

Materials

Training programme

The first (university teacher) and second author (advisor and researcher at a university of applied sciences) developed and implemented the training programme. Both have expertise in formative assessment and are well connected in the field of higher education. The goal of the training programme was twofold, namely: a) getting a more thorough understanding of the generic formative assessment cycle and b) implementing the scaffolding strategies (phase 5: adjusting education) in classrooms. This might remedy teachers’: a) knowledge gap about formative assessment (e.g. Robinson et al. 2014) and b) experienced difficulties with transferring their knowledge to educational practices (e.g. Van de Pol, Volman, and Beishuizen 2010; Gulikers and Baartman 2017).

The training programme consisted of two sessions of 90 min and was provided in Dutch. Session one focussed on increasing teachers’ insight into the whole generic formative assessment cycle. It is important that teachers grasp the overall meaning of the cycle and familiarise themselves with the first phases before they start with the actual adjustment of their teaching behaviour. During the first session, we used several online materials from the Dutch national platform ‘Learning from Assessments’ (https://www.lerenvantoetsen.nl/toolkit-formatief-toetsen). For example, the teachers filled in a scan of their formative assessment activities used in class. They received an example of a possible class situation and descriptions of actions a teacher could take during the first three phases of the formative assessment cycle (see Figure 2). At the end of the session, teachers got the assignment to practice using the first three phases in the classroom they selected to participate in this study. By doing so the teachers were expected to gain more insight into the specifics of and the interplay between the different phases. To make sure their ideas were as concrete as possible, they filled in their personal ‘action plan’ stating what they were planning on doing, how they wanted to do this, and when.

Figure 2. Practical example used in first training session.

The second session focussed specifically on how scaffolding strategies could be implemented in teachers’ daily practices (phase five formative assessment cycle). First, the theory on using scaffolding intentions and means was explained. After this, the example from the first session was repeated to practice the implementation of the explained theory. The trainers provided an example of a student’s response, upon which teachers were asked to determine a suitable next step with the use of scaffolding intentions and means (see Figure 3). Next, the teachers used what they had learned to determine what direction the next lesson of the teachers should take. Again, they wrote down an action plan to make sure ideas were as concrete as possible. Teachers were asked to implement their ideas in class after the second session. Upon request, the training materials can be made available by the first author.

Figure 3. Practical example of using scaffolding strategies, used in second training session.

Perceived Adaptivity Questionnaire

To examine the usefulness of the training programme, data about how students perceived their teachers’ adaptive teaching behaviour was collected. An adapted version of the Perceived Adaptivity Questionnaire (PAQ; Van de Pol et al. 2021) was administered to students participating in the training and non-training groups. Since the PAQ was originally developed for secondary education students, small adjustments were made to increase suitability for the higher education context. The focus on individual tasks was changed to a focus on group work through the use of ‘we’ instead of ‘I’. The questionnaire was translated to English for international students and the Dutch word for a secondary education student (leerling) was changed to university student (student).

The PAQ consisted of 27 items that were targeted at measuring teachers’ adaptive as well as non-adaptive behaviour. Similar to Van de Pol et al. (2021), we used six subscales to distinguish between the type of (non-)adaptive behaviour a teacher can show (see Table 1). The questions were formulated in terms of how teachers regulated student learning based on the diagnosis of their current mastery level of the subject matter (i.e. providing more or less content-related support or challenging tasks). Table 1 shows six example questions, one for each subscale. A five-point Likert-scale from (1) incorrect to (5) correct was used for answering the 27 questions.

Table 1. Constructs measured with the Perceived Adaptivity Questionnaire.

Construct	Items	Description	Example question
Adaptive
More support	6	Teacher provides more support when this is needed	When we really cannot continue our work, this teacher helps us along.
Challenge	5	Teacher provides more challenge when this is needed	When we get a good grasp of the material, this teacher makes it a bit more difficult.
No support	3	Teacher provides no support when no support is needed	When we understand the assignment well, this teacher lets us get started on our own.
Non-adaptive
More support	6	Teacher provides more support when this is not needed	When we are on a roll working on an assignment, this teacher still shows us how to do it.
Challenge	4	Teacher provides more challenge when this is not needed	When we are not able to do something yet, we still have to continue with the next assignment.
No support	3	Teacher provides no support when support is needed	When we find an assignment very difficult, this teacher lets us figure it out for ourselves.

To examine whether construct validity could be ensured we conducted a confirmatory factor analysis (CFA) using R’s Lavaan package (version 3.6.2, see R Core Team 2013). Table 2 shows the fit statistics for a one, two (i.e. adaptive versus non-adaptive) and six-factor model. Since the six-factor model revealed the best fit and moderate fit indices (Hu and Bentler 1999) for both pre-test and post-test measurements we decided to continue with six scales.

Table 2. Goodness-of-fit indicators of models for perceived adaptivity (n = 173).

Model	χ^2	df	χ^2 diff	TLI	CFI	RMSEA	SRMR
Pre-test
Six factor	496.06	309		.84	.86	.06	.09
Two factor	1017.06	323	521.00***	.44	.48	.11	.13
One factor	1048.89	324	31.83***	.41	.46	.11	.13
Post-test
Six factor	502.57	309		.85	.87	.06	.08
Two factor	968.86	323	466.28***	.53	.57	.11	.12
One factor	1039.72	324	70.86***	.48	.52	.11	.13

Note. TLI = Tucker-Lewis Index; CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation; SRMR = Standardised Root Mean-Squared Residual.

***p < .001.

The internal consistency of the six scales was examined with R’s CTT package. Since the commonly used Cronbach’s alpha has several disadvantages (e.g. no internal structure measure, underestimation of reliability; see Sijtsma 2009) the suggestion to use the Omega coefficient made by others (e.g. Cho and Kim 2015; Deng and Chan 2017) was followed. The obtained coefficients ranged between .85 and .95, which can be qualified as very good when making statements about individual persons (Oosterwijk, van der Ark, and Sijtsma 2019).

Interviews

The first author conducted semi-structured interviews with six teachers who participated in the training group (four teachers from university 1, and two teachers from university 2). The teachers were asked to openly reflect on the usefulness of the training. Since the aim was to close the knowledge gap on formative assessment (Robinson et al. 2014) and address the need for practical examples (Box, Skoog, and Dabbs 2015), more specific questions followed about the:

• content (both positive and negative, e.g. ‘What elements of the training programme did you consider as valuable?’);

• practicality of the training (both positive and negative, e.g. ‘What elements of the training programme did not help you in your practice towards becoming more adaptive?’).

Analysis

To test Hypothesis 1, a repeated measures MANOVA was run in SPSS (version 25) with the six adaptivity scales as six separate dependent variables, time as the within-subjects factor (pre- and post-test) and group as the between-subjects factor (training versus non-training). Pre-test and post-test scores from 173 of the 260 participating students (n = 80 non-training group; n = 93 training group) were analysed, since only these students provided all the required questionnaire data. The results from the interviews were used to test Hypothesis 2. All data from the interviews were first summarised. These summaries were member-checked with the participants for approval; based on the check no further adjustments were made. After this check, the data was segmented and coded, based on the five main topics, by the first author.

Results

Hypothesis 1: perceived adaptive teaching behaviour

The average perceived adaptivity scores (means and standard deviations) are presented per group in Table 3. At first glance, the descriptive statistics show several variations in adaptivity scores over time and between groups. Specific increases and decreases in adaptive (e.g. more support, challenge), as well as non-adaptive teaching behaviour (e.g. challenge, no support), are displayed.

Table 3. Average adaptivity pre-test and post-test scores for the non-training, training group, and total (n = 173).

	Non-training group						Training group						Total
	Pre-test		Post-test		Total		Pre-test		Post-test		Total		Pre-test		Post-test
	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD
Adaptive
More support	3.99	.52	3.81	.60	3.90	.05	4.13	.55	4.04	.59	4.08	.05	4.06	.04	3.93	.05
Challenge	2.84	.76	2.99	.75	2.92	.08	2.92	.77	3.12	.78	3.02	.07	2.88	.06	3.06	.06
No support	3.82	.54	3.78	.54	3.80	.05	4.05	.55	4.09	.56	4.07	.05	3.94	.04	3.94	.04
Non-adaptive
More support	2.68	.55	2.74	.53	2.71	.06	2.70	.57	2.69	.60	2.70	.05	2.69	.04	2.71	.04
Challenge	2.68	.71	2.83	.57	2.75	.07	2.69	.69	2.73	.70	2.71	.06	2.68	.05	2.78	.05
No support	2.37	.64	2.57	.65	2.47	.07	2.39	.73	2.47	.76	2.43	.06	2.38	.05	2.52	.05

Note. Range = 1–5 for all variables.

The results of the repeated measures analyses indicated several statistically significant differences over time (F (6, 166) = 4.71, Pillai’s Trace = 0.15, p < .001, partial η² = .15) and between groups (F (6, 166) = 3.10, Pillai’s Trace = 0.10, p = .007, partial η² = .10). No significant interaction effects between the two groups over time were found for any of the adaptivity scales (F (6, 166) = .45, Pillai’s Trace = 0.02, p = .842, partial η² = .02). The main differences over time and between groups are summarised per adaptivity scale in Table 4.

Table 4. MANOVA results over time and across groups (n = 173).

	MS	df	F	p	Partial η^2
Time
Adaptive
More support	1.55	1	9.09	.003	.05
Challenge	2.67	1	13.56	<.001	.07
No support	.00	1	.00	.975	.00
Non-adaptive
More support	.03	1	.26	.611	.00
Challenge	.73	1	4.21	.042	.02
No support	1.59	1	6.49	.012	.04
Group
Adaptive
More support	1.46	1	6.20	.014	.04
Challenge	.51	1	1.04	.310	.01
No support	3.17	1	13.82	<.001	.08
Non-adaptive
More support	.01	1	.03	.866	.00
Challenge	.08	1	.21	.650	.00
No support	.07	1	.19	.660	.00

When zooming in on the effects over time, the results indicate that - regardless of group - there were statistically significant differences between the pre-test and post-test scores for four of the adaptivity scales (see Tables 3 and 4). Students reported that teachers improved (2.88 to 3.06) their ability to give more challenge to students when this is needed (adaptive behaviour). The results on the other three scales with significant time-effects indicate a decrease in perceived adaptivity. Students reported less adaptivity of their teacher in providing more support, meaning that students reported a decrease from pre-test (4.06) to post-test (3.93) in teachers’ ability to give more support when support is needed. On the non-adaptive scales, students reported an increase (2.68 to 2.78) in their teachers’ tendency to provide more challenge when this is not needed and an increase (2.38 to 2.52) in teachers’ tendency to provide no support when support is actually needed.

When zooming in on the effects between groups, statistically significant differences were found between the non-training and training groups for teachers’ adaptive behaviour. Students in the training group reported higher scores on teachers’ provision of more support when this is needed (4.08), and no support when students do not need the support (4.07), than students in the control group (3.90 and 3.80 respectively; see Tables 3 and 4). These results indicate that, according to the students, teachers in the training group were showing more adaptive behaviour than the teachers in the non-training group. Since there is no interaction effect, the found difference in adaptive behaviour cannot be attributed to the training programme.

Figure 4 shows the profile plots of all six scales, illustrating the change in adaptivity scores for both groups separately. The plots show that students in the training group reported higher adaptivity scores for their teachers compared to the non-training group, consistent with the group effects found. When looking at the differences between the groups over time, the plots show that the differences between the adaptivity scores of the training and non-training group increase slightly from pre-test to post-test (albeit not statistically significant). Compared to the training group, teachers in the non-training group showed a smaller increase of adaptive behaviour over time on the challenge scale and a (bigger) decrease of adaptive behaviour on the no support and more support scales. For the scales that showed non-adaptive behaviour, teachers in the non-training group showed a slight increase in non-adaptive behaviour across all three scales. These results are robust across all adaptivity scales (i.e. perceptions in the non-training group develop less positively), indicating that the time differences found are mostly due to changes in perceptions of students in the non-training group.

Figure 4. Estimated marginal means of perceived adaptivity scales (n = 173).

Hypothesis 2: teachers’ experiences

Teachers generally reported being positive about the training programme. They perceived it as ‘useful’ (Teacher 2), ‘interesting’ (Teacher 6) and ‘an eye-opener’ (Teacher 1). The teachers concretised this during the interviews in relation to their practice:

It was useful because you get an awareness of the existence of those different phases. You also start checking: what do you do when the goal of your lesson is not reached? You measure during the lesson and adjust accordingly. (Teacher 2)

Immediately after the first training session, I saw leads for the lesson I had the next day... To say that I can now apply this universally in all types of lessons? No, but the awareness of what is behind all this - the didactics - do help me. I also saw an effect. (Teacher 3)

Teachers 1, 3, 4 and five also reported that the theory on scaffolding was a valuable addition to the training programme. Teacher 4 said: ‘I found the second part on scaffolding to be much more practical, I liked that. In a number of areas it made things more concrete, despite the fact that it was very short’. Teachers reported that the theory on scaffolding could help them in becoming more adaptive: ‘You start asking questions as to why someone is not keeping up’ (Teacher 3).

Even though the general tone about scaffolding and its use is positive, almost all of the teachers mentioned that they had not yet applied what they had learned in the second part of the training programme: ‘I have not really done anything with scaffolding yet’ (Teacher 5). Mostly this had to do with time and planning. After the first part of the training programme, teachers were left with the explicit task of putting to practice what they had learned. After the second part this was also the case, but there was no formal gathering to check and discuss the results. This resulted in teachers who got caught back up in the issues of the day:

It still seems like quite the puzzle to put this in your lesson plan. I notice now that we are too busy for this. For next years’ course, there is a new chance to look at it from that perspective. (Teacher 5)

Almost all teachers responded to these implementation problems by suggesting the addition of one extra part to the current two-part training programme: ‘In that regard, you should actually have at least three parts, in which you also apply what you say in the second part and then provide feedback and evaluation. That is important because it is the most difficult part’ (Teacher 3). Another one said: ‘The advantage of having three moments is that you have the third moment to reflect together instead of alone. Then you do not have to organise it yourself but you can still take a moment to do that’ (Teacher 5). Teacher 4 mentioned that more time and steps were needed to routinise what was learned: ‘I do not benefit from one bucket at the time, I benefit from one decilitre at the time’.

Some teachers recognise the problem that time is scarce and a longer training programme does not solve everything: ‘If we want to do something with this we also have to figure out for ourselves how to continue the work. We should not leave it here’ (Teacher 1). Besides this, the timing of the training programme was not always ideal. Some teachers only saw their class once during the entire programme, which meant they were not able to apply a lot of what they had learned in class: ‘For us, this programme would have fitted better if it came two weeks later or four weeks sooner’ (Teacher 4).

Discussion

This study aimed to support teachers in overcoming the difficulties in the formative assessment cycle when formulating and applying adequate follow-up actions based on students’ responses (e.g. Suurtamm, Koch, and Arden 2010; Schildkamp et al. 2020). In the training programme we designed, we incorporated scaffolding literature to provide teachers with the practical guidelines they are looking for (Box, Skoog, and Dabbs 2015). Interview data support the usefulness of scaffolding literature for this purpose: teachers valued the training programme and theory and guidelines from scaffolding literature. Teachers mentioned that the insights gained from scaffolding literature supported them in working on their adaptive teaching behaviour. While examples of concrete follow-up actions are rarely described in formative assessment literature (Forbes, Sabel, and Biggers 2015), scaffolding literature seems to provide useful insights by distinguishing scaffolding intentions and means (Many 2002; Silliman et al. 2000; Van de Pol, Volman, and Beishuizen 2010).

Despite teachers’ positive reactions to the training programme, we did not find statistically significant effects of the training on students’ perceptions of their teachers’ adaptivity. No interaction effects between time and group were found. Results between the training and non-training groups indicated that teachers who did not take part in the training programme were perceived as less adaptive than teachers who received training (in providing more support when needed and withdrawing support when possible). Teachers’ perceived adaptivity in both groups generally seemed to decrease over time (with three of the adaptivity scales indicating less adaptive behaviour and one scale indicating more adaptive behaviour). Students of the untrained teachers reported a bigger but non-significant decrease in adaptivity compared to students of the trained teachers.

The reported decreases in teacher adaptivity for both the training and non-training groups could be explained by the presence of response-shift bias (Howard 1980). Drennan and Hyde (2008) argue that a pre-test post-test design might lead to an underreporting of any change that occurs since students’ perceptions of the construct have changed. The presence of this bias would mean that students’ perceived teacher adaptivity in the post-test deviated from the pre-test because of the knowledge these students gained about the construct measured. Therefore students might have evaluated their teachers’ adaptive behaviour more critically in the post-test, explaining the decrease in reported adaptive teaching behaviour.

Another possible explanation is that students have not recognised the changes in their teacher’s teaching approach. In case teachers (slightly) adjusted their adaptive teaching behaviour, this adjustment might not have been clear or sufficient enough for the students to change their perceptions. This explanation aligns with the findings of Schütze et al. (2017), who used a questionnaire to capture the changes in teachers’ behaviours in the context of feedback training. They concluded that questionnaires might not be sensitive enough to capture (small) behavioural changes.

Implementing scaffolding strategies

Teachers valued the training programme and expressed the intention to adopt a more adaptive teaching approach in the near future. Previous research reported teachers’ feelings of insecurity regarding formative assessment (Gearhart and Osmundson 2009; Buck et al. 2010), which influences teachers’ intentions to implement formative assessment strategies in practice (Yan and Cheng 2015). In our study, teachers conveyed confidence in their ability to implement the newly taught scaffolding strategies in practice, indicating a high level of self-efficacy. These findings confirm findings of previous studies, which state that teachers’ intentions as well as their confidence or self-efficacy to implement formative assessment can be supported by formal training (e.g. Schütze et al. 2017).

Despite teachers’ reported confidence regarding the use of scaffolding strategies, some contextual barriers for implementation remained. In our study teachers voluntarily applied for the training programme but were not able to spend a lot of time on their professional development, since time spent on other teaching tasks could not be reduced. This may have prevented them from turning intentions into action. Contextual factors play an important role in the implementation of formative assessment strategies in class (Yan and Cheng 2015). Yan et al. (2021) conducted a systematic review on factors that affect teachers’ intentions and implementation of formative assessment. They mention the importance of a positive school environment, strong internal school support, supportive working conditions and encouraging external policies. Although the universities did encourage teachers to take part in the training program, the working conditions did not seem to provide teachers with enough time to change their teaching practice.

In line with previous research (e.g. Sach 2015; Yan and Cheng 2015), we conclude that teacher attendance in a training programme does not necessarily lead to changes in practice that are observable to students. Although the programme did seem to increase teachers’ self-efficacy regarding the implementation of formative assessment practices, contextual factors played an important role in the practical implementation of these practices.

Limitations and suggestions for improvement

When generalising this study’s results to future research and educational practice, it is advisable to take its limitations into mind. First, our study focussed on two universities of applied sciences in the Netherlands and relied on the voluntary participation of teachers. Although voluntary participation in this study had the advantage of attracting teachers who had some affinity and experience with formative assessment practices, the participating teachers are not a representative reflection of the whole university staff. The specific context and small sample of this study should be taken into account when generalising the findings to other contexts.

Second, we used students’ perceptions to measure teacher adaptivity. Student evaluations are commonly used in teacher education and professionalisation practices (e.g. Wubbels et al. 2012; Van der Lans, van de Grift, and van Veen 2015) but are also criticised. For example, the psychometric quality of student evaluation instruments is a topic of debate (Wolbring and Treischl 2015; Goos and Salomons 2017). The main point of criticism is that students’ perceptions of their teacher may reflect more aspects (e.g. personal traits, attractiveness, gender) than solely their perception of teaching behaviour (or a specific aspect of it). To, at least partly, account for any confounding factors we took several measures. First, the PAQ contains questions that focus mainly on observable behaviour instead of opinions or feelings (see also Table 1). With the use of a Likert scale, the measurement was based on a relative (i.e. 1 = incorrect to 5 = correct) instead of an absolute (i.e. agree/disagree) measure for which the effect of a potential bias may be less severe. The CFA for the pre-test and post-test resulted in the same factor structure, so if any confounding factors were measured this affected both tests similarly and, therefore, had a minor impact on the time-related comparisons within and between groups. Last, we collected two types of data (student perceptions and teacher interviews) to gather insights on the effectiveness of the training programme. Since the findings of both sources complement each other (interviews: teachers could not yet fully apply what they learned; questionnaire: minimal changes in teaching behaviour) it seems that confounding factors only played a minor role. To get an even better picture, we recommend using additional data about teaching behaviour in future research. For example, lesson observations appear to provide reliable results for measuring teaching strategies (Van de Grift, 2014). By using more data sources, more insight into teaching behaviour can be provided and used to place the results into a broader perspective.

Third, teachers indicated the need to discuss their ideas on formative assessment strategies after the second training session. The training programme did not provide a possibility for this. In the development of the training programme, we actively intended to find a balance between effectivity and efficiency, keeping the programme as short as possible but still useful. Teachers indicated they liked the duration of the programme, but also noted that more time needs to be devoted to the implementation of phase five of the formative assessment cycle. To this end, they suggested adding a third session in which teachers can further reflect and act upon their application of adaptive teaching based on the insights from the second session. Since the activities in this fifth phase are vital for exhibiting more adaptive teaching behaviour (Gulikers and Baartman 2017), we recommend extending the programme with an additional session. To establish whether this additional investment beneficially affects students’ perception of the adaptive teaching behaviour we also recommend conducting a post-test survey a few months after the training programme to better understand the long-term effects of the programme.

Implications for research and educational practice

This study combined formative assessment and scaffolding literature with the purpose to support teachers in implementing formative assessment programmes in their classrooms. By doing so, it aimed at providing suggestions for advancing the theoretical development as well as the quality of teacher training programmes. Based on our results, it seems feasible to further act upon the suggestion provided by others (Shepard 2005; Van de Pol, Volman, and Beishuizen 2012) to combine the literature from both fields. The scaffolding literature provides the concrete guidelines (i.e. interventions and means) that are lacking in the formative assessment literature (Heitink et al. 2016; Gulikers and Baartman 2017). Since the trained teachers generally valued the training it seems fruitful to develop and implement training programmes that combine the best of both worlds.

Research on formative assessment practices could benefit from measuring adaptive as well as non-adaptive teaching behaviour to gather more insight into the possible (in)effectiveness of formative assessment strategies. The gathered data can also help teachers to gain more insight into their specific strengths and areas for improvement regarding the application of phase five of the formative assessment cycle. The theoretical background and practical use of the PAQ might offer a valuable starting point for gathering this data.

Conclusion

Our study indicates that using scaffolding strategies has the potential to further concretise applying formative assessment strategies in practice. While in cyclical models of formative assessment (e.g. Ruiz-Primo and Furtak 2007; Gulikers and Baartman 2017) taking follow-up actions is considered a crucial final step, scaffolding literature provides practical insights in how to take those effective follow-up actions (Silliman et al. 2000; Van de Pol, Volman, and Beishuizen 2010). Although more research is needed, considering scaffolding intentions and means can provide the concrete starting point teachers need to focus and steer follow-up actions in education, completing the formative assessment process.

Notes on contributors

Stephanie M. A. Kruiper (MSc) is a teacher in Educational Science at the department of Education at Utrecht University. Her research interests concern formative assessment and feedback literacy, focusing on the application of research findings in teaching practice.

Martijn J. M. Leenknecht (PhD) is educational researcher and policy advisor at HZ University of Applied Sciences and coordinator of the Dutch National Platform Learning from Assessment. His current themes of research include motivational teaching approaches, formative assessment, and feedback seeking behaviour of students in higher education.

Bert Slof is an educational researcher and lecturer. He holds a PhD in educational sciences from Utrecht University. His research is aimed at studying the effects of methods (e.g., feedback and visualizations) fostering collaborative problem solving (task-related and interpersonal behavior) and the professionalization of (pre-service) teachers at their workplace. His specialties are: Problem-based learning, Computer Supported Collaborative Learning, External Representations, Instructional Design, Professional Development of Teachers.

Disclosure statement

No potential conflict of interest was reported by the authors.