Using structured reflection to encourage more active use of feedback

Abstract

Students are frequently dissatisfied with the feedback they receive but providing more feedback is often unrealistic for staff. This paper reports on activities which helped students to make the most of the feedback they receive. Students were set weekly tasks and received rapid feedback aimed at the whole group. In-class activities, including personal reflections, encouraged engagement with the feedback. These activities turned feedback from a passive ‘receipt’ process to an active dialogue, in which there was scope for future action. The class became more aware of the variety of feedback methods being used with them, increased in confidence and changed their perceptions of the value of feedback to inform their work.

Introduction

The quantity, quality and timing of feedback are currently ‘big issues’ in university education. The National Student Survey repeatedly finds that ‘Assessment and Feedback’ is the area with which students are least satisfied. In general, students want more, clearer and faster feedback (National Union of Students, 2008). At the same time, staff are juggling high workloads and increasing class sizes. This discord is heightened when students appear not to act upon the feedback they have been given. One solution is to look critically at the whole feedback cycle. Instead of focussing exclusively on the quality of written feedback, staff and students alike can gain by placing more emphasis on developing self-evaluation skills in students (Yorke, 2003).

Student-centred models of feedback emphasise its cyclical nature, inherent in which is the notion of feedback as ‘dialogue’ between staff and student (Nicol and Macfarlane-Dick, 2006; Nicol, 2008). Unfortunately, the reality often mirrors a linear process or staff ‘monologue’, where students do not engage as active participants in their own learning. Student performance improves when students’ self-evaluation skills are fostered (e.g. McDonald and Boud, 2003; Rust et al., 2003). The inclusion of self-management and independent learning skills in Benchmarking Statements highlights their importance in Higher Education (HE) (Q.A.A., 2007).

The traditional staff-student dialogue developed through tutorials or seminars does not work with large classes. Here, we focus on group feedback, which is time-efficient for staff to provide, but is often very poorly used by students. We present the key findings from a study in which structured reflection was used to encourage students in large classes to engage with feedback.

Methods

Module context

ENV201 Project Skills is compulsory for students studying Environmental Science (ES) (approx. 70–80 students at Lancaster). The module has dual aims of developing skills in data presentation and analysis, and in report writing. It is closely integrated with a seven-day residential field course (ENV200 Carrock Fell) running immediately prior to the module. The assessment for ENV200 includes a 2000-word report, due at the end of the term.

The report writing element is jointly taught by two of the authors. A writing topic is introduced each week (e.g. structuring an introduction, writing an abstract, structuring your results), followed by a short (< 1 page) practice task. These tasks are unassessed, but map onto components of the assessed ENV200 report (e.g. prepare an outline introduction / the abstract / outline results section for your ENV200 report). The tasks are submitted online the next week for tutor review. The tutor skim reads the submissions looking for patterns (e.g. common mistakes), then selects examples of work completed well or poorly for class discussion, and prepares group feedback. The following day a feedback session is used to analyse the submitted work and provide general feedback (Table 1). In total, 6 tasks are completed by the students during the 10 week module.

Intervention

For this project we focussed on improving the use of group feedback. At the end of each feedback session, we introduced a 5-minute reflection period to encourage engagement with the feedback material, during which the students were asked specific questions such as:

“What have I learnt during today’s session?”

“What will I change in the next draft of this piece of work?”

These responses were written down and entirely personal to the students. Together with the VLE discussion and the Post-it note questions (Table 1), these activities were designed to develop dialogue between staff and students.

Evaluation

At the start of the year, the whole class was asked to complete a baseline questionnaire about their experiences of tutor feedback on 1st year assignments, with an emphasis on five common methods (comments in the margin, paragraph at the end, marking grid, one-to-one and group feedback). There were nine questions (Table 2) and space to add comments about issues such as timing, detail, variation between assessments, particularly helpful feedback and how the feedback influenced ways of studying.

A focus group of 15 students was recruited; the participants were volunteers and reflected the gender composition of the cohort (40% female, 60% male) and the range of academic achievement. Recorded discussions, led by a colleague from outside the department (Ali), were held at the start and end of the year, in groups of 5 or 6 students. In the first focus group discussion focussed on how the students had experienced feedback in the 1st year. In the second focus group, the students were asked to reflect back on their experiences of the feedback strategies used in ENV201.

Table 1 Summary of feedback methods employed in ENV201.

Type of feedback	Description
Feedback sessions
Self-assessment in class	The feedback session starts with a reminder of the key features of the task. Students then assess their own or a fellow student’s work against given criteria.
Peer assessment in class
Group oral feedback	During the feedback session, examples of student work are projected for the whole class to see. Students led the annotation of the work, using a tablet laptop, assessing the work against the same criteria as used above. They are encouraged to fnd positive points as well as criticise the work. Examples are selected to illustrate a range of typical errors and demonstrate good practice.
Discussion of examples of student work
Verbal responses to individual questions (one-to-one)	Students are able to ask the tutors questions throughout the class. Tutors move around the class during the initial activities, providing opportunities for one-to-one questions.
Verbal responses to individual questions (whole group)
Post-it note questions	Students write questions on Post-it notes during the feedback session. The answers are either given in class if time allows or through the VLE.
Self-evaluation in class Written group feedback	See intervention questions below. At the end of the feedback session a summary (1 side A4) of key feedback points is distributed. The feedback summarises 3 things that were done well and 3 things that were incorrect, each ranked with ticks and crosses (e.g. 3 ticks, very good; 1 tick, OK).
Questions on VLE	At all times, students are encouraged to ask questions via the VLE. Answers are provided by fellow students or tutors.
Other feedback activities
Peer evaluation of group work	Students evaluate group posters and give oral feedback.
Written comment on group work	The tutor provides a written feedback summary on the group posters.
Model answers on VLE	Detailed worked answers are provided for statistics tests.
Individual feedback
Individual written feedback	Individual written feedback is provided for the report at the end of this module.

Table 2 Initial questionnaire questions.

Question 1	I read the feedback before looking at the grade (y/n)
Question 2	I did not read the feedback - I just looked at the grade (y/n)
Question 3	The feedback affected the way I felt about the grade given (y/n)
Question 4	I had a follow-up conversation with my tutor after receiving the grade/feedback (y/n)
Question 5	I found it easy to understand what the tutor meant in the feedback given (0 = very diffcult - 3 = very easy)
Question 6	I found the feedback helpful for indicating what I had done well (0 = not at all helpful - 3 = extremely helpful)
Question 7	I found the feedback helpful for understanding where I had gone wrong or omitted things (0 = not at all helpful - 3 = extremely helpful)
Question 8	The feedback caused me to think about how much I had gained from the module. (0 = not at all - 3 = a lot)
Question 9	I used the feedback to guide my next piece of assessed work. (0 = I did not use it at all - 3 = I used it a lot)

Results and discussion

The first year position

The majority of students responded to the questionnaire (91%, n = 67). All students had received 1st year feedback in the form of margin comments and a paragraph of text at the end of their work (Table 3). Approximately two-thirds had received feedback as a marking grid or group feedback. Only 25% had received one-to-one feedback.

The majority of students looked at the grade before looking at the feedback, and a significant minority did not read the feedback given at all. At least 50% of students stated that the feedback affected how they felt about their grade, with a paragraph of text having the most impact and group feedback having the least. Very few students sought out a follow-up conversation with their tutor.

Table 3 Percentage of students responding “yes” to each question for the fve feedback methods evaluated

Question	Comments	Paragraph	Grid	One-to-one	Group
N (receiving this type of feedback)	67	66	43	17	42
Percentage of all respondents	100%	99%	64%	25%	63%
I read the feedback before looking at the grade.	15%	21%	16%	18%	17%
I did not read the feedback — I just looked at the grade.	13%	14%	16%	6%	19%
The feedback affected the way I felt about the grade given.	52%	68%	56%	59%	48%
I had a follow-up conversation with my tutor after receiving the grade/feedback.	4%	8%	0%	24%	5%

Figure 1 summarises the responses to the Likert scale questionnaires. There are no strong differences between the different feedback methods in terms of usefulness to the students. In general, the students considered that all the feedback methods were relatively easy to understand, and that they were helpful both in indicating what the student had done wrong and what they had done correctly. However, the majority of students did not feel that the feedback caused them to reflect on what they had gained from the module, nor was it useful for guiding their next piece of work. The one exception was one-to-one feedback, which did promote a more significant level of reflection. In comparison to the other methods, group feedback performed similarly or slightly worse across all questions.

The most common issues raised in the free text responses were: more one to one feedback would be useful (24%); more detail and clearer explanations needed (24%) and not enough feedback given (21%). This comment is typical:

“Feedback … is often vague and doesn’t always explain where we did well / badly. Most of the time it’s just ticks in the margin, the odd “why” and then a mark at the end.”

Group feedback was not raised at all in the free text responses, perhaps indicating that the class did not perceive this feedback method as important to them.

The issues raised in the questionnaires were explored further in the focus groups. Overall, the tenor of the discussion was more negative about 1st year feedback than the questionnaire responses indicate. The majority of students in this class have moved directly from studying A-levels in a school or 6th-form college to university. During A-levels, the norm was to submit multiple drafts for correction by teachers. The students were surprised when, at university, they were expected to hand in a final version, with no opportunity for re-submission. However, they recognised that the school model did not help them become independent in their learning.

“… the nature of the feedback changes between ‘A’ level and university … (in school) it was basically ‘you should have put this in’ and now (at university) it’s like, ‘you’ve missed out something’ and you’ve got to … figure it out for yourself …”

It was clear that the students felt no sense of dialogue with their tutors, and most felt unable to seek one-to-one support, particularly students who felt they had done poorly in a piece of work. The students valued individual written comments on their own work, which they would ideally like to be followed up with a one-to-one conversation with the tutor. Finally, they were unable to recognise the links between their work for different modules, and did not consider using the feedback from one to guide the next.

End of year two responses

The initial question for the end of year focus groups was “What types of feedback did you receive in ENV201?”. The responses are summarised in Table 4. The students all mentioned the group feedback as their first or second recollection. Although individual written feedback was still prominent, the students were much more aware of the range and variety of feedback methods being used. They regarded questions and answers in class and on the VLE as feedback, unlike at the start of the year. The students all clearly valued the opportunities for classroom dialogue during the module.

Figure 1 First questionnaire responses, where “0” is “not at all helpful / useful” and “3” is “extremely helpful / useful”.

Table 4 Types of feedback used in ENV201 as recalled by the students 5 months after the completion of the module.

Type of feedback	Order of recollection
	Group 1	Group 2	Group 3
Group feedback
Written group feedback	1, 12	2	1
Group oral feedback	2	4, 11	2
Discussion of examples of student work	11	3	6
Peer evaluation of group work	4	10	7
Written comment on group work	4	10	7
Model answers on VLE	—	—	8
Questions on VLE	15	8	3
Feedback to group in response to individual’s question
Verbal responses to individual questions (one-to-one)	6	6, 9	—
Verbal responses to individual questions (whole group)	7, 9	7	8
Post-it note questions	—	12	5
Individual feedback
Individual written feedback	3, 5, 8	1	4
Self and peer assessment
Self-assessment in class	13	5, 14	2
Peer assessment in class	13	5	—
Self-evaluation in class (how would you improve this work?)	14	13	—

The discussion then turned to an evaluation of the feedback methods experienced during the module. Feedback methods the students found particularly helpful are outlined below, illustrated by student comments.

The task-based approach to report writing was positively received (the mean class submission rate for the weekly tasks was 83% in 2007 and 75% in 2008). The use of written feedback summaries was highlighted, the students reporting that the format was accessible and easy to apply to their own work. Perhaps more important were the built-in opportunities for follow up. The tasks enabled students to make mistakes in a non-assessed environment, with an opportunity to rectify these mistakes in later work (the ENV200 report).

“we went through … examples … of what they’d done well and done badly and then you could sort of, mark it against your one, so if you’d made the same mistake you’d write ‘oh, I did that wrong’ do it better like this”

“it’s easier to do something the second time round so I would say I’d been taught something, like how to put figures and tables into my report; I might do it the first time and get one or two things wrong but then the group feedback highlights those things and the second time round you know how to do it a lot better”

“… it’s not just the work that we did, we … processed the actual feedback in a way cos it was like a product … of what we’d done and like group discussions … so it sticks in your mind …”

The students made frequent references to “doing things differently the next time”. It was recognised that the focus of ENV201 (skills) lends itself to application across the curriculum. The students also noted that the fact that “the second year counts” towards their degree makes them much more interested in improving their work. Although the self-evaluation exercises were low on the students’ list of feedback methods, the students talked enthusiastically and in detail about their experiences of group feedback. The students valued the opportunities for dialogue during the module, talking at length about the different ways in which they could ask questions and obtain responses, notably via the VLE.

“There’s still … reluctance to actually put your hand up and say something (in a lecture). I think its just a little bit safer to put it on the message board …”

Several reasons were identified for this success:

• Investment of staff time (questions were answered in a reasonable time frame);

• Tutors only answered questions posed on the VLE discussion board. Students asking questions by email were asked to post them on the VLE to get an answer;

• Students answered questions, encouraged by tutors only responding to the VLE during working hours and adding supportive posts (e.g. “John’s answer was correct”, “Lucy has part of the answer, but you need to think about this as well.”);

• The VLE was used for several other aspects of the module (e.g. for downloading spreadsheets for the practicals).

The students appreciated several facets of the VLE: it was available when they needed it; it acted as an archive that could be returned to (unlike verbal conversations); and everyone could benefit from the responses not just the person asking the questions.

“… if you were stuck on something you would go on there (the VLE) and someone else had asked that question already and you could … look at the conversation they’d had and find the answer.”

The Post-it note exercise always generated many more questions than appeared on the VLE, because they were immediate, questions were asked in class while fresh in the mind, and fully anonymous. Although VLE posts can be anonymous to fellow students, the tutor can always identify the poster. Staff grouped questions by theme and responded to them on the VLE. The students found this processing useful, as it often clarified the question.

Conclusions and recommendations

In this module, group feedback, used actively by the tutor, proved to be a useful and effective method for developing independent learning skills of the students, with advantages for both the tutor and students. The students actively engaged with formative work, because they were able to see the benefit for themselves.

From a tutor perspective, generating group feedback took approximately 3 hours each week. Individual feedback could not have been provided in the same time and it was a much more positive experience for the tutor. Providing individual feedback can be repetitive and reinforce negative views of the students’ work, whereas looking for patterns is inherently more interesting. By standing back, the tutor can gain a greater sense of what has worked in the teaching and use the observations to review the curriculum.

For the students, the group feedback was not only as good as individual feedback, but had additional benefits. Receiving individual written feedback can be a passive experience, but the activities discussed here resulted in engagement with the comments. It was useful for the class to see comments that did not directly pertain to their own work. The students valued being able to position themselves relative to the class, and felt reassured that they were not alone in finding some tasks difficult, building their confidence during the module.

The methods discussed here are time-consuming, relative to standard assessment methods. However, by devoting one module to the development of these skills and incorporating activities designed to promote engagement with feedback, the whole programme benefits from students who are more able to learn independently.

Acknowledgements

This work was funded by the GEES Small Scale Project Fund.