Evaluating feedback mechanisms in the School of Earth and Environment

Abstract

Assessment feedback plays a crucial role in the learning experience. However, recent National Student Survey (NSS) results have shown there to be a large degree of student dissatisfaction regarding assessment feedback right across the Higher Education sector in the UK. Evaluating feedback mechanisms in the School of Earth and Environment was a project at the University of Leeds, designed to evaluate the provision of assessment feedback through in-depth consultation with staff and students about current practices, with a view to building upon good practice and creating a strategy for further improvement. Data gathered via focus groups and questionnaires was distilled into a key set of recommendations for ways forward. The recommendations focused on two main strategies: a) the creation of a new comprehensive feedback code of practice for staff, and b) raising student awareness of their rights regarding feedback provision as well as their responsibility to engage with feedback and how to get the most from it. These initiatives have been supported by a distinct, highly visual and sustained advertising campaign designed to act as a corollary to raising staff and student awareness of assessment feedback.

Introduction

Assessment feedback is currently a major topic concerning learning and teaching in Higher Education in the UK. Recently, the National Union of Students (NUS) launched a campaign entitled “The Great NUS Feedback Amnesty” which aims to highlight sector-wide student dissatisfaction with assessment feedback provision. The initiative’s goals are clearly set out in a briefing paper (NUS, 2009), which is prefaced with the NUS’s list of “Ten Principles of Good Feedback Practice” and contains a fairly comprehensive look at assessment and feedback issues from the student perspective. In addition to this, the NUS have recently launched a series of Higher Education newsletters entitled HE Focus, and the first volume is dedicated to the Feedback Amnesty initiative, containing several case studies detailing recent feedback initiatives from universities. There is also a short article promoting the assessment for learning (AfL) concept as an effective mechanism for designing assessment and providing feedback (Sambell and Graham, 2009). The concept aims primarily to maximise opportunities for learning formatively and gaining instant feedback, rather than placing importance on summative assessment where the end-point is a measurement rather than a learning opportunity. This idea has been championed by many workers in learning and teaching (Gibbs and Simpson, 2004; Juwah et al., 2004; Smith and Gorard, 2005; Brown, 2006). The role formative assessment plays in the learning process is often neglected, and shouldn’t be, considering the dynamic nature of the feedback that accompanies it. Nicol and Macfarlane-Dick (2004) have written an article which provides a theoretical model for implementation of effective formative assessment and associated principles of good feedback practice.

Phil Race has written papers and books which deal with learning and teaching and have excellent advice about how feedback is an integral part of the learning process (Race, 2006; Race and Pickford, 2007; Race, 2010). His website (http://phil-race.co.uk) also has many resources available to download including a useful compendium on assessment and feedback.

David Nicol (2008), of Strathclyde University, has outlined strategies for transforming assessment and feedback practice to empower first year students. The difference between leaving high school or sixth-form college and entering university is a large one, and students may have great difficulty in marrying their school experiences with the university learning and teaching environment. Nicol has suggested first year students may need more guidance regarding how best to take advantage of feedback, and in order to foster this, they need to be empowered and encouraged to engage more with formative assessment and feedback.

Engaging Students with Assessment Feedback is a project which was designed by the Higher Education Academy (HEA) under its Fund for the Development of Teaching and Learning (FDTL) initiative. It involved collaboration with a number of Higher Education institutions in the UK, and the final report was compiled by key collaborators from Oxford Brookes University (Handley et al., 2008). This project tackles a very key issue regarding assessment feedback: how to get students to engage effectively with feedback. It reports that the provision of feedback may be of a high standard, but if students do not recognize feedback or if they ignore it, then the efforts are wasted. It is vital to find methods of providing assessment and feedback that naturally foster student engagement.

Finally, Hughes and Boyle (2005) have written a very useful guide on assessment for the HEA’s Subject Centre for Geography, Earth and Environmental (GEES), now available for download from the HEA’s website (www.heacademy.ac.uk/resources/detail/resource_database/SNAS/Assessment_in_Earth_Sciences_Environmental_Sciences_and_Environmental_Studies).

Methodology

This paper outlines a University Teaching Fellowship (UTF) project in the School of Earth and Environment at the University of Leeds, designed to evaluate and improve current feedback provision.

The main method for gaining data about feedback policy and practice in the School was via discussion with students and staff. Initial informal discussions, in conjunction with a literature review on feedback, helped to inform the preparation of an agenda to be used for more detailed focus group sessions. Themes explored were:

• Current school policy

• Feedback as related to assessment aims/objectives

• Clarity of marking criteria

• Formative vs. summative assessment and resulting feedback

• Engaging with feedback (staff and student engagement issues)

• Mechanisms of feedback (written, verbal, electronic, group-based, individual)

• Innovative feedback mechanisms and suggestions for moving forward

Focus groups for staff and students were held separately. All teaching staff were invited to join and 23 people participated over three sessions. All formally appointed student representatives from each programme participated in the focus groups, and they gathered opinions from their peers prior to the meetings. All students from all years and programmes were invited to join the focus groups, and 87 did over three sessions. This approach was chosen as the most appropriate for obtaining as much data as possible. Although simple questionnaires employing closed questions, and matching agreement/disagreement to statements may have allowed for easier manipulation of data, the results would have been rather one dimensional and naturally led by the composition of the questions. The focus group sessions were recorded to ensure complete capture of the data. In order to bolster the data from students, questionnaires were used employing the same questions as in the focus groups. One hundred and sixty three questionnaires were returned.

Key recommendations and discussion

Our study’s findings are presented as key recommendations, split into short term and long term goals.

Short term goals:

Raise student awareness of the role feedback plays in their learning, and prime them to receive and engage with the feedback accordingly

Many students have a very poor idea of what they should expect in terms of feedback; similarly, many students do not realise the role they should play in engaging with feedback. Staff noted that students were focused on marks and tended not to engage much with feedback. In order for feedback to work, the students need to know in detail what to expect from staff and in turn, what is expected from them. Students are not often aware of the many different forms of feedback, and hence do not always realise when they are getting feedback and don’t engage with it. Furthermore, the amount and style of feedback students receive at university compared to school can be quite different, so managing expectations is important.

The issue of gaining skills and knowledge vs. bagging marks must be tackled; students should be at university for an education not just a qualification. Promoting a mature attitude towards formative assessment is vital. Raising awareness of feedback primes the students for receiving feedback and knowing how best to use it. It would be beneficial to speak to students during induction about taking responsibility and ownership of their learning, and how that role is different from their experience in college or school.

Create a thorough “Code of Practice for Feedback”

The current School policy was too simplistic, comprising of a “three term-time week” turnaround (meaning three weeks of term-allocated time, not including holidays or semester breaks), with a proviso that the feedback must be “useful”. There is much scope for interpretation regarding what constitutes “useful” feedback. What staff find useful may not equate with what students find useful. There needs to be guidelines setting out the minimum requirements for provision of feedback, frequency with relation to assessment, and an expectation that staff clearly communicate their chosen mechanisms and style of feedback provision with their students. This enables the students to know what to expect and further helps them to engage with feedback when they do get it.

Students would like to get feedback in a timely fashion, so they may use it to improve future work: however, they don’t mind waiting longer than three weeks provided they know beforehand exactly how long they must wait and that they don’t have to wait an unreasonable amount of time.

Where possible, make sure there are clear and concise aims and objectives for each set assessment and that marking criteria and feedback is clearly linked to these

Students are more confident in dealing with assessment when there are clear aims/objectives and where the feedback is related to a concise set of marking criteria. Staff who are engaged with providing good feedback often create a marking pro forma or feedback sheet which is tailored to the assessment aims/objectives and clearly indicates marking criteria, and those that do so get very positive comments from the students regarding their efforts. Generic feedback sheets exist for staff to use as a bare minimum, however, students reported an intense dislike for these as they often do not meet their needs for more subject-specific/task-specific feedback.

Encourage staff, where possible, to adopt mechanisms of feedback that students prefer

The most preferred mechanism for receiving feedback is written feedback on the actual assignment/assessment. This could be either handwritten or done electronically, and could be accompanied by a generic/tailor-made feedback sheet. Almost all students stated that, one-to-one verbal feedback aside, this type of written feedback is the most useful and the easiest to engage with, and they would like to see much more of it. However, it was noted that due to large class sizes, or issues related to retention of all marked work, it was not always possible to provide feedback as annotations on the work itself.

Strategies to enable work-annotated feedback include: providing dedicated feedback sessions where students can see their annotated work but it has to be handed back at the end of the session, and employing electronic submission and feedback.

Long term goals:

Aim to design courses/modules around assessment to make the assessment more formative

Current thinking with regards to assessment suggests the more formative it is, the better. Often, courses are designed firstly with content in mind and assessment is added later, with little thought as to why certain content is taught or if the methods of assessment are suitable. Skills building and assessment of deep learning are more difficult to achieve than simply testing students on facts regarding course content. Formative assessment, strategically planned so that progressive tasks build upon each other, favours the effective building of skills and promotes deeper learning whereby students automatically reflect on their previous work to inform them on how to progress with newer tasks (Brown, 2004; Gibbs and Simpson, 2004; Juwah et al. 2004; Smith and Gorard, 2005).

Improve links between modules, with a view to making assessment, and provision of feedback, more effective and harmonious

Students often commented on the huge demands they face from having many modules which have multiple assessments, hence the demand for provision of feedback can be very high. Staff suggested that larger and more coherent modules would allow for better designed, more formative assessment and promote deeper learning via cross-linking between subject areas.

Implementation

To deliver these goals, feedback was chosen as the School’s major learning and teaching theme for the 2010–11 academic session. Raising the profile of assessment feedback and improving its practice will be achieved through the implementation of several key strategies:

• Creation of a complete and comprehensive feedback code of practice for staff, taking into account the project’s recommendations.

• Distribution of a Feedback Handbook for students, with the aim of priming them about all forms of feedback, the role it plays in their learning and how to get the best from it. The handbook also serves to remind students of their responsibility to engage with feedback and to manage their expectations of assessment feedback at university.

• Creation of a distinct identity for the feedback theme, based around a logo with characteristic colours, to be employed in a highly visual campaign aimed to raise awareness and advertise feedback related events (Figure 1).

• Ask staff to:

  • Provide clear aim/objectives for each assessment

  • Provide concise marking criteria

  • Create feedback sheets and marking pro forma that are tailored to the assessment

• Ask students to:

  • Anticipate and actively look for feedback

  • Engage fully with all feedback they receive

  • If they don’t understand feedback, feel they need clarification or want more, to ask for help

• Display posters in key locations of the School alerting staff and students to the new feedback-related initiatives (Figure 1).

• Encourage staff to share good practice and provide an online resource to facilitate it.

• Plan a sustained calendar of feedback-related events including:

  • Mid semester one feedback surgery sessions

  • Early semester two personal tutorials on feedback

  • 1-to-1 exam feedback sessions for all finalists

  • Pre-NSS feedback meetings with Director of Student Education and key teaching staff

  • “Teaching Stars” awards, voted for by students

  • Semester two pre-exam feedback surgery sessions

Figure 1: Feedback GEES paper.

Results

The National Student Survey results for the 2010–2011 year saw a significant increase in student satisfaction for assessment and feedback within the School of Earth and Environment, jumping 13 points from 55 to 68. This is ahead of the University of Leeds average of 65. It is clear that the initiatives are working, although there is much more that can be done.

Next steps

The next phase of the project will involve scrutinising the efficacy of the new feedback initiatives and building upon successes. Focus group sessions with staff and students to be held at the end of the current academic session will provide a forum for gathering opinions on the strategies implemented. Extra support for staff in the form of resources and guidance will be created, with a focus on illustrating examples of good and bad practice and creating opportunities to improve feedback through the use of audio and visual technology.

In a bid to provide more high-quality, written feedback on students’ work, the School of Earth and Environment has devised and is piloting a project for the University called “Return Not Retain”. Current University policy is to retain all marked work for auditing purposes, which makes returning work annotated with feedback to students quite difficult or short-lived, limiting active reflection. Students now have work portfolios where they will collate assessments and feedback promoting active reflection. These are to be returned to the School at the end of each year for external examination purposes. The project will be evaluated internally within the School and externally via University management at the end of the 2011–2012 session.

Collaboration with other feedback-related projects at the University of Leeds is underway, with the aim of creating a University-wide strategy for improving assessment feedback.