Developing a learning-centred framework for feedback literacy

Abstract

There is an increasing focus on notions of feedback in which students are positioned as active players rather than recipients of information. These discussions have been either conceptual in character or have an empirical focus on designs to support learners in feedback processes. There has been little emphasis on learners’ perspectives on, and experiences of, the role they play in such processes and what they need in order to benefit from feedback. This study therefore seeks to identify the characteristics of feedback literacy – that is, how students understand and can utilise feedback for their own learning – by analysing students’ views of feedback processes drawing on a substantial data set derived from a study of feedback in two large universities. The analysis revealed seven groupings of learner feedback literacy, including understanding feedback purposes and roles, seeking information, making judgements about work quality, working with emotions, and processing and using information for the benefit of their future work (31 categories in total). By identifying these realised components of feedback literacy, in the form of illustrative examples, the emergent set of competencies can enable investigations of the development of feedback literacy and improve feedback designs in courses through alignment to these standards.

Keywords:

• Feedback
• learner agency
• feedback literacy

Introduction

Concern about feedback is prominent in current higher education scholarship. There is a considerable literature which explores why students commonly report more inadequacies about assessment and feedback than any other feature of their courses. Various routes to explain and explore these phenomena have been taken. These include a focus on improving the nature of comments provided to students (Hattie and Timperley 2007; Dunworth and Sanchez 2016), a focus on the mode of delivery of feedback information (Bennett et al. 2017; Ryan, Henderson, and Phillips 2019; Mahoney, Macfarlane, and Ajjawi 2019), and an emphasis on feedback dialogue between students and educators (Carless 2006; McLean, Bond, and Nicholson 2015; Winstone et al. 2017). In particular, there has been a shift away from an exclusive focus on what teachers do to initiate feedback (inputs), to what students do and how they might be more actively involved in feedback processes (Boud and Molloy 2013).

However, all of these approaches presuppose that students and educators are referring to the same object when considering what feedback is. There is also an assumption that educators embed effective design of feedback processes in their courses, and that students can effectively utilise feedback information to improve the quality of their learning. There is reason to question each of these assumptions. For example, it has been documented that students and teachers have different views about what feedback refers to (Carless 2006; Adcroft 2011; Dawson et al. 2019), and that feedback designs in higher education are wanting (Esterhazy and Damşa 2019). There is also a growing recognition that while teachers’ designs are important, feedback needs necessarily to be a learning-centred process, and as such, it is the students’ ability to effectively engage with and utilise feedback processes that needs to be given more attention. It is on the latter point that this paper focuses.

This paper is concerned with feedback literacy of students, that is, students’ ability to understand, utilise and benefit from feedback processes. It builds on the notion of feedback literacy articulated by Carless and Boud (2018), which in turn was stimulated by Sutton (2012) and earlier ideas of assessment literacy (eg. Price et al. 2012; Smith et al. 2013; Lees and Anderson 2015). A clear analysis of what student feedback literacy might encompass would in turn enable educators to develop their own feedback literacy and thus create ways in which feedback processes might be used effectively.

The paper starts by identifying the shift in conceptions of feedback in recent years and positions the development of learner feedback literacy as a key mechanism for maximising the potential of feedback processes. It then draws iteratively on student data from a large Australian study of feedback to map features of feedback literacy to lead to a discussion of the implications of this analysis for how feedback literacy might be promoted. To date, feedback literacy has been written about in conceptual terms only (Sutton 2012; Carless and Boud 2018). This study progresses beyond that of Carless and Boud (2018) by seeking and analysing the student perspective in feedback processes, prompted by empirical data.

The research question addressed was: What capabilities do students need to demonstrate in order to be feedback literate?

Shifting feedback to a learning-centred process

Educators and students typically enact feedback as if it were solely an input mechanism to students. The common phrase ‘I was giving feedback on students’ work’ betrays such an assumption. Dawson et al. (2019) identified that students predominantly hold this teacher-orientated receipt of information view, but that educators are somewhat more likely to consider the importance of designing activities to enable learners to take up and use feedback to improve their subsequent work. However, this espoused belief appears at odds with often cited enactment of practices and consequently merely highlights the need for mechanisms to pursue a more student-centred view.

With a teacher-driven model of ‘feedback as telling’ we limit feedback as merely an input, which at best, is hopefully useful information for the learner (Sadler 2010). Students must ‘pick up’ their notions of feedback and role expectations within the process somewhere. From a timing point of view, the very fact that most ‘feedback information’ is generated at the end of a sequence of learning, without time or opportunity to use the information to improve performance on related tasks (Dawson et al. 2019), might help to create, or reinforce the conception that feedback is normally a teacher-generated input. A recent discussion of feedback challenges this conception, and orientates it as a process that makes a difference to learning: ‘Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work’ (Boud and Molloy 2013, 6). Implicit in this definition is that learners must take an active role to seek information, make sense of it and undertake subsequent tasks, to enable translation of newly constructed knowledge into practice.

As one of the many recent moves to locate learners and learning as the object of feedback, Carless and Boud posited that student feedback literacy ‘denotes the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies’ (2018, 1315). Based on a synthesis of the broader feedback literature, they presented four key features of student feedback literacy: appreciating feedback; making judgments; managing affect; and taking action. We do not however have enough understanding of how learners themselves view and enact these capabilities, or whether there are other capabilities that have not been accounted for in these features. In this paper, we take a learning-oriented view of feedback in which students are positioned as active participants in the feedback process and that the generation of effects on their learning is a necessary part of it. We sought information from students to help identify what is needed in a framework of feedback literacy capabilities.

Method

This study set out to explore the notion of feedback literacy, primarily through an investigation of how students describe their practices relating to feedback events that they deemed to be successful. As learners are not likely to be familiar with the term or notion of ‘feedback literacy’, the approach taken was to utilise data from a large empirical set of student views of feedback and what they regard as beneficial feedback practices. Through thematic analysis, student strategies, capabilities and attitudes that were reported to support feedback were identified.

The data used in this paper were drawn from the first two stages of an 18-month project investigating what makes for effective feedback involving two large Australian universities. The first stage involved identifying feedback practices and experiences through a large-scale survey of 4514 students and five focus groups with 28 students (see Dawson et al. (2019) for the survey design and recruitment procedures, including demographics of participants). The full survey instrument can be viewed at feedbackforlearning.org/feedback/Publications resources. The second stage, focussed on seven case studies of nominated effective feedback, in which a total of 20 students were interviewed about their experience of feedback in each case, and how this experience compared with their engagement with feedback more generally. Approval was received from the Human Research Ethics Committees of both universities prior to all data collection.

The open-ended survey data were thematically analysed (Braun and Clarke 2006) using an initial a priori coding framework. The starting framework was based on the feedback literacy characteristics proposed by Carless and Boud (2018) but then extended and adapted through a grounded, constant comparative method (Braun and Clarke, 2006). The goal was not to establish the extent of representation of items in the data, but rather to ascertain their presence.

In doing so, the authors read subsets of the data, discussed the codes, consulted with an expert colleague, further developed the coding structure, and then iteratively continued with this process with the student focus group and student interview data until theoretical saturation was deemed to have been achieved. Following the analysis, categories were translated into statements in a form that represented the capabilities implied in them (see Table 1 with illustrative quotes from the student focus group and interview dataset). The final 31 characteristics were thematically organised into 7 groups for convenience of discussion.

Table 1. A learning-centred framework for feedback literacy.

A learner exhibiting well developed feedback literacy
Feedback Literacy Category	Empirical representation
Group 1: Commits to feedback as improvement
1. Establishes a disposition to use feedback to continually improve their work	‘So anytime that there is actual feedback, I tend to take it on board. So, it is not like - I don’t say, “Oh I’m going to change my behaviour because this one comment hit me hard somehow”. It is more, “Okay, so obviously I’ve got something here that is deficient. I need to remedy that and then I’ll do it”’. UG_STEM
2. Acknowledges that mastery/expertise is not fixed, but can change over time and context	‘I always like to improve. Sometimes when you look back and see more of an improvement than just grades and whatever, it’s a bit more meaningful. And at the end of my course, I know I need to come out with a certain set of skills, not so much terrific grades. My grades are good because I try to improve myself. But also, my skills are improving as well, which is I think really important. And I think a lot of students overlook that a lot’. PG_Health
Group 2: Appreciates feedback as an active process
3. Acknowledges the role of feedback processes in improving work and refining judgements and learning strategies	‘when the feedback is written in a way that you can understand, and you can accurately improve upon what you’ve written, that’s the sort of feedback that makes it [effective]’. UG_STEM
4. Recognises that effective learners are active in identifying their own learning needs	‘I understood here it’s more about you putting in effort to learn and not depending completely on your lecturer. Because I think where I’ve come from and where I’ve studied, a lot of stuff are given to you, I you know – like, okay read this, read that; do this, do that – but here it’s not that way’. PG_non-STEM
5. Anticipates their own learning needs and communicates these to appropriate others	‘I normally converse via email or after class, during class, about the direction of my assignment and possibly show a paragraph or a short sentence perhaps I’m concerned by, or ask for some help in finding additional readings and that kind of thing. Sometimes it works, sometimes it doesn’t’. PG_non-STEM
6. Understands the role of standards and criteria in judging the work of oneself and others	‘If you familiarise yourself with the rubric and then you know actually I have a baseline for a stand here, and then you’re looking at someone else’s work, less as a passive viewer and more actively, that can be a good learning experience. But again, the implementation needs to be done correctly, otherwise it is just a free for all’. UG_STEM
7. Identifies that they need to complete a feedback loop for information provided by others to be effective	“I think it’s helpful when the first assessment task kind of helps with the second one. …I had a lab report in our first assignment was to just write the introduction, and submit that. And we got feedback for that. And then the last assignment was to submit the whole lab report. So, you actually had the chance to include the feedback and, like, my comments had noted that they could see I had taken the feedback and applied it, which was good to see that that works.” PG_Health
8. Recognises that feedback should build capacity to develop their own evaluative judgment over time and over different learning outcomes	‘When you’re doing this particular assessment, because it is worked as one assessment task broken down to smaller sections, as you go through each one you can see week by week if the rubric is the same, whether or not you’re actually improving. So, you see, “Actually I did do this this week, because I know, okay this is what you expect, and this is how I’m meant to do it.” And then if you see an improvement in your marks and the rubric hasn’t so much changed, it is just a literature item, then you can actually see, yes there is a change my approach as it were’. UG_STEM
Group 3: Elicits information to improve learning
9. Realises that feedback requires active elicitation and does not wait for others to provide unsolicited information	‘In general, well every unit that I’ve had, they would give us a consultation time. So that would generally be when I’d flick them an email and say, “Oh I want feedback on this, or I need a bit of help with this particular part. Can I see you at this time?”’ UG_STEM
10. Uses a wide repertoire of strategies to elicit appropriate information from others to assist learning	‘What helps me improve is definitely that feedback before you even need it, the instructions, so you don’t go and constantly and be like - and I can imagine for a unit chair having 20 people constantly being like, “How do I do this?” is not going to be fun for them’. UG_STEM
11. Considers feedback from multiple sources – eg. teachers, trainers, peers, practitioners, consumers – to provide a different scope and opportunities for learning	‘I always seek feedback from my partner, who’s an engineer, because he’s very concise with what he says, whereas I ramble on in my sentences and that kind of thing. So that’s quite helpful to have someone different. I also use the research and learning team and the people up in the library who are there during the middle of the day to get some help’. PG_non-STEM
12. Recognises that different stakeholders may have different perspectives, experience and levels of investment in the process	‘If they say, “Oh, very specific argument, very clear,” but we all have our different ways of understanding what that means, to be clear. So, I guess that’s with the faculty with marking. It’s also subjective’. PG_non-STEM
13. Engages in dialogue to elicit useful information about standards, criteria and the nature of good work	‘Yeah if during the course of an assignment or something like that I’ve discussed it with a colleague, generally once the assignments come back and when I’ve got some feedback, I generally take it back to them so that we can talk through and workshop a little bit’. PG_STEM
14. Seeks out exemplars as a way to make sense of standards of work	‘I ended up researching those run-on sentences on YouTube and then they came up with examples on YouTube, just short videos of how, what to do better, or what to change. Whereas when they say, “Work on your grammar,” well I don’t know what that means’. UG_STEM
15. Seeks cues from the environment and the task itself that indicate the appropriateness of work	‘I think that’s the difference too, as you were saying, from subject and field, different assessors, different people, different requirements, and sometimes it’s the first assignment that you’ve got to try and navigate to what their requirements are before you can get to the final assignment hopefully to pick you up to get you to passing your degree or to get to the next level of the PhD or whatever it is you want to do. Navigating that minefield’. UG_non-STEM
Group 4: Processes feedback information
16. Identifies and utilises standards, criteria and exemplars	‘I was very happy with the unit because we got constant feedback and also sample answers like it contained what the tutors were expecting from us, like kind of an answer they were expecting. So, apart from feedback, I think it’s always better to have something in hand to look at to improve on it, but they also help us improve by looking at the sample’. UG_non-STEM
17. Recognises and interprets language peculiar to education containing important cues about the task or related outcomes	‘So, my first point of call is my assignment rubric, and then I can work out from that how much space or how many words I need for each thing. And also incorporated in the rubric will be particular words like collaborate or something, and you make sure you force them into your document so then they can see, “Oh, well, they match up.” And that shows then that the marker knows that you’ve read the rubric and know what they want, and that kind of thing. And if you don’t have that to a high standard, then you can’t put that into your own work’. PG_Health
18. Selectively accepts and rejects views of others in coming to their own appraisals	‘I have a particular writing style where I use complex sentence structure and I realised that not everybody is familiar with correct grammar, because there’s a whole generation that was not taught grammar skills. My piece, it made no sense, so I was marked high in the conceptual area and low in my language use. It doesn’t make sense; how can you convey something conceptually and have poor writing skills? So, my next piece I actually dumbed down my writing skills and just used simple sentence structure and I scored higher’. UG_non-STEM
19. Extracts key actionable information from others, which may require prompting for more detail or clarity	‘I did this professional writing unit, which was an elective as well, and I got my first ever research essay, I had never done a research essay before, so I had no idea how to get started. So, I did actually approach the tutor to get the exact answer, but I needed guidelines for how to get started, how to reference correctly and stuff. He was very helpful; he didn’t refuse to give any guidance he had. He recommended some materials, online materials and journals, for me to refer. He didn’t give any specific answers to the task, but he gave other resources for me to refer to, and how to get some knowledge about research writing’. UG_non-STEM
Group 5: Acknowledges and works with emotions
20. Demonstrates volition and sensitivity in approaching suitable others to elicit suggestions and to continue dialogue with them as needed	‘If you want to do well you’ve got to be pretty driven and go and catch that tutor during consultation time and really, not hassle them but, well you do kind of, you have to make a time with them, you’ve got to go and see them and ask them’. PG_STEM
21. Demonstrates openness to receiving comments from others without displaying defensiveness	‘I’ve sought feedback after class just to clarify ‘oh, I thought this was that’, or ‘I’m a bit confused about why you marked this wrong’ and every time I’ve asked a question like that, they will go through exactly why what I did was wrong, and usually they will also ask ‘did you understand that’ or ‘does it make sense now?’ and if I still don’t understand, they’ll change the way they’ve said it to a different way, which is really helpful. They are really taking the time to make sure you understand all your feedback so you can really improve, which shows they care, which is also another motivator for me. Someone cares about what I am doing’. UG_STEM
22. Builds trust in facilitating honest and meaningful information exchanges with others	‘When it comes to comments, what sort of comments are you after? UG1: I’d say honest ones. Respectful’. PG_Health
23. Recognises that feedback information comes in different modes with different capacities to mobilise emotions, eg. individual and group, written and through various other media, structured and informal	‘I think. as well, I’ve experienced in a few units people trying to give feedback in different ways. My tutor didn’t opt to do this, but it was an option. Some gave video feedback, as well as a little written feedback. I think that specifically in the education faculty they are trying to diversify the ways that they give feedback, which is really positive’. PG_non-STEM
24. Manages the emotional challenges of receiving and sifting information which may be unwelcome or misjudged	‘I thought, ‘Well I’ve got an emotional view on this’. Let’s pass this to somebody else, somebody who has not got any interest in this, do this, can you figure it out. “Oh, I can’t figure it out either.” So, it is not just me having such a prejudiced viewpoint that I can’t see the truth’. UG_STEM
25. Considers the influence of high stakes assessment on the way learners might engage in candid dialogue about their own performance, eg. declaring their own deficiencies in performance may impact on grades, or desire to score well may reduce learners’ receptivity to feedback information	‘I’m quite happy to be marked down as long as, like you said, the justification is there, and you’re told how you need to improve’. UG_non-STEM
Group 6. Acknowledges feedback as a reciprocal process
26. Recognises that they have roles as both user and provider of information and that skill in one role helps in the other	‘As part of our Education units, we had one where we do an assessment of a peer…., so basically you had a copy of the rubric in front of you. The lecturer would give her particular feedback and then two students were selected to review. See that actually became a good learning experience because if you familiarise yourself with the rubric and then you know actually I have a baseline for a stand here, and then you’re looking at someone else’s work, less as a passive viewer and more actively, that can be a good learning experience’. PG_Health
27. Composes useful information for others about the nature of their work	‘We did a PowerPoint presentation and then she [the teacher] would grade you on your presentation, but then you’d all have to log onto this online site and give feedback. And so that way - she’d give you feedback on the presentation and she would also give you the feedback that your team mates wrote, because it had a little box and they had to write something at the end. One I wrote to one of the guys was like, “Awesome job getting over your nerves. You nailed it on the day, well done.” So, it was friendly and stuff, but it was a different type of feedback that can be really useful’. UG_STEM
28. Exhibits cultural sensitivity through not assuming that others are likely to react in the same way as oneself in receiving and responding to information	‘If that was a student who was kind of struggling and that was their actual work, I don’t know why I would even continue at uni if that was the kind of feedback I was getting. It is tertiary education, but they still have people who struggle with simple things even. So just be nice’. PG_Health
Group 7: Enacts outcomes of processing of feedback information
29. Responds to feedback information from others through goal-setting and planning how it might be utilised in future work	‘Just before working on the next assignment, say, I’d go back to the feedback to see what about my writing style mostly can I improve on? Like when you say this argument you have to support it better or stuff like that. That’s how I use it for the next assignment’. PG_non-STEM
30. Analyses and records information in appropriate forms for the purposes of acting on it subsequently	‘I quite like the audio feedback too. It kind of forces me to write out everything I need to work on for my next assignment. So, I’m kind of forced to sit and listen to it all, when sometimes you can just kind of skim through written feedback and throw it away’. PG_Health
31. Monitors their own progress to discern where feedback might be helpful and to influence the setting of new learning goals	‘When you’re doing this particular assessment, because it is worked as one assessment task broken down to smaller sections, as you go through each one you can see week by week if the rubric is the same, whether or not you’re actually improving’. UG_STEM

NB Table 1 Key for illustrative quotes from student focus group and interview data. UG = Undergraduate courses. PG = Postgraduate courses. STEM = Science Technology Engineering and Mathematics. Non-STEM = Courses outside Science Technology Engineering and Mathematics. Health = programmes within the health professions.

Results

The resultant Student Feedback Literacy Framework (see Table 1) comprises the following:

• Group 1: Commits to feedback as improvement. Categories 1-2

• Group 2: Appreciates feedback as an active process. Categories 3-8

• Group 3: Elicits information to improve learning. Categories 9-15

• Group 4: Processes feedback information. Categories 16-19

• Group 5: Acknowledges and works with emotions. Categories 20-25

• Group 6. Acknowledges feedback as a reciprocal process. Categories 26-28

• Group 7: Enacts outcomes of processing of feedback information. Categories 29-31

Discussion

All of the 31 categories were represented within the student data set. They provide us with an elaboration of the forms of knowledge, capabilities and skills that a feedback literate student might be expected to possess and enact. The researchers had anticipated that some of the items on the initial coding framework may have remained aspirational – that is, that there may not have been evidence within the student data, however this was not the case. The groupings are discussed below with reference to the broader literature on feedback in higher education. Implications for pedagogical designs, and future research designs, are then outlined.

Commits to feedback as improvement

The first two categories reinforce the value of learners understanding the purpose of feedback as a process fundamentally orientated to learner improvement. Such explicit learner orientation to purpose has been argued by Boud and Molloy (2013) through the conceptual model of Feedback Mark 2, along with Winstone et al. (2017) in their review of active learner feedback recipience. Learners see feedback as a process they will use beyond their university life, and are challenged to acknowledge that expertise is not a fixed construct, but rather is context-dependant and continually evolving. By extension, if practice is evolving, then the need for feedback remains a constant, rather than serving as an artefact of university life.

Appreciates feedback as an active process

The second group of categories focus on the role that students take in the process. Without an understanding of what feedback is, how it works, and their own active role, it is unlikely students will be able to move beyond seeing feedback as an input. Students in both the survey and in-depth interviews expressed that when feedback worked for them, they were called to action in some way. In the examples provided in Table 1, the learner proactively went to different sources to build a picture of their performance. Others also described introducing their own, ‘early’ feedback loop, by seeking advice from peers and making refinements before submitting assignments. What were less visible in our data were examples of learners chasing their own opportunities to try new tasks (as a way to translate new knowledge into practice). Seeking performance-rich information was seen as part of their role, but seeking practice opportunities was not reported, and perhaps viewed as a job of teachers or unit co-ordinators.

Elicits information to improve learning

The categories in this group relate to learners seeking information to help their sense-making, and subsequent performance. Learners stated that not only was it important that they elicit feedback information from others but that they requested information relating to specific aspects of their work – ‘Oh I want feedback on this or I need a bit of help with this particular part’. This finding reflects results from a study of student feedback literacy in health professions education, where students commented on the revelation that ‘feedback was theirs for the taking’ and that their requests for focussed information in turn gave them more meaningful information they could pick up and use (Noble et al. 2019b).

Processes feedback information

This group of categories focuses on how students can operationalise their understandings of the purpose and process of feedback. It is not enough for them to know the importance of being an active player, they need the dispositions and capabilities to put these into practice and an awareness of the complexity of putting these skills together in context. Of note is the category relating to identifying standards or targets for ‘good work’ in order to make judgements about the quality of their work, which aligns closely to Tai et al.’s (2018) notion of developing evaluative judgement. This finding relating to understanding standards was also a feature of Johnson and Molloy’s (2018) study where productive feedback conversations were geared towards ‘what good practice would look like’ rather than a teacher-led monologue about what ‘the student’s work looked like’. In our data, students described both an a priori seeking out of notions of good work to help them sense-make (for example project criteria, or exemplar essays), as well as post-submission dialogue with teachers about what good work should resemble.

Acknowledges and works with emotions

The important role of emotions in feedback, and the learner’s role in acknowledging and working with emotion, is represented by a discrete grouping within the framework. Typically, literature on feedback treats affect as an interference to the transmission of clear, rational messages to the learner (Rowe 2017). In our data, students did not seem to have enough language to convey their discomfort in trying to wrestle with information about their work which dismayed them. The literature suggests that learner investment in the work itself and perhaps investment in ‘being a good judge of self’ may be factors in causing discomfort when an external appraiser provides a challenge to the work itself and learners’ evaluative judgements that accompany its production (Molloy, Borrell-Carrió, & Epstein 2012; Rowe 2017).

Of note in the wider data set, students most often attributed the emotional palatability of the information exchange to characteristics of the teacher; i.e. the teacher made it personal, the teacher was kind, the teacher praised more than they criticised. There were few occasions where students reported feeling a sense of responsibility for managing the relational and emotional qualities of feedback interactions. Helping learners to recognise the role of affect in learning, and ways to manage affect to achieve productive learning outcomes may be a key direction for improvements in feedback.

Acknowledges feedback as a reciprocal process

Learners becoming comfortable in ‘provider’ and ‘receiver’ roles may be an under-recognised mechanism in positioning feedback as a process to enable learning, rather than as a set of processes imposed on learners. Being charged with responsibility to consider others’ work and make and communicate judgements, necessarily positions learners as active, and it may make learners more sensitive to the relational processes that underpin feedback exchanges between people (Molloy, Ajjawi, and Noble, 2019). Engagement in genuine dialogues may give learners a chance to experience first-hand that individuals interpret the same event or production of work differently, and that individuals respond differently to the same ‘feedback information’, even if it is designed to help improve their future work.

Enacts outcomes of processing of feedback information

The final group of categories is about students acting to make feedback work for themselves. It encompasses the need to process information from wherever information can be located as well as how they can utilise such information. This increasing focus in the literature on what students do with performance cues is reflected in recent qualitative studies by Noble et al (2019a) in the medical education context, and Esterhazy and Damşa (2019) in the higher education setting. Group 7 recognises that benefiting from feedback is not an activity that takes place at a single point in time, but requires planning and follow up. That is, students need to make the most of whatever agency they possess. This might be limited in some situations, but there is always more they can do than is encompassed by the formalities of the feedback designs provided for them.

What might a learning-centred feedback literacy framework enable?

The student feedback literacy framework could be used to plan feedback development strategies for courses, and prompt further research to identify whether these capabilities have been developed as a result of particular curriculum events. As yet, there has been minimal focus on learners’ perspectives on, and experiences of, the role they play in these processes. It was striking that despite considerable efforts to find new distinct items from the student data, how few additional items were identified over and above recent conceptually based literature on feedback in higher education. What is also striking is that learners in our study reported they are able to act as agents within feedback processes, which challenges common rhetoric that students are not capable of making judgements about their own work, or of seeing benefit in peer to peer feedback dialogues (Tai et al. 2016a)

This study progresses the work by Carless and Boud (2018) by incorporating a student perspective on what it means to engage in feedback that works. It provides elaboration of items identified and points to areas in which their work did not venture. In particular, the new framework enables us to articulate the role of learners in actively seeking information, making judgements themselves, recognising feedback as a reciprocal process, and using information for the benefit of their future work. The framework sets the ground for students to see what feedback competencies they need to develop and to monitor their progress towards these targets. For educators the framework may help to diagnose the quality of feedback interventions based on their ability to influence student achievements. If interventions fall short, is it insufficient understanding of feedback and how it works? Is it learner resistance to being pro-active in feedback processes? Or, is it an inability of learners to plan and act on the basis of information received?

The creation of such a framework has obvious implications for course design. For instance, it may provide insight into the ways feedback literacy might be developed throughout courses. At this point, we do not have evidence to suggest how difficult these capabilities may be to attain at the level required for any given course, which in turn would impact how activities are designed and progressively loaded throughout programs. Through this research examining student perspectives and experiences of developing feedback capacities, we have some sense of how they might be developed, but this requires a focussed inquiry on the impact of curricular design on students’ development of feedback literacy.

Anticipating challenges to promoting student feedback literacy

While we might hope that having identified these capabilities, it would be a straightforward matter of incorporating activities to promote feedback literacy into courses, the process of embedding it is likely to be complex as it challenges some taken-for-granted assumptions about feedback and how it operates. Through our multi-institutional study, two particular challenges became apparent:

Shifting teachers from ‘information providers’ to facilitators of learner feedback literacy

Much data, across the survey, focus group and interviews, suggested that students viewed the purpose of feedback as enabling them to improve their performance. However, they also saw that this mechanism of improvement was often contingent on what they received – the sophistication or pitch or detail of comments from the teacher – reinforcing the limited notion that feedback equates to teacher comments on work.

This view is compounded by the ways feedback is inscribed in common discourse, particularly in prominent evaluation surveys (for example, the UK National Student Survey; https://www.thestudentsurvey.com/content/NSS2017_Core_Questionnaire.pdf) which positions feedback exclusively as an attribute of good teaching without acknowledging the role of learners (e.g survey item 10 reads ‘Feedback on my work has been timely’; survey item 11 reads ‘I have received helpful comments on my work.’). The problem we need to collectively confront is that even with the professional development of university teachers, and an institutional culture that values facilitation of learning rather than ‘telling’, learners and teachers may still have an expectation that feedback is in the domain of teachers and is judged in terms of the information they generate. Learners are often complicit in generating feedback exchanges that are characterised by educator monologues, with learners reporting benefits in ‘waiting to receive feedback comments’ rather than taking risks in making evaluative judgements about their own work should these be at odds with the judgement of teachers (Molloy 2009; Noble et al. 2019a). If we are asking learners and teachers across classroom and workplace learning settings to adopt different approaches to feedback, then evidence for the benefits of doing so, and strategies for ‘how to do it’ (as per illustrated in Table 1) will be needed.

Appropriately embedding feedback literacy

A key challenge we anticipate in enacting programmes to better develop student feedback literacy is the fear of curriculum crowding. The development of feedback literacy needs not to be an add-on displacing other activities but can be an embedded strategy as part of existing activities. For example, by asking students what type of comments they want when submitting assignments, or expecting students to have plans for responding to and utilising feedback inputs.

Pedagogical strategies around feedback processes, particularly those that position students as active learners, need to be introduced early in the first year to enable students to benefit more from the curriculum and not leave them dependent on limited opportunities for input from educators (which of course, further reinforces the notion of feedback as teacher-generated information). Inviting students to engage in opportunities that allow them to trace, first hand, the benefits of proactively engaging in feedback processes has a reinforcing function, and this may reduce the temptation for students to feel that ‘feedback literacy’ is just another competency to be ticked off as part of the assessment game. One of the clear advantages of having feedback literate students is that they are not dependent on the necessarily limited opportunities (resources, time, class size) for input on their work from staff. If students see peer generated feedback as a process that augments teacher comments, rather than replacing teacher inputs, they are more likely to be receptive to the process (Tai et al. 2016b).

Study limitations

A key limitation of this work is that qualitative survey results have been used to infer what students perceive feedback literacy to be. The capabilities are not, and could not be, drawn from responses about direct questions about this phenomenon unless we were using a sample that was a priori feedback literate – which could not be determined. The deductive nature of the analysis, involving coding against an analytical framework may have meant that we missed other potentially important aspects of feedback literacy that have not yet have been conceived by the literature, researchers in our project team or implied by students in our sample. It is also of course possible that students who chose to respond to a survey, and volunteer for focus groups are likely to be vigilant and proactive, and possibly by association, be relatively feedback literate students. Therefore, the behaviours they describe relating to understanding, soliciting and using feedback information to their advantage may be more sophisticated than the broader population of students in higher education.

As this framework draws from a particular dataset of students in particular contexts, it does not claim to be comprehensive or authoritative. However, it does serve as a launching pad for wider studies of feedback literacy that might extend it further. Future research may usefully engage in testing the items in the feedback literacy framework, including whether some items are more potent than others and in what circumstances they operate.

Conclusion

Through this study we have developed a framework for student feedback literacy, with seven interrelated groupings. By focusing on a more comprehensive view of what is typically regarded as ‘feedback’ by students, we are opening up notions of what feedback is, and what work it can do. For example, some of the categories in the framework relate to building evaluative judgement, academic skills and self-regulation, which are important not only for study within a course but for future employability.

The literature typically reports that students and teachers view feedback as an act of teachers, rather than something that students initiate or influence. However, our study provided multiple instances of students reporting that they had agency in the process and saw the benefits of learning from feedback inputs beyond their current work. The framework exemplifies a view of feedback where learners are active, and the consequences that follow from this. The adoption of teaching and learning practices arising from the framework could encourage students to make the shift from acting as attentive listeners to becoming active seekers and utilisers of feedback information, as well as generators of useful information for others. With the clearer articulation of student feedback literacy provided here, we can embark on programmes of research to explore the relationship between educational designs that position the learner as active and the development of feedback capabilities that have utility beyond university courses.

Acknowledgments

We acknowledge the contribution of David Carless, research project evaluator on feedbackforlearning.org, in helping to construct the conceptual elements within the preliminary feedback literacy framework and Paige Mahoney for assisting with data analysis. Thank you also to Gordon Joughin for offering helpful comments on the framework categories.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Australian Government Department of Education and Training [grant number ID16-5366].