Shifting feedback agency to students by having them write their own feedback comments

Abstract

In higher education, there is a tension between teachers providing comments to students about their work and students developing agency in producing that work. Most proposals to address this tension assume a dialogic conception of feedback where students take more agency in eliciting and responding to others’ advice, recently framed as developing their feedback literacy. This conception does not however acknowledge the feedback agency students exercise implicitly during learning, through interactions with resources (e.g. textbooks, videos). This study therefore adopted a different framing - that all feedback is internally generated by students through comparing their work against different sources of reference information, human and material; and that agency is increased when these comparisons are made explicit. Students produced a literature review, compared it against information in two published reviews, and wrote their own self-feedback comments. The small sample size enabled detailed analysis of these comments and of students’ experiences in producing them. Results show students can generate significant self-feedback by making resource comparisons, that this feedback can replace or complement teacher feedback, be activated when required and help students fine-tune feedback requests to teachers. This widely applicable methodology strengthens students’ natural capacity for agency and makes dialogic feedback more effective.

Keywords:

• Inner feedback
• comparison
• self-feedback comments
• student agency
• feedback literacy

Introduction

Researchers have identified self-regulation as a learning goal for students in university and feedback processes as critical to its development (Butler and Winne 1995; Nicol and Macfarlane-Dick 2006; Allal 2016; Panadero, Lipnevich, and Broadbent 2019). Yet there is a tension between students developing as self-regulating learners and teachers telling them how to improve their work through the provision of feedback, as written comments or through dialogue (Nicol and Macfarlane-Dick 2006; Boud and Molloy 2013). According to research, the way to address this tension, here referred to as the feedback agency tension, is for students to take a more active and agentic role in eliciting, processing and acting on feedback information from others, usually teachers but also peers (Nicol 2010; Carless et al. 2011; Van der Kleij, Adie, and Cumming 2019). This has recently been framed as developing students’ feedback literacy (Carless and Boud 2018; Molloy, Boud, and Henderson 2020).

Despite the central role that dialogue plays in feedback processes, reframing students’ as active agents in their dialogue with teachers and others is too narrow a framing to address the agency tension. Dialogue here refers to both written and verbal interactions with students about their work. From a practice perspective, power and knowledge differences between teachers and students will always impede attempts to shift agency in dialogue to students (Steen-Utheim and Wittek 2017). However, the main limitation of the dialogic perspective is conceptual rather than practical. This conception does not acknowledge the feedback agency which students exercise naturally and informally, for example, when they use information in a textbook to update an assignment they are working on (Nicol 2019, 2021; Jensen, Bearman, and Boud 2023). This is part of a wider concern in feedback research, that the context within which dialogue operates is not sufficiently acknowledged (Gravett 2022; Tai et al. 2023). Against this background, we devised an intervention that did not rely on dialogue as the primary locus for addressing the agency tension.

This intervention drew on Nicol’s inner feedback model (2019, 2021). Students produced some work, compared it against information in some material resources (in this case, information in journal articles) and wrote their own self-feedback comments before receiving any comments from their instructor. The hypothesis was that if students could author high-quality feedback comments on their own work, this would constitute robust evidence of their exercise of feedback agency. This was evaluated by a detailed examination of the nature of the comments that students produced, with complementary data also collected about how this approach influenced students’ perceptions of agency. The results provide compelling reasons to rethink current approaches to shifting feedback agency to students.

Literature review

Feedback literacy

Building on Sutton (2012), Carless and Boud (2018) define feedback literacy as ‘the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies’ (1315). They identify four inter-related capabilities and dispositions that students need for effective uptake of feedback: appreciating feedback; making judgements; managing affect; and taking action. This framework has been further developed by Molloy, Boud, and Henderson (2020). The main addition is that students producing feedback for peers is also identified as a component of feedback literacy.

Although feedback literacy is a relatively new concept, many approaches to developing student agency have been proposed based on this framing (Molloy, Boud, and Henderson 2020; Wood 2021; Malecka, Boud, et al. 2022; de Kleijn 2023), with some implemented (Little et al. 2023). The latter include designing learning opportunities and preparing students so that they proactively elicit, process and respond to feedback information from teachers and others, rather than passively waiting to receive it (Hill and West 2020; O’Connor and McCurtin 2021; Malecka, Ajjawi, et al. 2022; Hui et al. 2023), developing students’ capacity to make evaluative judgements through planned activities such as self-assessment, peer review and/or analysis of exemplars (Hoo, Deneen, and Boud 2022; Tam 2021).

Most proposals and interventions to develop feedback literacy are founded on socio-constructivist principles where feedback is conceived as a dialogue between teachers and students and student peers. This is evident in the language used in feedback literacy research, for example, that students need to develop proactive recipience for feedback (Winstone, Mathlin, and Nash 2019), that they co-construct feedback meanings with their teachers (Heron et al. 2023) or that we need to support students’ uptake of feedback (Wood 2022). While Carless and Boud (2018) do use the word information in their definition of feedback literacy, in their writing they primarily identify information as comments, as do Molloy, Boud, and Henderson (2020) in their update of the literacy framework. Also, a core pillar of that framework is managing affect which is about students managing ‘defensive responses to feedback, particularly when comments are critical, or grades are low’ (Carless and Boud 2018, 1317). Even activities such as self-assessment, peer review and analysis of exemplars have an underlying aim to support student agency in dialogue and in uptake of feedback from others. Indeed, dialogue is often integral to these interventions (Han and Xu 2020; Hoo, Deneen, and Boud 2022). For example, Smyth and Carless (2021) contend that analysis of exemplars will be more effective if teachers facilitate a discussion of them with students.

The centrality of dialogue is also signified in the recently developed instrument to measure students’ feedback literacy (Dawson et al. 2023). This instrument, which focuses on what students do in feedback processes, has 24 items. Sixteen mention the word comments (e.g. I check whether my work is better after I have acted on comments) and six imply comments (e.g. other people provide an input about my work, and I listen or read thoughtfully). While we do not dispute that human dialogue is central to feedback processes, and indeed to all learning, the argument here is that the dialogic conception on its own is problematic as a framing for feedback literacy and to address the feedback agency tension. There are pragmatic concerns but more importantly significant conceptual issues.

Problems with a sole focus on feedback as dialogue

Practical concerns

When feedback is conceptualised as a dialogue between teachers and students any attempt to address the agency tension is in danger of being compromised. First, the uneven power relationship will impede some students from taking more responsibility, especially lower-ability students (Orsmond and Merry 2013) and those from cultures ‘where strict hierarchies and great power distance’ are the norm (Rovagnati, Pitt, and Winstone 2022, 351). Second, as this concept involves teachers making judgements about students’ work, some students might come to rely on these judgements, rather than make their own. Even if teachers are sensitive in the way they handle dialogue, or dialogue is used to facilitate student self-assessment, peer review or exemplar-based activities, there is still the risk that students will follow the teacher’s insights rather than make the effort to independently form their own. Also, given that students require different levels of support and different support at different stages during learning, teachers will need considerable sensitivity and adaptability to meet individual students’ needs and to fade out support as student’s agency increases (Malcolm 2023). Similar issues arise with peer dialogue, but also different agency issues such as students not believing their peers have the capacity to make judgements or that it is desirable for them to do so (Kaufman and Schunn 2011).

Conceptual limitations

A more important issue is that the dialogic conception on its own is too narrow a framing of feedback processes. It does not acknowledge the feedback agency that students are already exercising implicitly and naturally in everyday life and during study (Nicol 2019, 2021; Jensen, Bearman, and Boud 2023).

Nicol (2021), drawing on earlier research by Butler and Winne (1995) and by others on student self-regulation (Nicol and Macfarlane-Dick 2006; Allal 2016; Yan and Brown 2017), maintains that conceiving feedback only as a communicative exchange results in an artificial separation of planned and formal feedback processes from natural and implicit feedback processes (see also, Jensen, Bearman, and Boud 2023). Students, like all of us, are exercising feedback agency and generating feedback all the time, even when there are no comments from or dialogue with others. Nicol (2022) refers to this as inner feedback and provides the following definition:

the new knowledge that students generate when they compare their current knowledge against some reference information, guided by their goals. (1)

In this framing, students exercise feedback agency whenever they compare their work against any relevant reference information, for example, when they look up a textbook to clarify their understanding of a concept or use information in resources found online to improve their work before submission. Nicol (2021) argues that the failure to acknowledge this agency, and build on it, is a fundamental gap in feedback thinking and research.

From a related perspective, there is now a growing body of research that challenges the dialogic conception underpinning feedback literacy on the grounds that it fails to take account of the context within which dialogue operates. Gravett (2022), for example, notes that in feedback literacy research feedback is usually portrayed as a ‘binary dialogic event between individuals’, rather than as an event situated in a social and material context (271). Context here is broad and includes the technology used, power, space, time, institutional structures and procedures as well as resources such as journal articles, diagrams and video recordings. Taking this wider view, agency does not reside in individuals, neither students nor teachers: rather, it is enmeshed within complex social and material arrangements (Tai et al. 2023). Although this socio-material framing helps us better understand feedback dialogue, its complexity - that everything interacts with everything else - makes it difficult for teachers to identify practice ideas they could easily implement to increase students’ agency, and that would impact in the short rather than long term. So, how might we address the feedback agency tension without overly relying on dialogic processes alone?

Addressing the feedback agency tension

In this study, we investigated feedback agency using the model of inner feedback proposed by Nicol (2021, 2022). A key concept in that model is comparison, seen as the core mechanism by which students generate inner feedback (i.e. ideas for improvement). To learn from teacher comments, even if provided during a dialogue, students must compare them, or more accurately their interpretation of them, against their work and generate inner feedback out of that comparison. However, as alluded to, students implicitly use other information for comparison and for feedback generation besides comments. As they are producing academic work (e.g. writing a report, solving a problem) they will compare their developing work against their goals and against different sources of information that will help them reach those goals. That information might be generated internally (e.g. from memories of past performance) or derived externally from resources (e.g. textbooks, videos, rubrics) or from observations of people or events (e.g. chemical reaction in the laboratory). In this study, the comparison information was in published journal articles.

Given that most feedback comparisons happen naturally and implicitly and are thus below conscious awareness, the most important principle in Nicol’s model (2021, 2022) is that to capitalise on their pedagogical power teachers must plan for them. This translates into the following implementation sequence which can be repeated cyclically across a task or course. Students do/produce some work, then compare what they have produced against some information in a resource (or resources) and make explicit/tangible the output of those comparisons, as written self-feedback comments and/or through discussion and/or in actions for improvement. In this investigation students were required to write their own feedback comments.

This perspective on feedback is neither purely cognitive nor socio-constructivist. Rather it is situated, as it acknowledges that what feedback meanings learners construct depends on their interactions with people and resources in the learning environment and beyond. In other words, resources and people serve as co-regulating influences on students’ self-regulatory processes (Allal 2020). Hence this conceptualisation also offers a practical way to begin to address concerns of socio-materialist researchers. Resources can be viewed as the proximal material context for dialogic feedback interactions (and vice versa). Information in disciplinary resources is already the subject matter of most feedback dialogue, but surprisingly is not usually harnessed deliberately by teachers as a specific driver to support student feedback agency. Yet, resources are one of the easiest contextual factors to bring into play, as recent research shows.

Research on feedback comparisons

In a study by Nicol and McCallum (2022), first-year students compared a 500-word essay they had written against two peer essays plus a high-quality exemplar essay. After each comparison they wrote what they learned as self-feedback comments. These comments not only matched the comments the teacher would have provided but went beyond them, in level of detail, number of action points and through inclusion of comments not given by the teacher (e.g. on alternative ways of structuring the essay). Berg and Moon (2022) had chemistry students compare their analysis of gold extraction against three different teacher-constructed exemplar analyses. Again, students generated significant self-feedback and improved their understanding of gold extraction without receiving teacher comments.

Lipnevich et al. (2014) had psychology students produce a draft research report, then different groups were given a rubric and/or three exemplar reports to help them update their draft report. All groups improved their grades from draft to redraft with the rubric-alone group showing most improvement. This shows not only that students can generate their own feedback from resources of different kinds, but also that this method can reduce teacher workload. In a follow-up study, secondary school students wrote an essay, and then updated it using either teacher comments or annotated exemplars, or both (Tomazin, Lipnevich, and Lopera-Oquendo 2023). All students made equal grade improvements from draft to redraft, evidencing that teacher comments are not necessarily the better comparator. Sambell and Graham (2020) showed that to benefit from inner feedback processes, students had to deliberately compare the exemplars against their own work. It was not enough just to provide exemplars.

The current study

Research focus

While the studies above show that resource comparison have potential to address the feedback agency tension, none of them provide any detail regarding the self-feedback that students generate from making comparisons. One provides quantitative data on how students’ self-feedback comments compare with teacher comments, others provide evidence of learning gains after making comparisons, and one provides a mix of data including students’ perceptions of their learning from comparisons. The comparators in each study differed (i.e. exemplars, rubrics, comments, peer works) and student feedback agency was not their prime focus.

Given this research gap, this study had two inter-related objectives:

1. to gain more insight into what self-feedback comments students generate when they compare their work against information in resources;

2. to determine whether writing such self-feedback helps alleviate the feedback agency tension.

To achieve these objectives, we examined both the self-feedback comments students produced through the lens of feedback agency (self-feedback data) and the effects of writing feedback comments on students’ perceptions of agency (perception data). Importantly, students wrote their self-feedback before receiving any feedback comments from an educator. Hence, this research, where students author their own feedback comments, provides both a distinctive and robust test of their feedback agency.

Context of implementation

The context of this study was the writing of a thesis literature review by final-year undergraduate economics students. Thesis writing is a complex task that is usually new to undergraduate students. Hence, supervisors invariably provide detailed comments on student’s writing, and many follow up with one-to-one feedback dialogue. At this level of study, students are also expected to take significant responsibility for their learning. The tension between supervisor feedback and students exhibiting agency in their thesis production is widely discussed in research on supervision; and like the research cited earlier, recommendations to address this tension invariably focus on how supervisor-student dialogue is enacted (Greenbank and Penketh 2009; Roberts and Seaman 2018; Malcolm 2023). Hence this constitutes an ideal context to investigate the feedback agency tension.

Methods

Student sample

Given the aim was to collate detailed data on students’ self-feedback comments, an in-depth exploratory investigation with a small number of students was appropriate (Sleker 2005). The participants were three students attending a UK university who were writing their dissertation thesis in economics. These students are identified by the pseudonyms Alex, Blake and Cameron or by the pronoun ‘her’. For ethical reasons it is not possible to provide more detail other than that they differed in nationality, gender, in the type of research they were engaged in (quantitative or qualitative), and in the quality of the draft review they produced before the comparison activities. All were aged between 18-25 and one was a joint honours student studying Economics and English literature. Ethical approval was provided by the University College of Social Sciences Ethics Committee for Non-Clinical Research Involving Human Subjects [reference number 400170120]. Students provided written consent to use their data anonymously.

Resources for comparison

Before the intervention reported here each student had already written a draft literature review. The resources for the comparison activities were two published literature reviews drawn from top quality economics journals. These two literature reviews differed from each other and from each student’s own review in economics subject content, as the intention was that students generate feedback on the writing process (argument, structure, use of evidence). This differs from most prior research where students compare their written work against peer works or exemplar works on the same subject topic (e.g. Nicol and McCallum 2022). We use the term ‘resources’ rather than exemplars in this study, as it is the information in resources that students compare their work against, and because resources can differ in subject content and in presentation format with reference to the students’ own work. An electronic proforma was created with prompts to collect data from the comparison activities (see supplemental materials).

The intervention

Students engaged in two different comparison activities in sequence then formulated feedback requests for the supervisor. The latter, comparing what they know versus what they would like to know, can also be seen as a comparison activity (Figure 1).

Figure 1. Sequence of comparison activities.

Comparing two published literature reviews

In the first comparison, students compared the two published literature reviews against each other and wrote down what they learned as self-feedback comments. Specifically, students were prompted to identify three similarities across the two reviews that would warrant calling them high quality and to provide a written rationale for each (see online supplemental materials). This was intended to encourage students to generate an internal standard of good quality writing. Prior research shows that when students compare two similar entities that differ in subject content but are similar at the deep structural level, they invariably see beyond the surface features of these entities and identify and extract the common concepts, principles and structures that they share (Gentner and Maravilla 2017). This supports transfer of knowledge to new contexts.

As the first comparison was sequenced after students had produced their own draft literature review, we anticipated students would not only generate comments about what makes a quality review but would also spontaneously generate internal feedback about their own literature review, as occurred in Nicol, Thomson, and Breslin (2014).

Comparing published reviews against own literature review

In the second comparison, students compared each published literature review, one after the other, against their own draft review and wrote self-feedback comments. They were also asked to rank all three literature reviews, to give a reason for their ranking decision and to write what improvements they would make to their own work. The externalisation of inner feedback in writing is critical at both comparison stages as are the instructional prompts as these also mediate students’ feedback productions (Nicol 2022).

Requests for supervisor feedback

The last prompt during the second comparison asked students to write feedback requests for the supervisor. This encouraged them to identify what feedback they thought they still required and/or perceived themselves as unable to generate. These requests were conceived as a bridge linking resource-generated feedback to supervisor provision of feedback. For research purposes, the supervisor wrote her initial feedback before she saw the students’ written feedback. She then added further feedback and addressed students’ feedback requests. Students thus made both comparisons and wrote their requests before receiving supervisor comments.

Data collection

Two sets of data were collected in this study: students’ written self-feedback comments and their perceptions of the feedback comparison approach. Ranking with explanations, stating what improvements students would make to their literature review and writing feedback requests all constitute self-feedback data. Perception data was collected using a questionnaire which students completed after all comparison activities (see supplemental materials). The authors also met students as a group to discuss their experiences and to enable them to elaborate on their questionnaire responses. Both the questionnaire and focus group provided data on the perceived value of different comparison activities and about the relative value of generating self-feedback comments versus receiving supervisor comments.

Data analysis

A qualitative interpretivist methodological approach was appropriate to this research (Denzin 2001). The students’ self-feedback comments were interpreted in relation to the notion of student feedback agency (as per the inner feedback model) with a specific focus on how resource comparisons help students exercise feedback agency. Demonstrating the practical value of the implementation sequence as a way of addressing the feedback agency tension was the prime focus, hence the small number of students and the detailed data on their self-feedback comments (Sleker 2005). The thematic analysis of the questionnaire and focus group data was mostly deductive in relation to ideas of feedback agency and the inner feedback model. Neither in the questionnaire nor focus group did we ask questions using the word agency or its equivalent. Both authors coded the data separately in relation to agency themes. There were no differences of any substance in their thematic coding.

Results

Given the extent of written feedback comments students generated, we present a more elaborate account of Alex’s data with that from Blake and Cameron used to enrich the analysis, mainly through highlighting similarities and differences. While each student’s self-feedback data is presented separately, the perception data for all three students is collated and presented in a separate section.

Alex’s self-feedback comments

Comparing two external literature reviews

All students were able to identify key principles that constitute a good quality literature review by comparing two published reviews. Table 1 shows what Alex generated.

Table 1. Alex: feedback generated from comparing two external literature reviews.

Both literature reviews open with a paragraph outlining their own methodology, aims and general approach. They also introduce seminal work early in the review and discuss it in terms of influencing their own methodology. Both reviews bring up other work in a very concise way and always within the context of their own paper. Previous work and similar approaches by other researchers are referenced but always compared and contrasted to the approach in the paper itself. Any reference to previous work… is discussed within the lens of the current paper, either differentiating their own work or underlining similarities. None of the citations are thrown away. Both literature reviews act like a second opportunity to discuss their methods… justify their research decisions. Both reviews make explicit references to empirical evidence and previous data. Both references to quantitative and empirical evidence happens towards the end of the review, where the authors… highlight the relevance… of their own models.

These principles are consistent with the main advice, in articles and guidance documents, on how to produce a good literature review (Maier 2013; Hart 2018). Common recommendations are that writer(s): (i) make the scope of their review clear at the outset; (ii) discuss major issues and debates around the research topic; (iii) only use external literature insofar as it is directly relevant to their own research; (iv) identify gaps in the extant literature (e.g. theoretical, methodological, empirical) and show how their own research addresses those gaps and adds to the literature.

Comparing published reviews with own literature review

Table 2 shows what feedback Alex produced when asked to compare two external literature reviews against her own draft review. Note that in response to the first question in Table 2, Alex benchmarks her own review back to the principles of quality that she identified through her prior comparison. From this she identifies a range of weaknesses in her own literature review, ‘it did not… compare and contrast’, ‘I made no references to my own methodology’, ‘I brought up evidence but not as succinctly’. These typify the comments a supervisor might provide but Alex generates them herself, by exercising her own feedback agency.

Table 2. Alex: feedback generated from comparing published reviews against own review.

To what extent is your literature review similar/different from the two external reviews? Like the two reviews above, I brought up seminal work and previously published research. However, my own literature review did not continuously compare and contrast this work within the framework of my own approach. Instead, my review was a collation of previous literature, or a bibliography of sorts. I made no references to my own methodology, so the review was devoid of context. I also brought up empirical data and evidence, but not as succinctly. My own references to empirical evidence acted more as an introduction to the field, rather than acting as justification for my own approach. Can you rank order the three literature reviews including your own? Review 2, review 1, my own. I liked how the authors in review 2 outlined their model in the review and brought up differences and similarities with their methodological choices and already published work. This provides the reader with a deeper understanding of each component of the model, whilst providing justification for the decisions in the form of reference to other works. Review 1 also did this but focused their discussion in one core paragraph. What will you do to improve your own literature review based on your answers so far? I will rewrite it almost completely… I don’t feel that it really works as a literature review, since the literature was not really “reviewed” but merely collected. Primarily, I am going to write it with a spotlight on analysing already published work in terms of differences and similarities with my own approach. A primary focus will be on methodological choices, rather than give a database of already published research.

Ranking and improvements

When asked to rank all three reviews Alex provides a coherent rationale for what makes literature review 2 better than 1, with both considered better than her own. For Alex, ranking activates feedback on the relative quality of her literature review against these high-quality external reviews. As such, it anchors Alex’s self-feedback to quality standards, which is what supervisors try to do with comments. When asked what she would do to improve her own literature review, Alex wrote: ‘I would rewrite it almost completely’. She then explains why. It is notable that this explanation also draws on her earlier comparison of the two published reviews as well as the ranking.

Self-generated feedback in relation to supervisor feedback

The supervisor provided eight comments on the writing and five on the subject content of Alex’s literature review. As expected, Alex provided no content feedback. All supervisor comments focused on how Alex had used the prior literature and research to position her own research:

You have picked up some interesting and related papers… However… missing in this review is that it is not always clear how these are related to your paper. Or even, how your paper contributes to the big picture. What you present here is a good first step… There should be next steps to refine this presentation by linking your own work in terms of similarities and differences… You can create sub sections and discuss literature in relation to those.

If one compares the feedback Alex produced (Tables 1 and 2) with the paragraph above, it is clear that Alex’s self-feedback not only matched the feedback the supervisor subsequently provided, but it was more detailed and elaborate. Nonetheless, although satisfied with her own feedback productions (see perception data below), Alex still had a few feedback requests for the supervisor:

I feel as though I have a clear idea of how I want my review to look after examining these pieces… although, I would appreciate feedback on specific issues. Is the first paragraph of my review better suited somewhere else? Does a literature review require a general introduction to the market being studied?

This request was easily addressed by the supervisor enabling her to tailor her feedback to the students’ perceived needs. She was also able to validate the feedback Alex had already generated by briefly annotating Alex’s answers to the comparison questions (e.g., ‘well noted’).

Action on feedback

All students had the opportunity to update their draft literature review after the feedback intervention. In her final submission, Alex successfully highlighted the centrality of her own work in connection to the wider literature producing a much higher quality literature review, based on the feedback she generated through resource comparisons, which was endorsed and reinforced by the supervisor comments.

Blake’s self-generated feedback

Comparing two external literature reviews

Blake derived similar principles to Alex from comparing the two published reviews. However, what distinguished her comparative analysis is that her approach reflected her status as a joint-honours student studying Economics and English Literature. This was evident in her use of the term discourse markers which is a phrase, often used at the beginning of a sentence, that plays a role in managing the flow and structure of a piece of writing:

Both [reviews]… begin and continue to use clear discourse markers, to signal the manner in which their piece relates to previous contributions to the field… both include statements… such as "this paper extends the approach…” "This study is related to several studies that have looked at… “and “To further highlight the differences”.

This difference in lens of analysis exemplifies how context (in this case the context in which Blake studies), mediates and nuances the feedback that students generate from making comparisons, even when students derive similar ideas or principles.

Comparing published reviews with own literature review

Blake also identifies differences (i.e. weaknesses) in her literature review with reference to the qualities she identified in the published reviews. For example:

Considering the extent to which I clarify what this thesis hopes to contribute to the field the amount of explanation I extend about the dissertation’s USP… [unique selling point] is unclear… it should be as clear as that of Lit. 2, who repeatedly clarify and contrast and outline their USPs.

Unlike Alex, Blake, at times, exhibits uncertainty about her own review (‘my literature review has a clear use of discursive markers - an overabundance perhaps!’) and about what to do about this uncertainty: ‘My literature review goes to greater lengths to define terms than both examples, I am not sure what to make of this’. When a supervisor sees that students’ self-feedback comments express doubt this is an opportunity for them to step in with focused advice, as the supervisor did here (see also Table 3).

Table 3. Examples of supervisor’s responses to Blake’s feedback requests.

Blake’s feedback requests	Supervisor responses
To what extent can I be more explorative and less linear with my discussion, compared to the other two Lit. Reviews?	You could be both explorative as well as linear if you are able to conclude and it does not impact the output of your research. Support all argument with evidence (data or research paper or books or using reliable and official sources)
They typically reviewed info from the same medium, am I at an advantage or disadvantage for looking at articles, journals, books etc?	You are not disadvantaged as long as you are supporting all arguments with evidence.

Ranking and improvements

Blake, like Alex, also ranks literature review 2 as best and her own as least good. She clearly identifies weaknesses in her own review ‘Features too many political and legislative facts for context purposes of which the relevance is not explicit’. Blake’s ranking also reflects her language studies background, for example, ‘Lit 2 shows the best linguistic dexterity’. Her ideas for improvement also refer back to the principles she derived from her earlier comparison of the two published reviews:

I aim to better explain how it is that my approach differs from that of the contributors I am mentioning, with tightly interwoven, shorter more succinct sentences.

Self-generated feedback in relation to supervisor feedback

For Blake, there were seven supervisor comments on her writing and none on content. Similar to Alex, the feedback that Blake generated overlapped significantly with the supervisor’s feedback comments. However, Blake had five feedback requests, more than the others. Hence the supervisor added significantly to the comments she had already provided to address these requests. Table 3 shows two examples of supervisor comments in relation to Alex’s requests. Feedback requests have been advocated in the feedback literacy literature (e.g. de Kleijn 2023). Yet, this investigation shows that anchoring such requests in prior resource comparisons would increase their power, as students will have invested effort in creating a more elaborate knowledge and prior feedback platform from which to formulate them (e.g. Table 3 first request).

Action on feedback

Blake’s draft literature review submission lacked structure and focus. She made significant improvements in the final version with better engagement with the wider literature and deeper connections with her own work, showing that she was able to implement her own feedback insights.

Cameron’s self-generated feedback

Comparing two external literature reviews

Cameron identified similar principles of a good literature review to Alex and Blake. However, she provided significantly more detail. From the first comparison, she produced 3093 words in contrast to 187 and 256 from Alex and Blake respectively. Most of this detail comprised elaborate examples, with citations drawn from the two published literature reviews, explaining how the writer had implemented the principle. The following shows the pattern:

These two literature reviews show the state of current studies in relation to the research questions. For example, review 1 shows three contemporaneous works which are related to its research. First… Gopinath (2006) studied the importance of shock in the trend in emerging countries… Second, Yue (2006) found the effect of the bargaining power of the lender and borrower on… Lastly, Cuadra and Sapriza (2006) discovered…

Cameron’s production of more self-feedback could be attributed to individual differences in ability, motivation, time available, prior experience, in writing process or style. When compared to Alex and Blake, Cameron’s feedback is not any better in quality: rather it is just more extensive. Hence, a key message for teachers is to expect individual differences in student’ feedback productions, and to acknowledge and build from there.

Comparing published reviews with own literature review

Cameron maintains that she had already implemented the principles she extracted from the first comparison in her own draft review. She provides considerable detail to prove this claim using extensive examples from her own review. The following, again, shows the pattern of her response:

My literature review represents a detailed outline as well as aims. For example, section 2 explains the outline that the research paper will explain the structure of the UK housing market and goes on to state its aim to analyse how the determinants impact house prices. In addition [my]… research paper shows the state of current knowledge in relation to a research question. In section 2.2, for example…

Ranking and improvements

Cameron ranks her own review as better than the two published reviews. She justifies this by identifying weaknesses and critiquing the other reviews. She backs her claims with evidence. Whether Cameron’s ranking is accurate is not the issue here. Her work was of high-quality, and she saw her literature review as written for a dissertation rather than for publication: so that was the basis of her ranking. What is important to note is the depth and extent of self-feedback that ranking generates when one of the items ranked is the students’ own work.

Even though she saw her own work as of high quality, Cameron still identifies improvements she would make to her draft review from the comparison activities:

To improve my literature review additional relevant studies to fully support criticisms raised [could be included]… For example, the review can include… Furthermore, the review can present more findings such as…

Student-generated feedback in relation to teacher feedback

The supervisor only provided three comments on the writing process in Cameron’s draft review, and one was an affirmation of the quality of Alex’s writing (‘you have indicated how these studies relate to yours’). This acknowledges that Cameron’s work was of high quality and that significant improvements were not needed, at least from the supervisor’s perspective.

Nonetheless, Cameron still believed that she would benefit from supervisor feedback, yet her request does not specify what feedback she would like.

At this stage, I think I would benefit most from some feedback provided by a professional researcher for evaluation of my work and any recommendation on how to improve my research and writing skills for the future.

This request is understandable as we all value human feedback, especially reader response feedback on our writing, as this differs in its merits and limitations from resource generated feedback. Hence, the argument in this article is not that we replace human feedback with resource-based feedback. Rather it is that we use both in productive ways so one supports the other. Staging resource comparisons before dialogic comparisons is one simple way of achieving this.

Action on feedback

Cameron produced a very-high quality draft literature review. Nonetheless, she still made significant improvements in her final submission. This shows that even students who have produced high quality work can be challenged by resource comparisons to further improve the standard of that work.

Student’s perceptions of the value of the different comparison processes

The data from the questionnaire and focus group provide further evidence, alongside students’ self- feedback comments, that making resource comparisons before receiving teacher comments increases students’ own feedback agency.

Comparing two external literature reviews

All students reported that comparing the two literature reviews helped them grasp what constitutes a good quality review. However, while acknowledging that making such comparisons was a natural process they might engage in implicitly, for example by consulting journal articles and handbooks they reported that the requirement to make the output explicit in writing promoted deeper thinking:

you are making all these comparisons in your head, but when you are actually forced to write it down you realise how much more you have to think about it. [Alex]

Students also noted the importance of the comparison sequence (i.e. ‘do’ then ‘compare’):

It is better to have something first then you’re aware of how you could apply what you find from the comparison and improve your own. [Blake]

They also discussed how comparing two literature reviews helps establish what constitutes quality and standards:

Had there only been a single piece of work, the consistency between good literature reviews would not have been obvious… comparing two ideal examples helps cement what I might otherwise have skipped… if you’ve got two you say, this is the standard, if you had more… you would get lost. [Alex]

Comparing the published reviews with their own draft review

When asked about their experiences from explicitly comparing the two literature reviews against their own, students responded that this was:

the most helpful exercise – it helps build criticism for my own work and understand what went wrong/right. The real benefit in the comparisons was comparing the consistency between the two literature reviews against mine, seeing what both the authors had done that I had not. [Alex]

Again, making explicit in writing the results of the comparison against their own work was deemed critical by students as were the instructional prompts:

I had always made implicit comparison with other work but having a structured framework, with which to work through, made the process more efficient and helped tease out better comparisons. [Alex]

Students also highlight the feedforward value of writing their own feedback:

Writing the results… has helped me organise the important points… also helped me reflect upon them thoroughly and make reference to them in the future so that I could apply the lessons to improve. [Cameron]

Added value of resource comparisons over supervisor feedback

When asked what the comparison process adds to their feedback over and above the feedback a supervisor provides, students’ responses centred on their agency, and on a reduction in their dependency on the supervisor. Indeed, Alex provides a compelling case for including this explicit comparison methodology from the first year of university:

The comparison process… because it takes more personal effort to generate… holds a lot of weight. Had I started this self-generated feedback process earlier in my university career, I would be a lot more proficient at it and think I would require less supervisor feedback as I would have the confidence to build my own feedback system.

Blake, who reported asking many questions of her teacher in the past, notes that the comparison processes resulted in her asking less and more focused questions. She also alludes to a greater sense of self-efficacy:

I feel empowered to ask more incisive questions, because I am more confident that I understand the basic tenets… having to ask basic questions, means I feel like I’m asking too many questions…[this] skews my perception of how well I am working independently. This [making comparisons] …has so much value, I have always needed feedback, but this drills down, it really is retained differently in my head.

Cameron notes that by making resource comparisons she can improve her own work, before receiving supervisor feedback:

The comparison process was an internal learning procedure which I believe needs to precede in order to benefit fully from the supervisor’s feedback… This has helped me structure my review… before I could further improve the draft based on the supervisor’s feedback.

Students also reported that generating their own feedback helps address timing issues usually associated with received feedback.

you get [supervisor] feedback but it is slow… with this method you are not waiting for supervisor feedback. Generating your own feedback is lot quicker and you can use it for anything. [Alex]

Importantly, students maintained that generating their own feedback before requesting feedback enabled them to identify how the supervisor could best help them.

Supervisor feedback is the fine tuning which is as it should be as they have a set of specific skills that… if you can figure something out by yourself then why not do it that way and then what is left over… and then it can be refined by someone who really knows what they are doing. [Alex]

What supervisors provide that making comparisons does not

Making comparisons against resources does not however diminish the importance of external feedback advice from a teacher. Indeed, all students gave reasons why they valued supervisor comments:

I think that it’s an important yardstick… I do still want faculty reassurance. [Blake]

The feedback from the supervisor… was really helpful because I couldn’t identify those errors myself, even after having read my dissertation [thesis] many times. [Cameron]

Supervisor feedback tends to cut to the core of issues more concisely… also provides guidance and reassurance… and a more trusted source… to trust my own self-feedback as well. [Alex]

Taking responsibility for own comparisons

Students confirmed that their experience with the comparison method would influence what they would do in future:

This experience has taught me that both comparing different studies and making comparisons with my own work are helpful in improving my project. For the future I… will definitely be using this method. [Alex]

Indeed, Alex reported in the focus group that she had already used the protocol of comparison questions provided for the literature review as a tool to compare different thesis abstracts and to improve her own abstract.

Discussion

Students as the generators of feedback

Overall, the results of this investigation show that students are able to generate high-quality self-feedback comments by making resource comparisons, with minimal supervisor prompting. This feedback not only mirrored the comments their supervisor subsequently provided, but it was also more detailed and individualised. Students successfully utilised this self-feedback to improve their work. These findings constitute compelling evidence of student feedback agency. The perception data add to this evidence by showing that students experience generating their own feedback as empowering, as reducing their reliance on supervisor comments. Students also believed they could use this comparison method to generate feedback immediately and at any time without waiting for supervisor feedback, with one student having already done so. Further, they reported that after generating their own feedback using resources, they were better able to formulate feedback requests for their supervisor.

Practical considerations

Nature of the comparison information

In this study, students only generated feedback on the writing process and not the subject content. This might suggest there are limits to students’ ability to exercise feedback agency. However, the focus on writing was deliberate. The published reviews were on a different topic from each other and from the students’ own, and the instructional prompts directed students towards the writing quality. If supervisors wished students to generate content feedback, they could instead provide literature reviews on the same topic as the student’s own and formulate appropriate prompts. However, such comparators might lead to concerns about copying of subject content (Handley and Williams 2011). Hence other comparators might be used such as a video of experts discussing different positions relevant to an aspect of the literature review content. This focus on different format comparators, in this case text against a video, is unexplored territory in feedback research.

Explicitness: students writing their own feedback comments

A distinctive feature of this study was the methodology of having students render visible their own internal feedback processes as written self-feedback comments. Students reported this was critical as were the instructional prompts, as it encouraged them to think more deeply about the feedback they generated and to organise feedback points for future reference. Explicitness is a key principle in the inner feedback model. Nicol (2021) maintains it has value, both for students and the teacher. For students it raises their metacognitive awareness of their own feedback capability. For teachers, it provides better diagnostic information on students’ learning. There are however other ways of making inner feedback tangible, for example, through peer discussion (Nicol and Selvaretnam 2022) or through improvements in work (Lipnevich et al. 2014), also deployed in this study. Nonetheless, there is special value in students authoring their own feedback comments, given that writing is a powerful learning activity in its own right (Klein and van Dijk 2019).

Role of instructional prompts

In this study, prompts were used to give focus to each comparison activity (i.e. identifying similarities, differences, ranking, requesting feedback and suggesting improvements). Based on this, readers might infer that prompting and providing feedback comments are similar in their effects on student agency. However, this is not how students perceived prompts, nor does it acknowledge that unlike comments, prompts are not judgements by others of students’ work: rather, they facilitate students to make their own judgements. This is important, as a prime cause of the feedback agency tension is students’ feelings of being judged. This is why models of feedback literacy emphasise that students learn to manage affect (Carless and Boud 2018; Molloy, Boud, and Henderson 2020). While feedback processes will always evoke some emotional response, it is notable that no student here reported discomfort or negative reactions from making resource comparisons. The perception data did however highlight an affective dimension, that supervisors, at times, reassure students about the value of their own feedback judgements.

Comparison sequence: before and after comparisons

This investigation differed from prior research (e.g. Nicol and McCallum 2022) in its two-stage comparison sequence. A question raised by this sequence concerns the timing of the first comparison. If the aim is to maximise student feedback agency, should it come before rather than after students produce their draft literature review? If before, students could use the results to generate better feedback while producing their work, rather than waiting until the redrafting stage. Research shows learning benefits when students compare similar entities before they produce their own work (Gentner and Maravilla 2017). However, if implemented, a different question is raised: What comparisons to enact after students produce the draft literature review so they can make further improvements? There are several possibilities. The same comparators could be used but with different instructional prompts, for example, that direct students to areas still weak in their work. Another possibility is that students compare their work against information in other resources and even those in a different format, for example, a flow-chart of the structures of some quality literature reviews. Again, this points to prospects for future research.

Workload implications for educators

Had this been solely a practice implementation rather than a research investigation, supervisor commenting workload might have been reduced, as she could have based her feedback not just on students’ draft literature review submission, but also on their written self-feedback comments. In other words, the supervisor could limit her comments to only those that students were not able to self-generate. Feedback requests also help supervisors tailor their comments to students’ individual needs, which makes their commenting more efficient. In effect, students’ self-feedback comments could either supplement supervisor comments or replace them, depending on the situation. However, whether supervisor workload is reduced or not will depend on the burden involved in reading students’ self-feedback, responding to their feedback requests and on whether this method will change the way educators enact feedback processes more generally. Hence studies investigating this will also prove valuable.

Theory and practice

Integrating resource comparisons and supervisor comment comparisons

While this investigation shows that students can write their own feedback comments, it is not an argument against supervisor-student feedback dialogue. Indeed, even though students reported this method increased their agency all expressed a need for and value in supervisor comments; for example, to help them identify weaknesses or gaps in their work they might not identify themselves. This aligns with the inner feedback model, that each comparison type, in this case dialogic and resources, have different merits and limitations, and that teachers create an appropriate mix.

The main recommendation here is to sequence feedback comparisons so that supervisor feedback comments come after students have generated their own feedback through resource comparisons. It is much easier to activate student feedback agency by harnessing resources than through modifications in students’ dialogic interactions with supervisors alone. Just as important is the finding that feedback requests after resource comparisons also increases students’ agency during dialogue with their supervisor. Feedback requests serve as a bridge, built by students out of resource-based comparisons, that productively links them to subsequent dialogic comparisons. This bridge can be strengthened further by having students engage in peer dialogue comparisons, after resource comparisons, with groups, instead of individuals, formulating feedback requests for the teacher. Indeed, resource comparisons and peer comparisons can be planned in iterative cycles across a series of learning activities or a course (Nicol and Selvaretnam 2022). The key, however, to increasing student feedback agency is always to end-load teacher comments, after other comparison activities.

Research methods and future studies

Generalisability of the findings or of the model

The small number of participants will for some raise issues about the generalisability of the findings to other student cohorts, disciplines and contexts. In response, it should be noted that this was not the intention of the study. The inner feedback model predicts that students will generate different self-feedback depending on individual differences and context. This was evidenced by the nature of the self-feedback the joint-honours student generated relative to others, and by the considerable detail that Cameron produced to the same instructional prompts. Nor was the purpose to prove that students could generate productive self-feedback. There is already evidence for this, direct and indirect. Rather, the purpose was to evaluate a methodology to increase students feedback agency by having them write their own feedback comments. It is this methodology, we argue, that can be generalised to other contexts. The main contribution of this article is in providing a relatively straightforward method of increasing students’ feedback agency - by sequencing resource comparisons before dialogic comparisons with feedback requests acting as the bridge between the two – that could easily be applied in other disciplines and contexts.

Future research

The scope is wide as this way of thinking about feedback is not commonplace in research or practice. Some questions related to student agency, for example might centre on:

• the disciplinary nature of comparators (e.g., medicine versus sociology versus STEM subjects),

• the relative effects on students’ agency of before and after comparisons and their combination,

• the merits of using comparators of a different format to the students’ work (e.g., comparing textual information against information in diagrams),

• instructional prompts and their varying effects in promoting agency,

• further extending students agency by having them select comparators and/or formulate prompts for each other.

• the effects of this methodology on teacher and student workload.

Research methodology

In thinking about future research, the methodology of students making tangible their own internal feedback as self-feedback comments will be important. This methodology not only has pedagogical value, but also research value in the data it provides. Data of this kind is essential if we are to scope the effects of different comparisons and prompts in different teaching and disciplinary contexts. A starting point would be to use this methodology to better understand what feedback students generate from teacher comments, as this is far from clear, despite its importance to feedback literacy research. For example, if collected, this would enable researchers to better understand the extent to which teacher comments help or hinder students’ agency development.

Conclusion

How to increase student agency in feedback processes is a pivotal issue permeating feedback research. Yet most models of feedback frame this issue in terms of students taking more agency in their dialogic interactions with instructors and with peers. This framing limits the potential of feedback for learning as it does not capitalise on the natural feedback processes that students engage in implicitly and incidentally when they compare their work against information in external resources (Nicol 2021). It also does not acknowledge that disciplinary resources are the immediate and intermingled context for dialogic feedback processes (Gravett 2022). Much more progress could be made in addressing the feedback tension by bringing into play comparisons against information in resources as a formal feedback method alongside dialogical comparisons. Overall, this research adds to the case for reconceptualising feedback as an inner process that relies, inter-alia, on different kinds of external information to fuel it (Nicol 2021). Another reason to embrace this conception is that it more accurately captures the common sense meaning of the word ‘information’ as used in the now widely cited definition of feedback literacy ‘the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies’ (Carless and Boud 2018, 1315).

Acknowledgements

Thanks to Bas Trimbos of the Department of Curriculum and Instruction in the Netherlands, Jenni Rose from Manchester Alliance Business School and John McCormick of Brandeis University, Massachusetts, USA for their helpful feedback on this article. Thanks also to the Economics students who participated in this investigation and shared their time, insights and enthusiasm.

Disclosure statement

No potential conflict of interest was reported by the authors.