Does perception mean learning? Insights from an online peer feedback setting

Abstract

Many peer feedback studies examined students’ perceptions. Yet, little is known about how perceived and actual learning are related, and how they are influenced by individual and contextual characteristics including success level, educational level, gender, and academic major. This exploratory study addressed this research gap. Students from a Dutch university (N = 284) attended a three-week online module during which they were engaged in peer feedback on argumentative writing. At the end of the module, we measured students’ perceptions of learning. The results showed no significant effect of success level on perceived learning. Master students reported higher perceived learning than bachelor students regardless of success level. We did not find gender or academic major effects on perceived learning across different success levels. The findings suggest that relying solely on students’ perceptions of their learning experiences may not accurately reflect their actual learning. Students may need support to help them align their perceived learning with their actual learning achievements.

Keywords:

• peer feedback
• perceived learning
• argumentative essay writing
• online learning

Introduction

Learning with and from peers is nowadays an essential instructional practice in many higher education contexts. To be prepared for life, students not only need to develop the skills to acquire knowledge, but they should also be able to monitor and evaluate their learning and that of others (Boud and Falchikov 2006), and to exchange feedback to improve performance. However, these skills can be difficult to acquire if students do not engage in learning activities in which they meaningfully practice these skills. One way of creating this experience for students is by involving them in peer feedback. We define peer feedback as “a learning activity where individuals or small group constellations exchange, react to and/or act upon information about their performance on a particular learning task with the purpose to accomplish implicit or explicit shared and individual learning goals.” (Alqassab, Strijbos, and Ufer 2018, p.12). Due to its significant benefits for academic performance (e.g. Wang et al. 2017), motivation (e.g. Chien, Hwang, and Jong 2020), reduced anxiety (e.g. Chien, Hwang, and Jong 2020), assessment skills (e.g. Alqassab, Strijbos, and Ufer 2018; Taghizadeh Kerman et al. 2022), and self-regulated learning (e.g. Bellhäuser, Liborius, and Schmitz 2022), peer feedback has been used to support students’ learning in various subject domains.

One area of research that has attracted particular attention in recent years is using online peer feedback to teach argumentative writing (Huisman et al. 2018; Latifi, Noroozi, and Talaee 2021, 2023; Noroozi et al. 2023). Online learning environments designed to teach argumentative essay writing through engaging university students in peer feedback have been shown to successfully improve students’ performance across different academic majors and levels of higher education (Noroozi et al. 2023). Students’ perceptions are commonly used to evaluate the impact of traditional and online peer feedback (see Alqassab et al. 2023). Several theoretical frameworks of feedback suggest that outcome variables such as perceived learning and actual performance are likely to be shaped by individual characteristics and contextual characteristics including success level, educational level, gender, and academic major (Narciss 2008; Panadero and Lipnevich 2022; Taghizadeh Kerman et al. 2023a). However, little is known about the relationship between perceived learning and actual performance in online peer feedback activities, and how this relationship might be influenced by individual and contextual characteristics. In this study, we attempt to fill in this gap of research by exploring their relationships.

Students’ perceptions in peer feedback research

Perceived learning and perceptions of the feedback (e.g. usefulness, fairness, acceptance) are frequently examined in peer feedback studies either as main outcomes (e.g. Beaver and Beaver 2011) or as complementary outcome measures (e.g. Misiejuk, Wasson, and Egelandsdal 2021). In their scoping review of studies on students’ perceptions of feedback, van der Kleij and Lipnevich (2021), found that these perceptions are hardly examined with performance outcomes. This also appears to be the case in peer feedback research, as only a limited number of studies investigated the relationship between students’ perceptions and learning outcomes. For instance, Kaufman and Schunn (2011) reported that students’ perceptions of fairness were not related to the revisions of their work. Brown, Peterson, and Yao (2016) found perceived helpfulness to be positively related to self-regulation but negatively related to grade point average. In a previous study, Alqassab, Strijbos, and Ufer (2019) showed that perceptions of peer feedback accuracy were positively related to the accuracy of peer feedback but not to task comprehension. Yet, this study was conducted in an experimental setting and students only underwent the peer feedback provider role. More recently, Taghizadeh Kerman, Banihashem, and Noroozi (2023b) found no significant relationships between students’ perceptions of peer feedback and peer feedback content and uptake. So far, no study has investigated the relationship between perceived learning and actual performance when peer feedback is used as an instructional activity, especially in the context of argumentative essay writing. This is very important, especially for interpreting findings from studies that solely rely on perceived learning to evaluate the impact of online peer feedback activities.

Actual and perceived learning in online peer feedback

Online peer feedback is not new to the peer feedback literature. According to a systematic literature review, the largest cluster of studies is represented by online peer feedback (Alqassab et al. 2023). This finding is unsurprising given the flexibility and adaptability that technology can afford to design and instructionally support peer feedback activities (Latifi, Noroozi, and Talaee 2021, 2023). In studies on online learning, including those with peer feedback elements, researchers use two types of measures to assess learning: performance measures and self-reported perceived learning measures. Research on online learning environments often relies on perceived learning measures as an evaluation criterion (Yunusa and Umar 2021). Perceived learning is defined as “a set of beliefs and feelings one has regarding the learning that has occurred. As such, perceived learning is a retrospective evaluation of the learning experience.” (Caspi and Blau 2008; p. 327). Unlike performance-based learning which is evaluated by others, perceived learning is subjective self-evaluations by the learners themselves (Barzilia & Blau, 2014), and can therefore be susceptible to inaccuracies. Yet, this measure is associated with several variables related to the learning environment including teaching effectiveness (Ryan and Harrison 1995), satisfaction with online learning (Baturay 2011), and intrinsic motivation (Ferreira, Cardoso, and Abrantes 2011). Students’ perceptions are important because they can shape how students approach the peer feedback provision and processing (Aben et al. 2019). This is not only limited to students’ perceptions of the feedback but also whether they perceive they have learned something after being engaged in a specific learning activity. The latter is assumed to be related to students’ engagement (Capsi & Blau, 2008).

Several scholars stressed that perceived and actual learning can be unrelated (e.g. Caspi and Blau 2008), with repeated findings from technology-enhanced learning research indicating that students overestimate their learning (see Caspi and Blau 2008; Barzilai and Blau 2014; Porat, Blau, and Barak 2018). In a study of online game-based learning, no significant relationship between perceived learning and performance was found (Barzilai and Blau 2014). Porat, Blau, and Barak (2018) found almost no significant associations between students’ perceived digital competencies and their actual performance. Prior studies even suggested that the misalignment between self-judgements and performance is greater when the learning material is presented on screen (e.g. Ackerman and Lauterman 2012). Students’ feedback perceptions were also found to be differently influenced by digital feedback compared to non-digital feedback (Ryan, Henderson, and Phillips 2019). Based on these findings it is quite evident that the relationship between perceived and actual learning in online peer feedback requires investigation.

Online peer feedback and argumentative essay writing

The positive impact of online peer feedback has been documented by studies in several subject domains such as language learning (Alemdag and Yildirim 2022), computer education (Wang et al. 2017), and argumentative essay writing (Latifi, Noroozi, and Talaee 2021, 2023, Noroozi et al. 2023; Awada and Diab 2023). In studies on argumentative essay writing, Latifi, Noroozi, and Talaee (2021, 2023) showed that scaffolded peer feedback activities significantly improved students’ argumentative writing performance. Awada and Diab (2023) compared the impact of online versus face-to-face peer feedback on argumentative writing and found that students in the online peer feedback condition outperformed those in the face-to-face condition. Using a pre-test and post-test design, Noroozi et al. (2023) showed that online peer feedback improved students’ argumentative writing across five different university courses.

Students’ learning experiences in studies of online peer feedback on argumentative writing have been captured by various outcome measures including the quality of argumentative essay writing (e.g. Banihashem et al. 2023; Noroozi et al. 2023), the content of the provided peer feedback (e.g. Noroozi et al. 2022; Taghizadeh Kerman et al. 2022), gains in domain-specific knowledge (e.g. Latifi, Noroozi, and Talaee 2021; Valero Haro et al. 2022), and students’ perceptions of the learning experience in terms of learning satisfaction and perceived learning (Banihashem et al. 2023; Noroozi et al. 2023). To better understand the effects of feedback on learning, students’ perceptions should be taken into account because of their possible moderating/mediating effect (Strijbos, Pat-El, and Narciss 2021) that can influence how the students make sense of the feedback, uptake feedback, and provide feedback (Narciss 2008; Aben et al. 2019). Importantly, studies that examined the relationship between these two constructs did not take into account the role of individual and contextual characteristics (e.g. success level, education level, gender, and academic major) as these factors are likely to influence the relationship between perceived learning and actual performance.

Role of individual and contextual characteristics in online peer feedback

Several recent feedback models (e.g. Aben et al. 2019; Panadero and Lipnevich 2022) but also prior feedback models (e.g. Narciss 2008) stress the central role of students’ individual characteristics and contextual characteristics in shaping feedback processes and outcomes. Here we follow the model by Panadero and Lipnevich (2022). Based on a systematic review of 14 existing models, this model presents five elements to be considered in the promotion of educational feedback. Namely: feedback message, implementation, context, agents, and individual characteristics. The latest is the central element modulating the interaction of the rest of the elements to generate a learning impact on students. In this study, we will focus on the elements of individual characteristics and context as we will explore how some of these factors influence the effects of online peer feedback on perceived and actual learning.

In research on online peer feedback, students’ characteristics were proposed as an element of a framework guiding the effective implementation of online peer feedback (see Taghizadeh Kerman et al. 2023a). When it comes to the design of peer feedback activities, these are typically the factors that researchers and educators do not manipulate but need to take into account when evaluating the impact of their design i.e. moderators/mediators (Alqassab et al. 2023). In a systematic review of online peer feedback studies in higher education, Taghizadeh Kerman et al. (2023a) identified three types of students’ characteristics that are likely to influence peer feedback namely, demographic characteristics (e.g. gender, language), academic background (e.g. ability, success level, educational level), and personality and psychological features (e.g. self-efficacy, perceptions, epistemic beliefs). In the current study, we examined the impact of success level, educational level, and gender. While various individual variables, including success level, education level, gender, ability, language, and epistemic beliefs can play a role in online peer feedback activities, this study primarily focused on success level, education level, and gender, as these were deemed to have a more direct and significant role for the perception and measurement of learning outcomes (Noroozi et al. 2022; Taghizadeh Kerman et al. 2022; Banihashem et al. 2023). While other variables such as ability, language, and epistemic beliefs are important and could potentially influence the learning process, they were not included in the current study due to limitations in scope and data availability.

Also, recent research indicates that in online peer feedback, students’ domain knowledge and success level significantly impact the feedback they provide and receive, as well as their learning outcomes (Day, Saab, and Admiraal 2022; Taghizadeh Kerman et al. 2022; Valero Haro et al. 2023). Day, Saab, and Admiraal (2022) discovered that students with medium and low abilities improved their presentation skills more than high-ability peers, potentially due to a ceiling effect. Although related to domain-knowledge level, students’ level of success in a learning activity can serve as a better indicator of learning gains from the peer feedback activity. Prior studies of online peer feedback on argumentative essay writing showed that successful students tend to produce higher-quality arguments (Valero Haro et al. 2022) and receive feedback that varies according to their success level (Taghizadeh Kerman et al. 2022). In contrast, educational level (bachelor vs. master) showed no significant effect on performance improvements, although master students reported greater learning satisfaction (Noroozi et al. 2023). This suggests that perceived learning, involving self-evaluation of knowledge gain, may also be influenced by success and educational levels. Regarding peer feedback in academic writing, studies have not established significant relationships between feedback perceptions and performance, or examined the influence of success or educational level on these perceptions (Strijbos, Narciss, and Dünnebier 2010; Huisman et al. 2018). These findings underscore the necessity to explore how students’ success and educational levels affect both perceived and actual learning in online peer feedback contexts. Since prior studies did not find significant relationships between perceived and actual learning (Caspi and Blau 2008; Barzilai and Blau 2014; Porat, Blau, and Barak 2018), we studied this relationship in light of success level, assuming that those who learn more will be more accurate at judging their perceived learning.

Additionally, gender is increasingly recognized as a crucial factor in online learning environments, including peer feedback (Taghizadeh Kerman et al. 2023a). Research on online peer feedback has shown varying outcomes based on gender, with female students excelling in certain aspects of written argumentation (Noroozi et al. 2022; Banihashem et al. 2023), while males showing advantages in others (Noroozi et al. 2020). Moreover, the types of peer feedback provided by males and females seem to differ (Noroozi et al. 2020; Ocampo, Panadero, and Díez 2023). Studies evaluating online courses found that female students often report higher learning perceptions compared to males (e.g. Rovai and Baker 2005). These findings suggest that perceived learning in online peer feedback could be gender-dependent, warranting further investigation.

Alqassab et al. (2023) identified students’ academic major as a potential moderator/mediator, of peer feedback outcomes, however, a very limited number of studies investigated this factor so far, with more studies focusing on its impact on bias in student ratings (e.g. Aryadoust 2016). Planas-Lladó et al. (2018) observed that students enrolled in technology-related majors were less positive towards peer feedback compared to non-technology majors. This revelation, coupled with the limited research on academic majors in peer feedback, underscores the need to explore its impact on perceived versus actual learning.

Aim and research questions

To address the research gaps described above, this study aimed to explore the impact of students’ success level, educational level, gender, and academic major on their perceived learning in an online peer feedback module on argumentative essay writing. The study sought to answer the following research questions:

• RQ1: To what extent does students’ perceived learning differ between successful and less successful students in online peer feedback on argumentative essay writing?

• RQ2: To what extent does perceived learning of bachelor and master students differ in relation to their success level in online peer feedback on argumentative essay writing?

• RQ3: To what extent does perceived learning of female and male students differ in relation to their success level in online peer feedback on argumentative essay writing?

• RQ4: To what extent does perceived learning of students in different academic majors differ in relation to their success level in online peer feedback on argumentative essay writing?

We decided to take an exploratory approach to our research questions since there is a lack of adequate research on the correlation between perceived and actual learning in relation to these variables, and therefore we did not formulate any specific hypotheses.

Method

This exploratory study was part of a large study conducted in the academic year 2020-2021 on designing, implementing, and evaluating online-supported peer feedback to teach argumentative writing for students. The study took place at a medium-sized public university in the Netherlands in the domain of life sciences, especially food and health, sustainability, and the healthy living environment. Findings on peer feedback content in relation to improvement in argumentative writing and other outcomes are published in previous work (Taghizadeh Kerman et al. 2022; 2023b; Noroozi et al. 2023).

Participants and ethical measures

In total, 330 university students participated in this study, however, only 284 students completed all steps of the study. Among the participants, 195 students were female (68%) and 89 students were male (32%), with a mean age of 22 years (range 18-33). In addition, 148 students were undergraduates (52%), while 136 students were graduates (48%). The selected courses were from different academic majors including Social Sciences (N = 56, 20%, Female = 27, Male = 29), Plant Sciences (N = 29, 10%, Female = 20, Male = 9), Health & Social Sciences (N = 47, 16%, Female = 31, Male = 16), Environmental Sciences (N = 101, 36%, Female = 70, Male = 31), and Food Sciences (N = 51, 18%, Female = 37, Male = 14). All participants were informed that their data would be collected and used anonymously. Students were provided with the option to withdraw from the study and request the exclusion of their collected data and gave their consent for the collection and utilization of their data for research purposes. Ethical approval for this research was obtained from the Social Sciences Ethics Committee of the university.

Procedure

A module titled ‘Argumentative Essay Writing’ was developed and integrated into an online platform called Brightspace. This module spanned three consecutive weeks. In the first week, students received an introduction to the module, followed by an assignment of writing an argumentative essay (task 1). The students were provided with three topics to choose one topic from. This approach aimed to minimize potential bias related to students’ domain-specific knowledge of a particular topic, as some students might possess extensive knowledge of a specific subject, while others may not. In the second week, the students were tasked with providing feedback on two essays authored by their peers. They received criteria and prompts associated with each criteria to guide the peer feedback provision. The criteria for providing feedback were integrated into the online environment (task 2) (see Supplementary Online Material 1). Within the Brightspace platform, we used the FeedbackFruits tool to automatically and randomly assign students to review two sets of their peers’ argumentative essays. Finally, in the third week, the students were asked to revise their essays based on the feedback they received from their peers during the previous task (task 3). At the end, students’ perceptions of learning were collected. Details of the characteristics of the peer feedback activity are reported in the Supplementary Online Material 2.

Measurement

Measurement of students’ actual learning

We utilized a rubric developed by Noroozi, Biemans, and Mulder (2016) to assess students’ actual learning in argumentative essay writing. This rubric was constructed based on the key elements of high-quality argumentative essay writing (Toulmin 1958; Noroozi, Biemans, and Mulder 2016). It includes eight essential components: (1) introduction to the topic, (2) taking a position on the topic, (3) arguments for the position, (4) justifications for arguments for the position, (5) arguments against the position (counter-arguments), (6) justifications for arguments against the position, (7) response to counter-arguments, and (8) conclusion and implications.

Each element in the rubric was assessed on a scale ranging from zero (the lowest quality level) to three (the highest quality level) (for the rubric see Noroozi et al. 2023). The total score for the quality of the argumentative essay was obtained by summing up the scores across all these elements. The assessment of students’ argumentative essays was conducted in two stages involving the original essay and the revised version. In this study, the actual learning of the students was measured based on their progress from the original argumentative essay to the revised version. In subsequent analyses, these grades were categorized into two groups: successful and less successful, based on the students’ improvement from the pre-test to the post-test. For the distribution of grades of the original and revised assays refer to Supplementary Online Material 3. The classification of students into successful and less successful groups was based on the median score with successful students falling within 50% and above, and less successful students falling below 50%. The scoring of students’ essays was done by five coders, each possessing expertise in the field of education. To determine the inter-rater reliability among coders, we employed Fleiss’ Kappa statistic (Fleiss 1971). The analysis revealed a substantial level of agreement among the coders, with an overall agreement rate of 75% (Fleiss’ Kappa = 0.75 [95% CI: 0.70-0.81]; z = 26.08; p < 0.001).

Measurement of students’ perceived learning

We measured perceived learning using two subscales which are part of a survey on students’ learning satisfaction used in previous studies with acceptable internal consistency (e.g. Noroozi et al. 2023). The two subscales consisted of a total of 11 items measuring domain-specific (5 items; e.g. “The module broadened my knowledge on the topic”) and domain-general learning (6 items; “The module helped me learn how to offer argumentative peer feedback”). The students responded on a five-point Likert scale, ranging from “almost never true = 1”, “rarely true = 2”, “occasionally true = 3”, “often true = 4”, to “almost always true = 5”. The internal consistency for the two subscales was good, McDonald’s ω = 0.88 and 0.86 respectively.

Data analysis

To investigate students’ actual versus perceived learning experiences, we initially utilized a percentile rank measurement to classify students into two distinct groups based on their progress from the original argumentative essay writing to the revised argumentative essay writing: less successful students (progress below the 50th percentile) (N = 110, 39%) and successful students (exceeded the 50th percentile) (N = 167, 59%). Subsequently, we conducted a one-way ANOVA test to compare the perceived learning experiences between the less successful and the successful students. We conducted two-way ANOVAs to compare the perceived learning experiences with regard to education levels, gender, and academic majors across different success levels (less successful and successful).

Results

RQ1: differences in perceived learning between successful and less successful students

Although, successful students exhibited slightly higher perceptions of their learning experience compared to less successful students, these differences were not statistically significant (F (1, 231) = 0.56, p = 0.45, η² = 0.01) (see Table 1).

Table 1. Differences in perceived learning between less successful and successful students.

Group	Perceived learning experiences (from 1 to 5)
	Mean	SD
Less successful	3.59	0.66
Successful	3.66	0.71
Total	3.64	0.69

RQ2: differences in perceived learning of bachelor and master students in relation to their success

In general, bachelor students scored lower in terms of their perceived learning experiences compared to master students. These differences were statistically significant (F(1, 229) = 8.49, p < 0.01, η² = 0.04). However, the interaction between education level and success level was not statistically significant (F(1, 229) = 0.56, p = 0.45, η² = 0.00) (see Table 2).

Table 2. Differences in perceived learning experiences of less successful and successful bachelor and master students.

Education levels	Group	Perceived learning experiences (from 1 to 5)
		Mean	SD
Bachelor	Less successful	3.39	0.73
	Successful	3.61	0.60
	Total	3.52	0.66
Master	Less successful	3.82	0.50
	Successful	3.71	0.79
	Total	3.75	0.70

RQ3: differences in perceived learning of female and male students in relation to their success level

Female students scored slightly higher in terms of their perceived learning experiences compared to male students. However, these differences were not statistically significant (F(1, 229) = 0.39, p = 0.54, η² = 0.01). The interaction between gender and success level was not statistically significant (F(1, 229) = 0.02, p = 0.86, η² = 0.00) (see Table 3).

Table 3. Differences between perceived learning experiences of less successful and successful female and male students.

Gender	Group	Perceived learning experiences (from 1 to 5)
		Mean	SD
Female	Less successful	3.63	0.56
	Successful	3.67	0.70
	Total	3.65	0.65
Male	Less successful	3.54	0.61
	Successful	3.63	0.89
	Total	3.59	0.79

RQ4: differences in perceived learning of students in different academic majors in relation to their success level

Students in the academic major of food sciences reported slightly higher perceived learning compared to students in other majors. However, these differences were not statistically significant (F(1, 223) = 0.00, p = 0.99, η² = 0.00). Also, the interaction between academic major and success level was not statistically significant (F(1, 223) = 1.36, p = 0.25, η² = 0.01) (see Table 4).

Table 4. Differences between perceived learning experiences of less successful and successful students from different majors.

Education course	Group	Perceived learning experiences (from 1 to 5)
		Mean	SD
Social Sciences	Less successful	3.92	0.59
	Successful	3.67	0.87
	Total	3.74	0.80
Plant Sciences	Less successful	3.75	0.57
	Successful	3.51	0.74
	Total	3.60	0.68
Health & Social Sciences	Less successful	3.50	0.83
	Successful	3.60	0.63
	Total	3.56	0.70
Environmental Sciences	Less successful	3.34	0.69
	Successful	3.61	0.59
	Total	3.50	0.64
Food Sciences	Less successful	3.79	0.34
	Successful	3.93	0.69
	Total	3.87	0.58

Discussion

This study examined how students’ success level, educational level, gender, and academic major influenced their perceived learning experiences in an online peer feedback on argumentative essay writing. Regarding the first research question, we did not find statistically significant differences between successful and less successful students in terms of their perceived learning after the peer feedback module. Although the more successful students improved their writing argumentation skills more, we did not find any evidence for the same pattern in perceived learning. These findings are consistent with previous studies on online learning that did not find a significant relationship between perceived learning and actual performance (Barzilai and Blau 2014; Porat, Blau, and Barak 2018). Although we expected that the successful students would be more accurate in judging their perceived learning than the less successful students, their reported perceived learning after the peer feedback module did not seem to align with their actual learning. This study, indeed, is not the first study on peer feedback showing the mismatch between students’ perceived learning versus actual learning even for students with more gains. In a study of peer feedback provision on mathematics argumentation tasks, even though students with higher levels of domain knowledge improved the type of peer feedback they provided, the perceptions of learning significantly decreased regardless of students’ level of domain knowledge (Alqassab, Strijbos, and Ufer 2018). Caspi and Blau (2008, 2011) asserted that when students evaluate their learning experience, they rely on both cognitive as well as socio-emotional sources. While the cognitive aspect is related to whether knowledge has been acquired, the social-emotional has to do with feelings and experiences such as difficulty, enjoyment, or engagement (Capsi & Blau, 2011). The socio-emotional aspects and not the cognitive aspects of perceived learning were found to be related to perceived learning in online learning (Barzilai and Blau 2014). This can be one plausible explanation for our findings. As our study only measured cognitive perceived learning, we do not know if students relied more or less on socio-emotional aspects to judge their perceived learning especially since they were involved in peer feedback which is a social learning activity.

Regarding the second research question, we found that students at the master level reported significantly higher perceived learning than bachelor students and the difference did not depend on the level of success. Although in the previous study by Noroozi et al. (2023), no impact of education level was found on the improvement of argumentative writing, the master students in that study also had higher perceptions of learning which is consistent with the current study. These findings suggest that students at more advanced levels of higher education might possess better writing argumentation skills. Nevertheless, it is important to stress that no impact of the level of success was observed on perceived learning in our study which might suggest that students in the more advanced stages considered the learning activity as more relevant—which is also reflected in the higher perceived learning—and they ended up being more engaged in the module. Yet, this requires further examination in future studies by measuring socio-emotional aspects of perceived learning (Caspi and Blau 2008) in peer feedback research as we discussed earlier.

In our third research question, we investigated the impact of gender on perceived learning in relation to students’ success levels in online peer feedback. Our findings showed no significant differences between male and female students in terms of their perceived learning and for the successful and the less successful students. Unlike earlier studies of online courses which showed that females had higher perceptions of learning than males (e.g. Rovai and Baker 2005), we did not find statistically significant differences although descriptively females reported slightly higher perceived learning in our online peer feedback module. The previous studies on online peer feedback on argumentative writing that examined gender differences in argumentation skills overall reported no gender differences on most of the criteria of argumentative writing (Noroozi et al. 2022; Banihashem et al. 2023). Together with the results of the current study, these findings seem to support the results of Hsu et al. (2018) who also found no significant differences between females and males. Our observation is that gender differences do not seem to exist when collective measures of argumentation skills are used as is the case in our study. Previous studies only found differences in sub-categories of argumentation skills and not in the overall measure (Hsu et al. 2018; Noroozi et al. 2022; Banihashem etal. 2023).

In our last research question, we explored the impact of academic majors on the differences in perceived learning for successful and less successful students. We found no significant differences across different academic majors. This result is inconsistent with the study by Planas-Lladó et al. (2018) who reported significant differences in perceptions of peer feedback between students who attended technology majors compared to non-technology majors. Notwithstanding, in our study, the majority of students come from natural science majors (e.g. Environmental Sciences, Plant Sciences) or natural science combined with social science (Health & Social Sciences) and thus might have more disciplinarity similarities compared to technology versus non-technology majors. While in our study perceived learning was explored in relation to academic majors, Planas-Lladó et al. (2018) compared broader perceptions of peer feedback including motivation, fairness, and openness to criticism.

Limitations and future directions

This study has several limitations. First, we examined the impact of students’ level of success only in terms of their argumentation skills which can be assumed—based on how the essays were assessed—as domain independent. Students’ level of success can also be measured in terms of gaining domain-specific knowledge, and this was found to be related to argumentation behavior (Valero Haro et al. 2022). Prior studies showed that peer feedback skills require domain-specific knowledge (Van Zundert et al. 2012; Alqassab, Strijbos, and Ufer 2018), which might suggest that the alignment between students’ self-evaluation of learning (i.e. perceived learning) and actual learning in online peer feedback on argumentative essay writing are likely to be influenced by domain-specific knowledge as well. Accordingly, the degree to which perceived and actual learning in peer feedback are more or less influenced by domain-specific and domain-general skills is open to further research.

Second, as discussed earlier, the perceived learning instrument used in our study mainly captured the cognitive aspects of learning, and we do not know how much the socio-emotional dimensions (Caspi and Blau 2008, 2011) contributed to students’ evaluation of their learning in the argumentative essay writing course. Future peer feedback studies should consider measures of perceived learning that align with the complexity of this learning activity by tapping multiple dimensions of learning including cognitive, and socio-emotional aspects.

Third, to measure success level, we divided participants into two groups only (successful and less successful) based on those below and above the 50^th percentile. However, it is important to note that even within these groups, there may still be a wide range of scores that can make it difficult to determine the impact of success on perceived learning.

Fourth, we did not assess how perceived and actual performance differentially relate to other important factors such as epistemic beliefs or learning satisfaction (Banihashem et al. 2023). Fifth, in this study, it is essential to acknowledge the limitation concerning the generalization of the findings, given that the research was conducted at a single university. Therefore, the results should be interpreted within the context of this specific university, and caution should be exercised when attempting to generalize the findings to a broader population.

Finally, our study was cross-sectionally conducted on a skill that is known to be challenging even for higher education students (Ferretti and Graham 2019) using peer feedback activity which is also known to be cognitively demanding for students (Van Zundert et al. 2012). Students’ perceived learning might become more similar to their actual performance with more experience with online peer feedback. Future studies should explore how perceived and actual learning in (online) peer feedback activities evolve, and if with more experience students will become more accurate at assessing their perceived learning.

Research and practical implications

Our findings suggest that relying only on perceived learning to evaluate online peer feedback activities is not sufficient and might indeed be affected by ‘illusions of understanding’ (De Ridder 2022). We believe that both measures can provide valuable insights into students’ experiences in online peer feedback and should both be correspondingly used to evaluate online learning environments. While performance measures provide more objective measures of the impact of the design of the online peer feedback activity, they might miss aspects of the learning experience that motivate students to be engaged in online learning activities (Caspi and Blau 2011). Perceived learning can also contribute to the affective and self-regulation mechanisms of learning, especially when the information is fed back to the students using learning analytics tools for instance (Banihashem et al. 2022). The differences we observed in perceived learning experiences as a function of educational level suggest that bachelor students might need more instructional scaffolding in online peer feedback on argumentative essay courses. Students can benefit from different types of instructional scaffolds in the form of graphical (e.g. diagrams, tables, visualization) or text-based (e.g. scripts, prompts, hints) representations (Valero Haro et al. 2019). Our findings also show that all students need support in terms of raising their awareness about their actual performance and perceived learning in peer feedback so that they can adjust their perceptions accordingly. A learning dashboard with students as end-users can be employed in future studies to achieve this goal (Jivet et al. 2017).

Conclusions

This exploratory study shed light on the role of several individual and contextual characteristics (success level, educational level, gender, and academic major) on perceived learning when students engage in online peer feedback on argumentative essays. We found that educational level influenced perceived learning experiences suggesting that bachelor students might need instructional support to ensure successful peer feedback outcomes. While we did not find a relationship between perceived learning and success level in online peer feedback, this can be a starting point to reconsider how perceived learning is conceptualized and evaluated in peer feedback studies and most importantly if it is a valid measure of learning in peer feedback studies.

Author contributions

Omid Noroozi: Project administration, Conceptualization, Writing- Review & Editing. Maryam Alqassab: Writing- Original Draft. Nafiseh Taghizadeh Kerman: Formal analysis, Methodology, Writing- Original Draft. Seyyed Kazem Banihashem: Supervision, Conceptualization, Writing- Review & Editing. Ernesto Panadero: Writing- Review & Editing.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by the Ministry of Education, Culture, and Science, the Netherlands, the SURF organization, and Wageningen University and Research [funding number: 2100.9613.00. OCW] to the first author.

Notes on contributors

Omid Noroozi

Omid Noroozi is a faculty member in the field of Technology-Enhanced Transformative Learning at Wageningen University and Research, the Netherlands; His research interest are Educational Technology, Collaborative Learning, Peer Learning, Peer Feedback, Artificial Intelligence, and Computer Supported Collaborative Learning.

Maryam Alqassab

Maryam Alqassab is a Postdoctoral Researcher at the Open University of the Netherlands. Her research interests encompass formative (peer) assessment and feedback, technology-enhanced learning, learning design, and learning analytics.

Nafiseh Taghizadeh Kerman

Nafiseh Taghizadeh Kerman holds a PhD in Educational Administration from Ferdowsi University of Mashhad, Iran. Her research focuses on technology-enhanced learning, feedback, assessment, and argumentation.

Seyyed Kazem Banihashem

Seyyed Kazem Banihashem is an Assistant Professor of Educational Technology at Open University of the Netherlands. His research focuses on technology-enhanced learning, learning analytics, assessment, and feedback.

Ernesto Panadero

Ernesto Panadero is the Director at the Centre for Assessment Research, Policy and Practice in Education (CARPE) at Dublin City University. He is also Honorary Professor at the Centre for Research in Assessment and Digital Learning (CRADLE), Deakin University (Australia). His research focuses on self-regulated learning and educational assessment, especially self & peer assessment, teachers’ feedback and the use of rubrics. He is the PI of the Education, Regulated Learning & Assessment (ERLA) research group since its creation in 2016. He received the Erik de Corte 2017, an award given by the EARLI to a promising early career European researcher in the field of Learning and Instruction.