Student Experience of Peer Assessment on an MSc Programme

Abstract

The validity of peer assessment as well as students experience and understanding of the process are recurrent themes in the educational literature. We studied postgraduates’ experience of summative peer assessment, in particular their perceptions of the process. Student experience was generally positive, and not affected by characteristics such as first language (English or not), area of origin (European Union versus international), previous degree (medical versus science), or current academic performance in the MSc class. Several key themes emerged and are discussed further with possible implications for both practice and future research.

Keywords:

• Peer assessment
• student experience

Introduction

Various studies have shown that, used effectively, peer assessment may help students develop a number of key skills including critical thinking (Li and Steckelberg 2004), the ability to identify and articulate comments on both good and bad practice (Davies 2000, Harris 2006) and the development of self-reliant and self-directed learning (Sivan 2000, Orsmond 2004). It may also give students a greater awareness of what constitutes a ‘high-level performance’ (Gibbs 1999). Race (2001) maintains that ‘when it comes to measuring individuals relative contribution to group work, the only people who really know what the relative contributions are, are the students themselves’ an aspect which highlights that peer assessment is not merely a easy option for tutors. Todd and Hudson (2007) found students were motivated to learn and participate when they anticipated being graded on the quality of the peer assessment. However, Holroyd (2000) found that students doubted each other’s ability to make valid assessments of peers, a view which has been expressed by teaching staff as well. Additionally, simply making comments on, or grading, others work does not necessarily promote development of the requisite skills (Althauser and Darnall 2001, Sadler and Good 2006, Li et al. 2010).

Having used summative peer assessment as part of the integral course philosophy in a one year taught masters (MSc) programme in Medical Genetics for several years (Fig. 1 and Table 1), we were aware from free text responses on course evaluation questionnaires that a number of students appeared to feel peer review was inaccurate and unfair. However, our own experience, as well as previous studies, have shown that students can be good assessors of each others work (Falchikov and Goldfinch 2000, Sahin 2008). We therefore decided to investigate in a more structured manner the perceptions of the entire class, to clarify whether or not our views of what students gained from the experience were similar to student perceptions. Using anonymous questionnaires our study aimed to gather feedback from students relating to their experience of the peer assessment process.

Figure 1 Overview of the ‘Case Investigations’ and peer assessment. This component represents 30 of the 180 credits required for the MSc in Medical Genetics. Students are given extensive guidance before undertaking three case studies (each lasting two weeks) and producing reports of ~ 50 pages. The grades for these reports are divided amongst the students in the group, weighted by peer assessment that students undertake on each other. It is the students’ experience of the peer assessment part of the process that is addressed in this study.

Table 1 Example of peer marking

		Marks awarded to:		Average
A	B	C	D	E	F	G	H
Marks from:	A	18	16.25	17.25	13.25	10.75	15.75	18.25	17.75	15.9
B	17.75	17.25	16.25	9.75	9	14.25	18	17	14.9
C	19.5	18.75	19.25	15.75	16.5	18.25	20.5	19.25	18.5
D	17.75	15.5	17.25	16.5	12.5	15	18	16.5	16.1
E	19.75	12^†	14.5	11.5	12	13	15	18	14.5
F	18.5	16.75	17	13.75	13.5	15.75	18.25	17.75	16.4
G	19.75	17	17	10.5	11.25	14.5	21.25	20.5	16.5
H	18.5	17	16.75	12.25	12	15.75	18.25	17.5	16.0
Average		18.8	16.2	16.6	12.4	12.2	15.2	18.1	18.1	15.9
Adjusted		19.9	17.3	17.7	13.5	13.3	16.3	19.2	19.2	17.0

Note: Each student awards grades to their peers under four categories on a 22 point scale. They also grade their own performance. The average of the peer marks for each student (disregarding the mark they gave themselves) is calculated. Since the average of these combined peer marks is 15.9 (1.1 marks lower than the staff grade for the three written reports, which was 17.0), each student’s grade is adjusted by 1.1 so that the marks awarded by students reflect the overall grade awarded by staff for the group. Self-marks are not used in the analysis.

† Where marks are significantly different from the other peer marks this is noted and may be taken into account at the examiners meeting if a students grade is at a borderline.

Aims of Study

• To investigate whether any particular student characteristics such as international/home status, English as first language or not and previous type of degree (BSc or other), performance in other assessments, had an impact on students’ experiences of peer assessment.

• To explore the respondent’s experiences of peer assessment more generally.

Method

The sample size was predetermined by the size of the MSc class (40 students). In the week after the peer assessments were submitted students completed a questionnaire asking them about their experiences. There were both scale and free text answer questions. The questions, which required a Likert-scale response (example given for first question), were:

• Did you feel you received adequate instruction on how to review your peers?

  (1) No, not at all, I was completely unclear about what was expected, (2) No, was slightly unsure of what to do, (3) Neutral, (4) Yes, was reasonably sure of what to do, (5) Yes was very happy with the instruction received.

• How ‘fair’ do you think peer assessment is?

• Would you recommend this type of assessment to a lecturer who was planning a new MSc?

• How useful do you think the skill of peer assessment will be to you in your future career?

Identifying information was removed, leaving only key information: previous degree, nationality, English as first language or not and a rough estimate as to their final position in the MSc class and in the Case Investigations component (first, second, third or fourth quartiles). Data were analysed using non-parametric statistical analysis. The free comments gathered from the questionnaire were analysed in order to summarise emergent themes.

Results

Of the 40 students 18 responded to the questionnaires (45%). The response was generally positive (Fig. 2) with 13/18 students either ‘recommending’ or ‘strongly recommending’ this type of assessment. A further two students were unsure and three would not recommend it. Interestingly, 16/18 students thought that peer assessment would be useful or very useful in their future career including three students who were unsure/would not recommend it in a new MSc course. Also, 10/18 students perceived peer assessment as being ‘fair’ or ‘very fair’, four were neutral, and four thought it was ‘unfair’ or ‘very unfair’. Perhaps most enlightening was the fact that only 9/18 students felt clear about what was required of them in the peer assessment process while five were neutral and four felt unclear or very unclear.

Figure 2 Responses to ‘Experience of Peer Assessment’ Questionnaire. Overall responses are shown for the questions: (A) Did you feel you received adequate instruction on how to review your peers?; (B) How ‘fair’ do you think peer assessment is?; (C) Would you recommend this type of assessment to a lecturer who was planning a new MSc?; (D) How useful do you think the skill of peer assessment will be to you in your future career?

Statistical analysis was carried out to determine whether any particular student characteristics had an impact on students’ experiences of peer assessment (Table 2 and Fig. 3).

Table 2 Perception of peer assessment according to first language, fee status and first degree.

	First Language		Fee status		First Degree
English	Other	UK	EU	OS	BSc	MBChB
Understanding?	3.2	3.5	3.4	3.0	3.5	3.3	3.6
p = 0.31 (MW)	p = 0.20 (KW)	p = 0.31 (MW)
Fairness?	3.6	3.2	3.4	3.0	3.3	3.2	3.6
p = 0.83 (MW)	p = 0.30 (KW)	p = 0.83 (MW)
Recommend?	3.6	3.7	3.7	5.0	3.3	3.5	4.0
p = 0.57 (MW)	p = 0.46 (KW)	p = 0.81 (MW)
Usefulness?	4.5	4.0	4.2	5.0	4.5	4.3	4.2
p = 0.37 (MW)	p = 0.67 (KW)	p = 0.37 (MW)

Note: Figures represent mean scores of answers where 1 was a very negative response, 3 was neutral and 5 was very positive; the four questions related to how well students understood the peer assessment process, how fair they thought peer assessment was, whether they would recommend peer assessment, and whether they thought that peer assessment was a useful skill to learn. Non-parametric tests (KW = Kruskal–Wallis analysis and MW = Mann–Whitney analysis) were used for comparison between groups. UK: United Kingdom; EU: European Union; OS: overseas.

Figure 3 Responses to questionnaire grouped according to performance in A: Case Investigations and B: the overall MSc. Figures represent mean scores of answers where 1 was a very negative response, 3 was neutral and 5 was very positive; the four questions related to how well students understood the peer assessment process, how fair they thought peer assessment was, whether they would recommend peer assessment, and whether they thought that peer assessment was a useful skill to learn. Correlation coefficients (R values) show the strength of the correlation between the quartile the student was in and the response to the question (R values in the range 0–0.5 are weak correlations while values of between 0.5–1 demonstrate a stronger relationship between answers to the questions and quartile. Negative R values demonstrate a negative relationship between answers to the questions and quartile). p-Values show the significance of this correlation.

Firstly we also wanted to determine if factors such as international/home status, English as first language or not and previous type of degree (BSc or other) affected student responses to the questions. Using between-groups analysis (Kruskal–Wallis and Mann–Whitney) we found no significant differences between any of the groups when it came to understanding, perceived fairness, and likelihood to recommend peer assessment or expectation of its usefulness (Table 2).

A number of further questions explored the respondent’s experiences more generally. The responses to these questions were divided into subtopics and recurrent themes are shown in Table 3.

Table 3 Thematic analysis of free text responses on the subject of student experience of peer assessment

Expectations and understanding	Apprehension about the process:‘A little sceptical about fairness’ ‘Would be difficult to get a fair assessment of a person’s ability from peer assessment – personality conflicts and issues may get in the way’ ‘I thought I cannot say bad things about other people’; it was ‘a different idea … new way of grading our work’	Positive attitudes to the idea despite having no prior exposure: ‘I thought it would be interesting and different from what I am used to’ ‘I felt it to be an effective way of gauging my performance’	Confusion about the process: ‘At first I was a bit confused as it was new to me but then when it was worked out in the mock investigation it became clear’ ‘I understood what I had to do but was confused about the marking system’	Adequacy of instruction: ‘We were provided with all the information required and even practice examples’ ‘Was not sure others read the instructions properly’
Positive and negative aspects of being the reviewer	Empowerment: ‘I was able to voice my opinions about other members of my group, whether praising or being critical’ ‘I got the chance to assess my peers and express my opinion on the group work, that most times would not be taken into account’	Development of skills: ‘I learnt a lot on how to grade an individual taking into account various factors into consideration’ ‘I learned how to criticise people to help them improve their performance’	Difficulty in giving constructive criticism: ‘Sometimes it was hard to comment about the drawbacks of others as we are not too sure if that is harsh or not’ ‘I needed to fully analyse their performance and think about them in depth before. That was an interesting challenge’	Insecurities about marking: ‘Sometimes they mark based on relationships and not the actual work’ ‘Marking was based on a popularity contest’
Staff versus students as reviewers	Students clearer understanding: ‘Fellow students can recognise traits that are missed by staff as the staff might not understand fully the background psychological games’ ‘Genuine feedback on working with people in a group especially as the contact with the group was greater than the contact with staff’ ‘Other students realised how much work was involved’	Insecurities about peer assessment: ‘I prefer the impartiality of staff marking’ ‘Group work is great but when it comes to marks, the ugly side of humans emerges’ ‘Peer review is very effective if done constructively and as objectively as possible, however I’m not entirely convinced that this can always be the case’	Insecurities about peer assessment (cont): ‘I do not like to depend on my peers to determine my grades. It feels unsafe to have my fate lying in the hands of peers’ ‘The people who were not pleased with my behaviour can reduce my marks’ ‘Personally I don’t want to peer review students or to receive it from them. I will anyway do it in job environment. In the course I just want to get knowledge without wasting my nerves’	Appreciation of critical review: ‘When given a good grade it was nice to know my peers appreciated my effort and felt that I was a valuable group member’ ‘Really helped me understand how people perceived me and my work – this was useful in learning how to work better in a team’ ‘I was not quite satisfied about the negative aspects that were said as I wanted … to be more detail so that it would allow me to improve better’ ‘a great way to know yourself better’
Future usefulness	Preferences for being assessed: ‘I would like to be assessed by people with whom I work with and those that see you working’ ‘… by people who know the field better than me, in this case by our teachers’	Why peer assessment is useful: ‘Important to have a realistic opinion of yourself and how you function with others in a working environment’	Skill Development: ‘In future whether a job or PhD, working in a group is what happens. So peer review helped me to interact with people and improved my communication skills’	‘Soft Skills: Patience, effective communication skills, team work (as no man can do all things by himself) and the level of my emotional quotient has levelled up. Critical thinking and feedback skills’ ‘Learned teamwork, trust’

Note: Student comments were grouped into the themes of ‘Expectations and understanding’, ‘Positive and negative aspects of being the reviewer’, ‘Staff versus students as reviewers’ and ‘Future usefulness’. The comments in the table are representative of the range of views expressed.

Discussion

Perhaps the most enlightening aspect of this study is that background of the student does not influence their experience of peer assessment significantly. Sivan (2000) found that adult learners were more positive than school leavers in regard to all aspects of the peer assessment process. In the present study, in which all students had completed at least one prior university degree, and are thus closer to Sivan’s ‘adult learners’, we have ascertained that student perceptions of the fairness and benefits of peer assessment are not influenced by their level of academic performance, i.e. the less well performing students are as likely as the better performing students to perceive peer assessment as a valuable process.

One interesting finding from this research was that there are a significant minority of students who feel they are unprepared for peer assessment and that this was the only aspect which approached significance, i.e. students who were further down the class rankings tended to have difficulties with understanding what was required of them. The concept of ‘adequate preparation’ as fundamental for the success of peer assessment has been raised in several other studies (Jordan 1999, Boud et al. 2001, Orsmond 2004, Vu and Dall’Alba 2007). From their review of the literature, Van Zundert et al. (2010) suggested that didactic sessions outlining peer review are not nearly as effective as engaging students in discussion of the practice while they are actually undertaking it. Although we have a ‘mock’ case investigation with peer assessment, the very fact that this is not a formally assessed exercise means that many students do not engage with it fully. We have now introduced more group discussion of the requirements of peer assessment to address the problem.

The perception that students may not appreciate being graded by those at the same level as them was not generally supported by the current study; in contrast students felt they were in a better position to judge the contributions made by their team members since most of the group activity takes place without the presence of staff (Table 3). Some did raise the issue of perceived fairness of peer assessment; it is possible that this relates to a tendency for less able students to overrate themselves (Boud and Falchikov 1989). By using grade input from an entire group the effect of an occasional ‘outlier’ grade is diluted (Table 1), an effect also noted by others (Orsmond 2004, and the case study described therein by Cogdell et al.). In general we observe good correlation between staff versus student assessments, although students tend to grade each other more generously than staff, trends which have been observed in other studies (see for example the meta-analysis by Falchikov and Goldfinch, 2000). Thus available evidence suggests that students do in fact know how to assess each other accurately. How to convey this to students is another issue, and one to which there is unlikely to be an easy solution.

When the class was analysed as a whole, students on average performed two points higher (on a 22 point scale) on Case Investigations than in the other components of the MSc programme. To an extent, though, this higher grading does reflect staff observations that the high-performing students in a group have a tendency to over-compensate for the poorer performance of some of their peers by contributing extra work.

It is clear from the literature that academics appreciate the potential benefits of peer assessment in preparing students for many aspects of their future careers (Orsmond 2004), so it was encouraging from our study to find that the majority of students seem to perceive peer assessment as a useful skill, irrespective of their background or performance level within the class; this is perhaps as a result of the emphasis on team working being a requisite skill for almost all forms of employment.

The comments summarised in Table 3 provide a fascinating insight into students’ perceptions and experiences. One aspect worthy of consideration is the extent to which a student’s efforts to develop and demonstrate good team-working skills reflects the knowledge that their peers will assess them on these attributes, since (as reviewed by Al-Kadri et al. 2012) students clearly adapt their approaches to tasks in order to try to succeed in the assessments.

Follow up to this study is ongoing. In particular, we would like to follow a cross-section of a class from before the summative Case Investigations begin, through to its conclusion, and use focus groups to explore more fully the issues raised in the thematic analysis reported here. It is possible that this could be done using students as co-researchers in order to try to gain more complete, unbiased insights.

Acknowledgements

With thanks to Douglas Marks for advice on statistics and to Caitlin Welsh for administrative assistance.