Evaluating the quality of peer and self evaluations as measures of student contributions to group projects

ABSTRACT

Educators in higher education commonly use peer and self evaluations to help assess student performance on group projects. Although these evaluations provide multiple benefits, many educators are wary of using them due to concerns about their quality. This study addresses three questions debated in the literature regarding the quality of these assessments. How much do students differentiate among peer contributions through their ratings? How reliable are peer ratings? How much agreement exists between peer and self ratings? Although these questions have been addressed to varying degrees in past work, their answers have been far from settled. While many studies focus on just one of the questions, this study’s data make it possible to address all three questions for the same group of students as well as examine each question by student performance level. The evaluations assessed in this study were completed by a large number of students under conditions associated with obtaining more valid and reliable ratings. Overall, the results provide support for using peer and self evaluations to help assess student contributions to group projects. Peer ratings were largely reliable as group members generally agreed on the scores given to their peers. In addition, most students differentiated among group member contributions through their ratings. Students also tended to rate themselves higher than their peers rated them. This study has implications for how peer and self evaluations can be most effectively used by educators to measure student performance in group work.

KEYWORDS:

• Assessment
• evaluation
• Collaborative learning

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. An alternative measure is the standard deviation of scores given by a student to their group members. The correlation between the ranges and standard deviations was 0.978, and the substantive results were the same for both measures.

2. An alternative measure is the standard deviation of scores given to a student by their group members. The correlation between the ranges and standard deviations was 0.976, and the substantive results were the same for both measures. As discussed by Zhang et al. (2008), there are difficulties in using traditional reliability measures (e.g., correlation coefficient) when measuring consensus in peer ratings of group members.

3. A paired-samples t-test indicated that the scores were significantly higher when given by students to themselves than when given to students by peers (p < 0.001, d = 0.56).

4. Independent-samples t-tests indicate that the point spreads given to peers by students were significantly higher for higher performers than for average performers (p < .001, d = 0.71) and lower performers (p < .001, d = 0.68).

5. Independent-samples t-tests indicate that the ranges in scores were significantly lower for average performers than for lower performers (p < .001, d = 0.74) and higher performers (p < .01, d = 0.51).

6. Paired-samples t-tests indicate that the scores given by students to themselves were significantly higher than scores given to them by peers among lower performers (p < .001, d = 1.10), average performers (p < .01, d = 0.60), and higher performers (p < .05, d = 0.41).

7. Independent-samples t-tests indicate that the difference between self and peer scores was significantly higher for lower performers than for average performers (p < .001, d = 0.71) and higher performers (p < .01, d = 0.62).