Peer assessment in group projects accounting for assessor reliability by an iterative method

Abstract

This study proposes an advanced method to factor in the contributions of individual group members engaged in an integrated group project using peer assessment procedures. Conway et al. proposed the Individual Weight Factor (IWF) method for peer assessment which has been extensively developed over the years. However, most methods associated with IWF use a simple average algorithm which still has limitations in effectively reflecting abnormal situations. Therefore, this study proposes an iterative method for measuring a fair IWF by considering each assessor's reliability. The proposed algorithm considers each assessor's reliability which is evaluated by an analysis of variance of the scores given by the assessor. This relative reliability factor is used to improve the fairness of the IWF and reduce the effect of unfair or unreliable assessors' scores. A comparison with previous methods demonstrates that the proposed method resulted in substantially more fair grades being given to individual students for group projects.

Keywords:

• peer-assessment
• iterative method
• group project
• individual weight factor
• reliability