Depth-Perception-Based Representation in Holistic Rating on ESL Essay Writing

Abstract

This paper proposes to use depth perception to represent raters’ decision in holistic evaluation of ESL essays, as an alternative medium to conventional form of numerical scores. The researchers verified the new method’s accuracy and inter/intra-rater reliability by inviting 24 ESL teachers to perform different representations when rating 60 essays written by Chinese ESL learners. During numerical representation, the raters expressed their evaluation in the form of numbers, while with the DP-based method, the raters expressed their evaluation by marking distances with nail tags on a wood ruler with hidden scale marks. Then the researchers translated the distance results into numbers and compared the accuracy and inter/intra-rater consistency of the two approaches by referring to these essays’ criteria scores from eight expert raters. The results showed that DP-based method could improve raters’ performance, producing more similar results to experts’ scores, with higher consistency both among different raters and within the same individual rater.

Author contributions

Lian Li: conception, investigation, literature review, data processing, composing the article. Jiehui Hu: methodology, literature review, conclusions, and limitations. Ping Zhou: literature review, data curation, rater training. Yu Dai: conception, methodology, rater training, experiment management, writing-original draft preparation. Wanhong Zhang: data processing and verification, methodology.

Disclosure statement

No potential conflict of interest was reported by the author(s).