Accountability in peer assessment: examining the effects of reviewing grades on peer ratings and peer feedback

References

• Babik, D., E. F. Gehringer, J. Kidd, F. Pramudianto, and D. Tinapple. 2016, June. “Probing the Landscape: Toward a Systematic Taxonomy of Online Peer Assessment Systems in Education.” Paper presented at the Workshop at Educational Data Mining (EDM), Raleigh, NC.
   Google Scholar
• Balfour, S. P. 2013. “Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer Review TM.” Research & Practice in Assessment 8: 40–48.
   Google Scholar
• Bruffee, K. A. 1980. A Short Course in Writing: Practical Rhetoric for Composition Courses, Writing Workshops, and Tutor Training Programs. Boston, MA: Little, Brown.
   Google Scholar
• Cho, K., and C. MacArthur. 2011. “Learning by Reviewing.” Journal of Educational Psychology 103 (1): 73–84. doi:10.1037/a0021950.
   Web of Science ®Google Scholar
• Cho, K., and C. D. Schunn. 2007. “Scaffolded Writing and Rewriting in the Discipline: A Web-based Reciprocal Peer Review System.” Computers & Education 48 (3): 409–26. doi:10.1016/j.compedu.2005.02.004.
   Web of Science ®Google Scholar
• Cho, K., C. D. Schunn, and D. Charney. 2006. “Commenting on Writing: Typology and Perceived Helpfulness of Comments from Novice Peer Reviewers and Subject Matter Experts.” Written Communication 23 (3): 260–94. doi:10.1177/0741088306289261.
   Web of Science ®Google Scholar
• Cho, K., C. D. Schunn, and R. W. Wilson. 2006. “Validity and Reliability of Scaffolded Peer Assessment of Writing from Instructor and Student Perspectives.” Journal of Educational Psychology 98 (4): 891–901. doi:10.1037/0022-0663.98.4.891.
   Web of Science ®Google Scholar
• de Alfaro, L., and M. Shavlovsky. 2014, March. “CrowdGrader: A Tool for Crowdsourcing the Evaluation of Homework Assignments.” Paper presented at the 45th ACM Technical Symposium on Computer Science Education, Atlanta, GA.
   Google Scholar
• Denny, P., A. Luxton-Reilly, and J. Hamer. 2008. “The PeerWise System of Student Contributed Assessment Questions.” Paper presented at the Tenth Conference of Australasian Computing Education (ACE 2008), Wollongong, Australia.
   Google Scholar
• Elbow, P. 1973. Writing Without Teachers. New York: Oxford University Press.
   Google Scholar
• Evans, J. S. B. T. 2011. “Dual-process Theories of Reasoning: Contemporary Issues and Developmental Applications.” Developmental Review 31: 86–102. doi:10.1016/j.dr.2011.07.007.
   Web of Science ®Google Scholar
• Falchikov, N., and J. Goldfinch. 2000. “Student Peer Assessment in Higher Education: A Meta-analysis Comparing Peer and Teacher Marks.” Review of Educational Research 70 (3): 287–322. doi:10.3102/00346543070003287.
   Web of Science ®Google Scholar
• Gehringer, E. F. 2014. “A Survey of Methods for Improving Review Quality.” In New Horizons in Web Based Learning: ICWL 2014 International Workshops, SPeL, PRASAE, IWMPL, OBIE, and KMEL, FET, Tallinn, Estonia, August 14-17, 2014, Revised Selected Papers, edited by Y. Cao, T. Väljataga, J. K. T. Tang, H. Leung, & M. Laanpere, 92–97. Springer International.
   Google Scholar
• Grabill, J., B. Hart-Davidson, and M. McLeod. 2012. “Whitepaper: Eli Review.” http://elireview.com/wp-content/uploads/2014/12/elireview_whitepaper_final.pdf.
   Google Scholar
• Hamer, J., K. T. K. Ma, and H. H. F. Kwong. 2005. “A Method of Automatic Grade Calibration in Peer Assessment.” Paper presented at the Seventh Australasian Computer Science Education Conference (ACE’2005), Newcastle, Australia.
   Google Scholar
• Hartberg, Y., A. B. Gunersel, N. J. Simspon, and V. Balester. 2008. “Development of Student Writing in Biochemistry Using Calibrated Peer Review.” Journal of the Scholarship of Teaching and Learning 2 (1): 29–44.
   Google Scholar
• Kaufman, J., and C. Schunn. 2011. “Students’ Perceptions About Peer Assessment for Writing: Their Origin and Impact on Revision Work.” Instructional Science 39 (3): 387–406. doi:10.1007/s11251-010-9133-6.
   Web of Science ®Google Scholar
• Lu, J., and N. Law. 2012. “Online Peer Assessment: Effects of Cognitive and Affective Feedback.” Instructional Science 40 (2): 257–75. doi: 10.1007/s11251-011-9177-2
   Web of Science ®Google Scholar
• Luxton-Reilly, A. 2009. “A Systematic Review of Tools that Support Peer Assessment.” Computer Science Education 19 (4): 209–32. doi: 10.1080/08993400903384844
   Google Scholar
• Moffett, J. 1968. Teaching the Universe of Discourse. Boston, MA: Houghton Mifflin Company.
   Google Scholar
• Nguyen, H. V., and D. Litman. 2014, June. “Improving Peer Feedback Predictions: The Sentence Level Is Right.” Paper presented at the 9th Workshop on Innovative Use of NLP for Building Educational Applications (BEA), ACL 2014 Workshop, Baltimore, MD.
   Google Scholar
• Nguyen, H. V., W. Xiong, and D. Litman. 2014, June. “Classroom Evaluation of a Scaffolding Intervention for Improving Peer Review Localization.” Paper presented at the 12th International Conference on Intelligent Tutoring Systems (ITS), Honolulu, HI.
   Google Scholar
• Patchan, M. M., D. Charney, and C. D. Schunn. 2009. “A Validation Study of Students’ End Comments: Comparing Comments by Students, a Writing Instructor, and a Content Instructor.” Journal of Writing Research 1 (2): 124–52. doi:10.17239/jowr-2009.01.02.2.
   Google Scholar
• Patchan, M. M., C. D. Schunn, and R. J. Clark. 2011. “Writing in Natural Sciences: Understanding the Effects of Different Types of Reviewers on the Writing Process.” Journal of Writing Research 2 (3): 365–93. doi:10.17239/jowr-2011.02.03.4.
   Google Scholar
• Piech, C., J. Huang, Z. Chen, C. Do, A. Ng, and D. Koller. 2013, July. “Tunes Models of Peer Assessment in MOOCs.” Paper presented at the 6th International Conference on Educational Data Mining (EDM 2013), Memphis, TN.
   Google Scholar
• Ramachandran, L. 2013. “Automated Assessment of Reviews.” Doctoral Diss., North Carolina State University, Raleigh, NC.
   Google Scholar
• Russell, A. A. 2004. “Calibrated Peer Review: A Writing and Critical-Thinking Instructional Tool.” In Invention and Impact: Building Excellence in Undergraduate Science, Technology, Engineering and Mathematics (STEM) Education, 67–71. Washington, DC: American Association for the Advancement of Science.
   Google Scholar
• Schoenfeld, A. H. 2007. “What Is Mathematical Proficiency and How Can It Be Assessed?” Assessing Mathematical Proficiency 53: 59–74. doi: 10.1017/CBO9780511755378.008
   Google Scholar
• Schunn, C. D. 2016. “Writing to Learn and Learning to Write Through SWoRD.” In Adaptive Educational Technologies for Literacy Instruction, edited by S. A. Crossley and D. S. McNamara, 243–59. New York: Taylor & Francis, Routledge.
   Google Scholar
• Shute, V. J. 2008. “Focus on Formative Feedback.” Review of Educational Research 78 (1): 153–89. doi:10.3102/0034654307313795.
   Web of Science ®Google Scholar
• Song, Y., Z. Hu, and E. F. Gehringer. 2015, October. “Pluggable Reputation Systems for Peer Review: A Web-service Approach.” Paper presented at the IEEE Frontiers in Education Conference (FIE), El Paso, TX.
   Google Scholar
• Song, Y., Z. Hu, E. F. Gehringer, J. Morris, J. Kidd, and S. Ringleb. 2016, June. “Toward Better Training in Peer Assessment: Does Calibration Help?” Paper presented at the Computer-Supported Peer Review in Education (EDM workshop), Raleigh, NC.
   Google Scholar
• Tinapple, D., L. Olson, and J. Sadauskas. 2013. “CritViz: Web-based Software Supporting Peer Critique in Large Creative Classrooms.” Bulletin of the IEEE Technial Committee on Learning Technology 15 (1): 29–35.
   Google Scholar
• Topping, K. J. 2005. “Trends in Peer Learning.” Educational Psychology 25 (6): 631–45. doi:10.1080/01443410500345172.
   Google Scholar
• Topping, K. J., R. Dehkinet, S. Blanch, M. Corcelles, and D. Duran. 2013. “Paradoxical Effects of Feedback in International Online Reciprocal Peer Tutoring.” Computers & Education 61: 225–31. doi:10.1016/j.compedu.2012.10.002.
   Web of Science ®Google Scholar
• Wooley, R. S., C. Was, C. D. Schunn, and D. Dalton. 2008. “The Effects of Feedback Elaboration on the Giver of Feedback.” Paper presented at the Cognitive Science, Washington DC.
   Google Scholar
• Xiong, W., D. Litman, and C. Schunn. 2012. “Natural Language Processing Techniques for Researching and Improving Peer Feedback.” Journal of Writing Research 4 (2): 155–76. doi: 10.17239/jowr-2012.04.02.3
   Google Scholar