Comparison of e-rater® Automated Essay Scoring Model Calibration Methods Based on Distributional Targets

REFERENCES

• Attali , Y. 2007 . On-the-fly customization of automated essay scoring , Princeton , NJ : Educational Testing Service . (Research report: RR-07-42)
   Google Scholar
• Attali , Y. 2011 . Modified CVA features , Princeton , NJ : Educational Testing Service . (Unpublished internal report)
   Google Scholar
• Attali , Y. and Burstein , J. 2006 . Automated essay scoring with e-rater® V.2 . Journal of Technology, Learning, and Assessment , 4 ( 3 ) Retrieved from www.jtla.org
   Google Scholar
• Attali , Y. and Powers , D. 2009 . Validity of scores for a developmental scale based on automated scoring . Educational and Psychological Measurement , 69 : 978 – 993 .
   Web of Science ®Google Scholar
• Bridgeman , B. , Trapani , C. and Attali , Y. 2012 . Comparison of human and machine scoring of essays: Differences by gender, ethnicity, and country . Applied Measurement in Education , 25 : 27 – 40 .
   Web of Science ®Google Scholar
• Burstein , J. , Braden-Harder , L. , Chodorow , M. , Hua , S. , Kaplan , B. , Kukich , K. and Wolf , S. 1998 . Computer analysis of essay content for automated score prediction: A prototype automated scoring system for GMAT® Analytical Writing Assessment Essays , Princeton , NJ : Educational Testing Service . (Research report: RR-98-15)
   Google Scholar
• Davey , T. 2009 May . Principles for building and evaluating e-rater models Paper presented at the annual meeting of the National Council of Measurement in Education, San Diego, CA
   Google Scholar
• Davey , T. , Higgins , D. , Trapani , C. , Vezzu , S. and Williamson , D. M. 2007 October . e-rater data analysis and model evaluation report for Western Governors University , Princeton , NJ : Educational Testing Service . (Unpublished internal report)
   Google Scholar
• Educational Testing Service . 2011 . Criterion® Online Writing Evaluation Retrieved from http://ericae.net/pare/42~getvn.html
   Google Scholar
• Fleiss , J. L. and Cohen , J. 1973 . The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability . Educational and Psychological Measurement , 33 : 613 – 619 .
   Web of Science ®Google Scholar
• Haberman , S. J. and Sinharay , S. 2008 . Sample-size requirements for automated essay scoring , Princeton , NJ : Educational Testing Service . (Research report: RR-08-32)
   Google Scholar
• Kozlowski , S. W. J. and Hattrup , K. 1992 . A disagreement about within group agreement: Disentangling issues of consistency versus consensus . Journal of Applied Psychology , 77 : 161 – 167 .
   Web of Science ®Google Scholar
• Page , B. 1966 . The imminence of grading essays by computer . Phi Delta Kakkan , 48 : 238 – 243 .
   Google Scholar
• Powers , D. E. , Burstein , J. C. , Chodorow , M. , Fowles , M. E. and Kukich , K. 2000 . Stumping e-rater: Challenging the validity of automated scoring , Princeton , NJ : Educational Testing Service . (Research report: GRE-98-08)
   Google Scholar
• Rudner , L. and Gagne , P. 2001 . An overview of three approaches to scoring written essays by computer . Practical Assessment, Research & Evaluation , 7 ( 26 ) Retrieved from http://pareonline.net/getvn.asp?v=7&n=26
   Google Scholar
• Salton , G. , Wong , A. and Yang , C. S. 1975 . A vector space model for automated indexing . Communications of the ACM , 18 : 613 – 620 .
   Web of Science ®Google Scholar
• Williamson , D. M. 2009 May . A framework for implementing automated scoring Paper presented at the annual meeting of American Educational Research Association and the National Council of Measurement in Education, San Diego, CA
   Google Scholar