Automated writing evaluation for formative assessment of second language writing: investigating the accuracy and usefulness of feedback as part of argument-based validation

References

• Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003). Help seeking and help design in interactive learning environments. Review of Educational Research, 73, 277–320. doi:10.3102/00346543073003277
   Web of Science ®Google Scholar
• Anthony, L. (2014). AntConc (Version 3.4.3) [Computer Software]. Tokyo: Waseda University. Retrieved from http://www.laurenceanthony.net/
   Google Scholar
• Aryadoust, V. (2013). Building a validity argument for a listening test of academic proficiency. Newcastle: Cambridge Scholars.
   Google Scholar
• Attali, Y. (2004). Exploring the feedback and revision features of Criterion. Paper presented at the National Council on Measurement in Education Annual Meeting, San Diego, CA.
   Google Scholar
• Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18, 5–25. doi:10.1080/0969594x.2010.513678
   Google Scholar
• Bereiter, C., & Scardamalia, M. (1987). The psychology of written composition. Hillsdale, NJ: Lawrence Erlbaum.
   Google Scholar
• Burstein, J., Chodorow, M., & Leacock, C. (2004). Automated essay evaluation: The Criterion online writing service. AI Magazine, 25, 27–36.
   Web of Science ®Google Scholar
• Cárdenas-Claros, M. S., & Gruba, P. (2009). Help options in CALL: A systematic review. CALICO Journal, 27, 69–90.10.11139/cj.27.1
   Google Scholar
• Chan, A. Y. W. (2010). Toward a taxonomy of written errors: Investigation into the written errors of Hong Kong Cantonese ESL learners. TESOL Quarterly, 44, 295–319.10.5054/tq.2010.219941
   Web of Science ®Google Scholar
• Chapelle, C. (2012). Validity argument for language assessment: The framework is simple…. Language Testing, 19, 19–27.
   Web of Science ®Google Scholar
• Chapelle, C. A., & Chung, Y. R. (2010). The promise of NLP and speech processing technologies in language assessment. Language Testing, 27, 301–315. doi:10.1177/0265532210364405.
   Web of Science ®Google Scholar
• Chapelle, C. A., Cotos, E., & Lee, J. Y. (2015). Validity arguments for diagnostic assessment using automated writing evaluation. Language Testing, 32, 385–405. doi:10.1177/0265532214565386
   Web of Science ®Google Scholar
• Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign Language. New York, NY: Routledge.
   Google Scholar
• Chen, C., & Cheng, W. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12, 94–112.
   Web of Science ®Google Scholar
• Chodorow, M., Gamon, M., & Tetreault, J. (2010). The utility of article and preposition error correction systems for English language learners: Feedback and assessment. Language Testing, 27, 419–436.10.1177/0265532210364391
   Web of Science ®Google Scholar
• Chodorow, M., Tetreault, J. R., & Han, N.-R. (2007). Detection of grammatical errors involving prepositions. In F. Costello, J. Kelleher, & M. Volk (Eds.), Proceedinngs of the Fourth ACL-SIGSEM Workshop on Prepositions (pp. 25–30). Prague: Association for Computational Linguistics.
   Google Scholar
• Clauser, B. E., Kane, M. T., & Swanson, D. B. (2002). Validity Issues for performance-based tests scored with computer-automated scoring systems. Applied Measurement in Education, 15, 413–432.10.1207/S15324818AME1504_05
   Web of Science ®Google Scholar
• Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17.10.1016/j.asw.2014.03.006
   Web of Science ®Google Scholar
• Doe, C. (2015). Student interpretations of diagnostic feedback. Language Assessment Quarterly, 12, 110–135. doi:10.1080/15434303.2014.1002925
   Web of Science ®Google Scholar
• Ferris, D. R. (2011). Treatment of error in second language student writing (2nd ed.). Ann Arbor: The University of Michigan Press.
   Google Scholar
• Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32, 365–387. doi:10.2307/356600
   Google Scholar
• Friedl, J. E. (2006). Mastering regular expressions: Powerful techniques for Perl and other tools. Sebastopol, CA: O’Reilly Media.
   Google Scholar
• Han, N.-R., Chodorow, M., & Leacock, C. (2006). Detecting errors in English article usage by non-native speakers. Natural Language Engineering, 12, 115–129. doi:10.1017/S1351324906004190
   Google Scholar
• Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77–89.10.1080/19312450709336664
   Google Scholar
• Kane, M. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.10.1037/0033-2909.112.3.527
   Web of Science ®Google Scholar
• Kane, M. T. (2012). Validating score interpretations and uses. Language Testing, 29, 3–17.10.1177/0265532211417210
   Web of Science ®Google Scholar
• Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.10.1111/jedm.2013.50.issue-1
   Web of Science ®Google Scholar
• Kane, M. T. (2015). Explicating validity. Assessment in Education: Principles, Policy & Practice, 1–14. doi:10.1080/0969594x.2015.1060192
   Web of Science ®Google Scholar
• Lavolette, E., Polio, C., & Kahng, J. (2015). The accuracy of computer-assisted feedback and students’ responses to it. Language Learning & Technology, 19, 50–68. Retrieved from http://llt.msu.edu/issues/june2015/lavolettepoliokahng.pdf
   Google Scholar
• Leacock, C., Chodorow, M., Gamon, M., & Tetreault, J. (2010). Automated grammatical error detection for language learners. Synthesis lectures on human language technologies, 3, 1–134.
   Google Scholar
• Lee, H., Li, J., & Hegelheimer, V. (2012, September). The impact of Criterion on error reduction: A longitudinal study. Paper presented at TSLL conference, Ames, IA.
   Google Scholar
• Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18. doi:10.1016/j.jslw.2014.10.004
   Web of Science ®Google Scholar
• Manchón, R. M. (Ed.). (2011). Learning-to-write and writing-to-learn in an additional language. Amsterdam: John Benjamins.
   Google Scholar
• Mayer, R. E. (2001). Multimedia learning. New York, NY: Cambridge University Press.
   Google Scholar
• Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 63–71. doi:10.1207/s15326985ep3801_8
   Web of Science ®Google Scholar
• Qualtrics Inc. (2015). Qualtrics survey software. Retrieved from www.qualtrics.com
   Google Scholar
• Quinlan, T., Higgins, D., & Wolff, S. (2009). Evaluating the construct-coverage of the e-rater® scoring engine. ETS Research Report Series, i–35. doi:10.1002/j.2333-8504.2009.tb02158.x
   Google Scholar
• Schmidt, R. W. (1994). Deconstructing consciousness in search of useful definitions for applied linguistics. AILA review, 11, 11–26.
   Google Scholar
• Ware, P. (2011). Computer-generated feedback on student writing. TESOL Quarterly, 45, 769–774. doi:10.5054/tq.2011.272525
   Web of Science ®Google Scholar
• Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. Pedagogies: An International Journal, 3, 22–36. doi:10.1080/15544800701771580
   Google Scholar
• Weigle, S. C. (2013a). English as a second language writing and automated essay evaluation. In M. D. Shermis & J. Burstein (Eds.), Handbook of automated essay evaluations: Current applications and new directions (pp. 36–54). New York, NY: Routledge.
   Google Scholar
• Weigle, S. C. (2013b). English language learners and automated scoring of essays: Critical considerations. Assessing Writing, 18, 85–99.10.1016/j.asw.2012.10.006
   Web of Science ®Google Scholar
• Williamson, D. M., Xi, X., & Breyer, F. J. (2012). A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice, 31, 2–13. doi:10.1111/j.1745-3992.2011.00223.x
   Google Scholar