3,334
Views
21
CrossRef citations to date
0
Altmetric
Review

A review of AWE feedback: types, learning outcomes, and implications

ORCID Icon, ORCID Icon, ORCID Icon & ORCID Icon

References

  • Bahari, S. F. (2010). Qualitative versus quantitative research strategies: Contrasting epistemological and ontological assumptions. Technology Journal, 52, 18–26.
  • Bailey, S. (2014). Academic writing: A handbook for international students. Routledge.
  • Bitchener, J., & Ferris, D. (2012). Written corrective feedback in second language acquisition and writing. Taylor & Francis.
  • Bruton, A. (2009). Designing research into the effects of grammar correction in L2 writing: Not so straightforward. Journal of Second Language Writing, 18(2), 136–140. https://doi.org/10.1016/j.jslw.2009.02.005
  • Calvo, R. A., & Ellis, R. A. (2010). Students’ conceptions of tutor and automated feedback in professional writing. Journal of Engineering Education, 99(4), 427–438. https://doi.org/10.1002/j.2168-9830.2010.tb01072.x
  • Chen, X., Zou, D., Cheng, G., & Xie, H. (2020a). Detecting latent topics and trends in educational technologies over four decades using structural topic modeling: A retrospective of all volumes of Computers & Education. Computers & Education, 151, 103855. https://doi.org/10.1016/j.compedu.2020.103855
  • Chen, X., Zou, D., & Xie, H. (2020b). Fifty years of British Journal of Educational Technology: A topic modeling based bibliometric perspective. British Journal of Educational Technology, 51(3), 692–708. https://doi.org/10.1111/bjet.12907
  • Chen, X., Zou, D., Xie, H., & Cheng, G. (2021a). Twenty years of personalized language learning. Educational Technology & Society, 24(1), 205–222.
  • Chen, X., Zou, D., Xie, H., & Su, F. (2021b). Twenty-five years of computer-assisted language learning: A topic modeling analysis: Twenty-five years of CALL. Language Learning & Technology, 25(3), 151–185.
  • Chen, X., Zou, D., Xie, H., & Wang, F. L. (2021c). Past, present, and future of smart learning: A topic-based bibliometric analysis. International Journal of Educational Technology in Higher Education, 18(1), 1–29. https://doi.org/10.1186/s41239-020-00239-6
  • Cook, D. A., & Hatala, R. (2015). Got power? A systematic review of sample size adequacy in health professions education research. Advances in Health Sciences Education: Theory and Practice, 20(1), 73–83. https://doi.org/10.1007/s10459-014-9509-5
  • Cotos, E. (2014). Genre-based automated writing evaluation for L2 research writing: From design to evaluation and enhancement. Palgrave Macmillan.
  • Denham, A. R. (2019). Using the PCaRD digital game-based learning model of instruction in the middle school mathematics classroom: A case study. British Journal of Educational Technology, 50(1), 415–427. https://doi.org/10.1111/bjet.12582
  • Devitt, A. (2004). Writing genres. Southern Illinois University Press.
  • Duman, G., Orhon, G., & Gedik, N. (2015). Research trends in mobile assisted language learning from 2000 to 2012. ReCALL, 27(2), 197–216. https://doi.org/10.1017/S0958344014000287
  • El Ebyary, K., & Windeatt, S. (2010). The impact of computer-based feedback on students’ written work. International Journal of English Studies, 10(2), 121–142. https://doi.org/10.6018/ijes/2010/2/119231
  • Fatawi, I., Degeng, I. N. S., Setyosari, P., Ulfa, S., & Hirashima, T. (2020). Effect of online-based concept map on student engagement and learning outcome. International Journal of Distance Education Technologies (IJDET), 18(3), 42–56.
  • Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109. https://doi.org/10.3102/00346543074001059
  • Graesser, A. C., McNamara, D. S., & Louwerse, M. M. (2003). What do readers need to learn in order to process coherence relations in narrative and expository text? In A. P. Sweet & C. E. Snow (Eds.), Rethinking reading comprehension (pp. 82–98). Guilford.
  • Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers: A Journal of the Psychonomic Society, Inc, 36(2), 193–202. https://doi.org/10.3758/bf03195564
  • Graham, S., Hebert, M., & Harris, K. R. (2015). Formative assessment and writing: A meta-analysis. The Elementary School Journal, 115(4), 523–547. https://doi.org/10.1086/681947
  • Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment, 8(6), 4–43.
  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
  • Hoang, G. T. L., & Kunnan, A. J. (2016). Automated essay evaluation for English language learners: A case study of MY Access. Language Assessment Quarterly, 13(4), 359–376. https://doi.org/10.1080/15434303.2016.1230121
  • Hsu, Y. C., Ho, H. N. J., Tsai, C. C., Hwang, G. J., Chu, H. C., Wang, C. Y., & Chen, N. S. (2012). Research trends in technology-based learning from 2000 to 2009: A content analysis of publications in selected journals. Educational Technology & Society, 15(2), 354–370.
  • Hung, H.-T., Yang, J. C., Hwang, G.-J., Chu, H.-C., & Wang, C.-C. (2018). A scoping review of research on digital game-based language learning. Computers & Education, 126, 89–104. https://doi.org/10.1016/j.compedu.2018.07.001
  • Hwang, G.-J., & Fu, Q.-K. (2019). Trends in the research design and application of mobile language learning: A review of 2007–2016 publications in selected SSCI journals. Interactive Learning Environments, 27(4), 567–581. https://doi.org/10.1080/10494820.2018.1486861
  • In’Nami, Y., & Koizumi, R. (2010). Database selection guidelines for meta-analysis in applied linguistics. TESOL Quarterly, 44(1), 169–184. https://doi.org/10.5054/tq.2010.215253
  • Kellogg, R. T., Whiteford, A. P., & Quinlan, T. (2010). Does automated feedback help students learn to write? Journal of Educational Computing Research, 42(2), 173–196. https://doi.org/10.2190/EC.42.2.c
  • Leontjev, D. (2014). The effect of automated adaptive corrective feedback: L2 English questions. Apples-Journal of Applied Language Studies, 8(2), 43–66.
  • Lin, H. C., & Hwang, G. J. (2019). Research trends of flipped classroom studies for medical courses: A review of journal publications from 2008 to 2017 based on the technology-enhanced learning model. Interactive Learning Environments, 27(8), 1011–1027. https://doi.org/10.1080/10494820.2018.1467462
  • Link, S., Mehrzad, M., & Rahimi, M. (2020). Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning, 33, 1–30.
  • Liu, S., & Kunna, A. J. (2016). Investigating the application of automated writing evaluation to Chinese undergraduate English majors: A case study of WriteToLearn. CALICO Journal, 33(1), 71–91. https://doi.org/10.1558/cj.v33i1.26380
  • Lopez, X., Valenzuela, J., Nussbaum, M., & Tsai, C. C. (2015). Some recommendations for the reporting of quantitative studies. Computers & Education, 91, 106–110. https://doi.org/10.1016/j.compedu.2015.09.010
  • Luo, Y., & Liu, Y. (2017). Comparison between peer feedback and automated feedback in college English writing: A case study. Open Journal of Modern Linguistics, 7(4), 197–215. https://doi.org/10.4236/ojml.2017.74015
  • Mirmotahari, O., Damsa, C., & Berg, Y. (2018). Individualized assessment and automated feedback in undergraduate computer science education. Proceedings of International Conference of the Learning Sciences, ICLS, 3(2018), 1577–1578.
  • Nunes, A., Cordeiro, C., Limpo, T., & Castro, S. L. (2021). Effectiveness of automated writing evaluation systems in school settings: A systematic review of studies from 2000 to 2020. Journal of Computer Assisted Learning. https://doi.org/10.1111/jcal.12635
  • Nystrand, M. (1989). A social-interactive model of writing. Written Communication, 6(1), 66–85. https://doi.org/10.1177/0741088389006001005
  • Onkas, N. A. (2015). Interpretation theory and creative writing. The Anthropologist, 22(2), 196–202. https://doi.org/10.1080/09720073.2015.11891869
  • Plonsky, L. (2015). Quantitative considerations for improving replicability in call and applied linguistics. Calico Journal, 32(2), 232–244. https://doi.org/10.1558/cj.v32i2.26857
  • Ranalli, J. (2021). L2 student engagement with automated feedback on writing: Potential for learning and issues of trust. Journal of Second Language Writing, 52, 1–16. https://doi.org/10.1016/j.jslw.2021.100816
  • Ranalli, J., Link, S., & Chukharev-Hudilainen, E. (2017). Automated writing evaluation for formative assessment of second language writing: Investigating the accuracy and usefulness of feedback as part of argument-based validation. Educational Psychology, 37(1), 8–25. https://doi.org/10.1080/01443410.2015.1136407
  • Shadiev, R., Hwang, W.-Y., & Huang, Y.-M. (2017). Review of research on mobile language learning in authentic environments. Computer Assisted Language Learning, 30(3–4), 284–303. https://doi.org/10.1080/09588221.2017.1308383
  • Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795
  • Stevenson, M. (2016). A critical interpretative synthesis: The integration of automated writing evaluation into classroom writing instruction. Computers and Composition, 42, 1–16. https://doi.org/10.1016/j.compcom.2016.05.001
  • Stevenson, M., & Phakiti, A. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65. https://doi.org/10.1016/j.asw.2013.11.007
  • Su, F., & Zou, D. (2020). Technology-enhanced collaborative language learning: Theoretical foundations, technologies, and implications. Computer Assisted Language Learning, 1–35. https://doi.org/10.1080/09588221.2020.1831545
  • Tardy, C. M. (2009). Building genre knowledge. Parlor Press.
  • Trotter, R. T. (2012). Qualitative research sample design and sample size: Resolving and unresolved issues and inferential imperatives. Preventive Medicine, 55(5), 398–400. https://doi.org/10.1016/j.ypmed.2012.07.003
  • Twining, P., Heller, R. S., Nussbaum, M., & Tsai, C. C. (2017). Some guidance on conducting and reporting qualitative studies. Computers & Education, 106, A1–A9. https://doi.org/10.1016/j.compedu.2016.12.002
  • Ware, P. (2011). Computer-generated feedback on student writing. TESOL Quarterly, 45(4), 769–774. https://doi.org/10.5054/tq.2011.272525
  • Wilson, J., Olinghouse, N. G., & Andrada, G. N. (2014). Does automated feedback improve writing quality? Learning Disabilities: A Contemporary Journal, 12(1), 93–118.
  • Zhang, R., & Zou, D. (2020). Types, purposes, and effectiveness of state-of-the-art technologies for second and foreign language learning. Computer Assisted Language Learning, 1–47. https://doi.org/10.1080/09588221.2020.1744666
  • Zhang, R., & Zou, D. (2021a). Types, features, and effectiveness of technologies in collaborative writing for second language learning. Computer Assisted Language Learning, 1–31. https://doi.org/10.1080/09588221.2021.1880441
  • Zhang, R., & Zou, D. (2021b). A state-of-the-art review of the modes and effectiveness of multimedia input for second and foreign language learning. Computer Assisted Language Learning, 1–27. https://doi.org/10.1080/09588221.2021.1896555
  • Zheng, P., Liang, X., Huang, G., & Liu, X. (2016). Mapping the field of communication technology research in Asia: Content analysis and text mining of SSCI journal articles 1995–2014. Asian Journal of Communication, 26(6), 511–531. https://doi.org/10.1080/01292986.2016.1231210
  • Zhu, M., Liu, O. L., & Lee, H. S. (2020). The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing. Computers & Education, 143, 103668. https://doi.org/10.1016/j.compedu.2019.103668
  • Zou, D., Huang, Y., & Xie, H. (2021). Digital game-based vocabulary learning: Where are we and where are we going? Computer Assisted Language Learning, 34(5–6), 751–777.
  • Zou, D., Luo, S., Xie, H., & Hwang, G. J. (2020). A systematic review of research on flipped language classrooms: Theoretical foundations, learning activities, tools, research topics and findings. Computer Assisted Language Learning, 1–27. https://doi.org/10.1080/09588221.2020.1839502

Appendix A. The coding schemesCoded papers

  • Akcapinar, G. (2015). How automated feedback through text mining changes plagiaristic behavior in online assignments. Computers & Education, 87, 123–130. https://doi.org/10.1016/j.compedu.2015.04.007
  • Bai, L., & Hu, G. (2017). In the face of fallible AWE feedback: How do students respond? Educational Psychology, 37(1), 67–81. https://doi.org/10.1080/01443410.2016.1223275
  • Barrot, J. S. (2021). Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2021.1936071
  • Bond, M., & Pennebaker, J. W. (2012). Automated computer-based feedback in expressive writing. Computers in Human Behavior, 28(3), 1014–1018. https://doi.org/10.1016/j.chb.2012.01.003
  • Chapelle, C. A., Cotos, E., & Lee, J. (2015). Validity arguments for diagnostic assessment using automated writing evaluation. Language Testing, 32(3), 385–405. https://doi.org/10.1177/0265532214565386
  • Chen, C.-F E., & Cheng, W.-Y E. C. (2008). Beyond the design of automated writing evaluation: Pedagogical practices and perceived learning effectiveness in EFL writing classes. Language Learning & Technology, 12(2), 94–112.
  • Chen, M. H., Huang, S. T., Chang, J. S., & Liou, H. C. (2015). Developing a corpus-based paraphrase tool to improve EFL learners’ writing skills. Computer Assisted Language Learning, 28(1), 22–40. https://doi.org/10.1080/09588221.2013.783873
  • Cheng, G. (2017). The impact of online automated feedback on students’ reflective journal writing in an EFL course. The Internet and Higher Education, 34, 18–27. https://doi.org/10.1016/j.iheduc.2017.04.002
  • Cotos, E. (2011). Potential of automated writing evaluation feedback. Calico Journal, 28(2), 420–459. https://doi.org/10.11139/cj.28.2.420-459
  • Cotos, E., Link, S., & Huffman, S. (2017). Effects of DDL technology on genre learning. Language Learning & Technology, 21(3), 104–130.
  • Dikli, S., & Bleyle, S. (2014). Automated Essay Scoring feedback for second language writers: How does it compare to instructor feedback? Assessing Writing, 22, 1–17. https://doi.org/10.1016/j.asw.2014.03.006
  • Dwyer, H. J., & Sullivan, H. J. (1993). Student preferences for teacher and computer composition marking. The Journal of Educational Research, 86(3), 137–141. https://doi.org/10.1080/00220671.1993.9941152
  • El Ebyary, K., & Windeatt, S. (2019). Eye tracking analysis of EAP students’ regions of interest in computer-based feedback on grammar, usage, mechanics, style and organization and development. System, 83, 36–49. https://doi.org/10.1016/j.system.2019.03.007
  • Gao, J. W., & Ma, S. (2019). The effect of two forms of computer-automated metalinguistic corrective feedback. Language Learning & Technology, 23(2), 65–83.
  • Gao, J. W., & Ma, S. (2020). Instructor feedback on free writing and automated corrective feedback in drills: Intensity and efficacy. Language Teaching Research, https://doi.org/10.1177/136216882091533.7
  • Garcia-Gorrostieta, J. M., Lopez-Lopez, A., & Gonzalez-Lopez, S. (2018). Automatic argument assessment of final project reports of computer engineering students. Computer Applications in Engineering Education, 26(5), 1217–1226. https://doi.org/10.1002/cae.21996
  • Guo, Q., Feng, R. L., & Hua, Y. F. (2021). How effectively can EFL students use automated written corrective feedback (AWCF) in research writing? Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2021.1879161
  • Han, Y. X., Zhao, S., & Ng, L. L. (2021). How technology tools impact writing performance, lexical complexity, and perceived self-regulated learning strategies in EFL academic writing: A comparative study. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2021.752793
  • Hassanzadeh, M., & Fotoohnejad, S. (2021). Implementing an automated feedback program for a foreign language writing course: A learner-centric study implementing an AWE tool in a L2 class. Journal of Computer Assisted Learning, 37(5), 1494–1507. https://doi.org/10.1111/jcal.12587
  • Jiang, L. J., & Yu, S. L. (2020). Appropriating automated feedback in L2 writing: experiences of Chinese EFL student writers. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2020.1799824
  • Lachner, A., Burkhart, C., & Nuckles, M. (2017). Mind the gap! Automated concept map feedback supports students in writing cohesive explanations. Journal of Experimental Psychology. Applied, 23(1), 29–46. https://doi.org/10.1037/xap0000111
  • Lai, Y. H. (2010). Which do students prefer to evaluate their essays: Peers or computer program. British Journal of Educational Technology, 41(3), 432–454. https://doi.org/10.1111/j.1467-8535.2009.00959.x
  • Landauer, T., Lochbaum, K., & Dooley, S. (2009). A new formative assessment technology for reading and writing. Theory into Practice, 48(1), 44–52. https://doi.org/10.1080/00405840802577593
  • Lee, C. (2020). A study of adolescent English learners’ cognitive engagement in writing while using an automated content feedback system. Computer Assisted Language Learning, 33(1–2), 26–57. https://doi.org/10.1080/09588221.2018.1544152
  • Lee, C., Wong, K. C. K., Cheung, W. K., & Lee, F. S. L. (2009). Web-based essay critiquing system and EFL students’ writing: A quantitative and qualitative investigation. Computer Assisted Language Learning, 22(1), 57–72. https://doi.org/10.1080/09588220802613807
  • Li, J. R., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18. https://doi.org/10.1016/j.jslw.2014.10.004
  • Liao, H. C. (2016). Using automated writing evaluation to reduce grammar errors in writing. Elt Journal, 70(3), 308–319. https://doi.org/10.1093/elt/ccv058
  • Liu, M., Li, Y., Xu, W., & Liu, L. (2016). Automated essay feedback generation and its impact on revision. IEEE Transactions on Learning Technologies, 10(4), 502–513. https://doi.org/10.1109/TLT.2016.2612659
  • Morch, A. I., Engeness, I., Cheng, V. C., Cheung, W. K., & Wong, K. C. (2017). EssayCritic: Writing to learn with a knowledge-based design critiquing system. Educational Technology & Society, 20(2), 213–223.
  • O’Neill, R., & Russell, A. (2019). Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly. Australasian Journal of Educational Technology, 35(1), 3795. https://doi.org/10.14742/ajet.3795
  • Proske, A., Narciss, S., & McNamara, D. S. (2012). Computer-based scaffolding to facilitate students’ development of expertise in academic writing. Journal of Research in Reading, 35(2), 136–152. https://doi.org/10.1111/j.1467-9817.2010.01450.x
  • Reynolds, B. L., Kao, C. W., & Huang, Y. Y. (2021). Investigating the effects of perceived feedback source on second language writing performance: A quasi-experimental study. The Asia-Pacific Education Researcher, 30(6), 585–595. https://doi.org/10.1007/s40299-021-00597-3
  • Roscoe, R. D., Wilson, J., Johnson, A. C., & Mayra, C. R. (2017). Presentation, expectations, and experience: Sources of student perceptions of automated writing evaluation. Computers in Human Behavior, 70, 207–221. https://doi.org/10.1016/j.chb.2016.12.076
  • Saricaoglu, A. (2019). The impact of automated feedback on L2 learners’ written causal explanations. ReCALL, 31(2), 189–203. https://doi.org/10.1017/S095834401800006X
  • Shang, H.-F. (2019). Exploring online peer feedback and automated corrective feedback on EFL writing performance. Interactive Learning Environments, 30(1):4–16.
  • Sherafati, N., Largani, F. M., & Amini, S. (2020). Exploring the effect of computer-mediated teacher feedback on the writing achievement of Iranian EFL learners: Does motivation count? Education and Information Technologies, 25(5), 4591–4613. https://doi.org/10.1007/s10639-020-10177-5
  • Sung, Y. T., Liao, C. N., Chang, T. H., Chen, C. L., & Chang, K. E. (2016). The effect of online summary assessment and feedback system on the summary writing on 6th graders: The LSA-based technique. Computers & Education, 95, 1–18. https://doi.org/10.1016/j.compedu.2015.12.003
  • Tian, L. L., & Zhou, Y. (2020). Learner engagement with automated feedback, peer feedback and teacher feedback in an online EFL writing context. System, 91, 102247. https://doi.org/10.1016/j.system.2020.102247
  • Wade-Stein, D., & Kintsch, E. (2004). Summary street: Interactive computer support for writing. Cognition and Instruction, 22(3), 333–362. https://doi.org/10.1207/s1532690xci2203_3
  • Wang, Y., Harrington, M., & White, P. (2012). Detecting breakdowns in local coherence in the writing of Chinese English learners. Journal of Computer Assisted Learning, 28(4), 396–410. https://doi.org/10.1111/j.1365-2729.2011.00475.x
  • Wang, Y. J., Shang, H. F., & Briody, P. (2013). Exploring the impact of using automated writing evaluation in English as a foreign language university students’ writing. Computer Assisted Language Learning, 26(3), 234–257. https://doi.org/10.1080/09588221.2012.655300
  • Wilson, J., & Roscoe, R. D. (2020). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764
  • Zaini, A. (2018). Word processors as monarchs: Computer-generated feedback can exercise power over and influence EAL learners’ identity representations. Computers & Education, 120, 112–126. https://doi.org/10.1016/j.compedu.2018.01.014
  • Zhang, Z. (2017). Student engagement with computer-generated feedback: A case study. Elt Journal, 71(3), 317–328. https://doi.org/10.1093/elt/ccw089
  • Zhang, Z. (2020). Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions. Assessing Writing, 43, 78–91. https://doi.org/10.1016/j.asw.2019.100439
  • Zhang, Z. V., & Hyland, K. (2018). Student engagement with teacher and automated feedback on L2 writing. Assessing Writing, 36, 90–102. https://doi.org/10.1016/j.asw.2018.02.004
  • Zhu, M. X., Lee, H. S., Wang, T., Liu, O. L., Belur, V., & Pallant, A. (2017). Investigating the impact of automated feedback on students’ scientific argumentation. International Journal of Science Education, 39(12), 1648–1668. https://doi.org/10.1080/09500693.2017.1347303
  • Zhu, M. X., Liu, O. L., & Lee, H. S. (2020). The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing. Computers & Education, 143, 103668. https://doi.org/10.1016/j.compedu.2019.103668

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.