The comparison of two automated feedback approaches based on automated analysis of the online asynchronous interaction: a case of massive online teacher training

References

• Ackerman, R., Parush, A., Nassar, F., & Shtub, A. (2016). Metacognition and system usability: Incorporating metacognitive research paradigm into usability testing. Computers in Human Behavior, 54, 101–113. https://doi.org/10.1016/j.chb.2015.07.041
   Web of Science ®Google Scholar
• Alemayehu, L., & Chen, H.-L. (2021). Learner and instructor-related challenges for learners’ engagement in MOOCs: A review of 2014-2020 publications in selected SSCI indexed journals. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1920430
   Google Scholar
• An, H., Shin, S., & Lim, K. (2009). The effects of different instructor facilitation approaches on students’ interactions during asynchronous online discussions. Computers & Education, 53(3), 749–760. https://doi.org/10.1016/j.compedu.2009.04.015
   Web of Science ®Google Scholar
• Beggrow, E. P., Ha, M., Nehm, R. H., Pearl, D., & Boone, W. J. (2014). Assessing scientific practices using machine-learning methods: How closely do they match clinical interview performance? Journal of Science Education and Technology, 23(1), 160–182. https://doi.org/10.1007/s10956-013-9461-9
   Web of Science ®Google Scholar
• Binali, T., Tsai, C.-C., & Chang, H.-Y. (2021). University students’ profiles of online learning and their relation to online metacognitive regulation and internet-specific epistemic justification. Computers & Education, 175, 104315. https://doi.org/10.1016/j.compedu.2021.104315
   PubMed Web of Science ®Google Scholar
• Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64(1), 417–444. https://doi.org/10.1146/annurev-psych-113011-143823
   PubMed Web of Science ®Google Scholar
• Burns, A., Holford, P., & Andronicos, N. (2022). Enhancing understanding of foundation concepts in first year university STEM: Evaluation of an asynchronous online interactive lesson. Interactive Learning Environments, 30(7), 1170–1182. https://doi.org/10.1080/10494820.2020.1712426
   Google Scholar
• Chen, B., & Huang, T. (2019). It is about timing: Network prestige in asynchronous online discussions. Journal of Computer Assisted Learning, 35(4), 503–515. https://doi.org/10.1111/jcal.12355
   Web of Science ®Google Scholar
• Chi, M., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
   Web of Science ®Google Scholar
• Cole, A. W., Anderson, C., Bunton, T., Cherney, M. R., Cronin Fisher V., Draeger Jr. R., Featherston M., Motel L., Nicolini K. M., Peck B., & Allen M. (2017). Student predisposition to instructor feedback and perceptions of teaching presence predict motivation toward online courses. Online Learning, 21(4), 245–262. https://doi.org/10.24059/olj.v21i4.966
   Web of Science ®Google Scholar
• Crane, R., & Comley, S. (2021). Influence of social learning on the completion rate of massive online open courses. Education and Information Technologies, 26(2), 2285–2293. https://doi.org/10.1007/s10639-020-10362-6
   Web of Science ®Google Scholar
• Daniels, L. M., & Bulut, O. (2020). Students’ perceived usefulness of computerized percentage-only vs. Descriptive score reports: Associations with motivation and grades. Journal of Computer Assisted Learning, 36(2), 199–208. https://doi.org/10.1111/jcal.12398
   Google Scholar
• Deeva, G., Bogdanova, D., Serral, E., Snoeck, M., & De Weerdt, J. (2020). A review of automated feedback systems for learners: Classification framework, challenges and opportunities. Computers & Education, 162, 104094. https://doi.org/10.1016/j.compedu.2020.104094
   Web of Science ®Google Scholar
• Delfino, M., & Persico, D. (2007). Online or face-to-face? Experimenting with different techniques in teacher training. Journal of Computer Assisted Learning, 23(5), 351–365. https://doi.org/10.1111/j.1365-2729.2007.00220.x
   Google Scholar
• Fletcher, T., & Bullock, S. M. (2015). Reframing pedagogy while teaching about teaching online: A collaborative self-study. Professional Development in Education, 41(4), 690–706. https://doi.org/10.1080/19415257.2014.938357
   Web of Science ®Google Scholar
• Fong, C. J., Schallert, D. L., Williams, K. M., Williamson, Z. H., Lin, S., Kim, Y. W., & Chen, L.-H. (2021). Making feedback constructive: The interplay of undergraduates’ motivation with perceptions of feedback specificity and friendliness. Educational Psychology, 41(10), 1241–1259. https://doi.org/10.1080/01443410.2021.1951671
   Web of Science ®Google Scholar
• Frey, B. A., & Alman, S. W. (2003). Applying adult learning theory to the online classroom. New Horizons in Adult Education & Human Resource Development, 17(1), 4–12. https://doi.org/10.1002/nha3.10155
   Google Scholar
• Galikyan, I., & Admiraal, W. (2019). Students’ engagement in asynchronous online discussion: The relationship between cognitive presence, learner prominence, and academic performance. The Internet and Higher Education, 43, 100692. https://doi.org/10.1016/j.iheduc.2019.100692
   Web of Science ®Google Scholar
• Gao, F., Zhang, T., & Franklin, T. (2013). Designing asynchronous online discussion environments: Recent progress and possible future directions. British Journal of Educational Technology, 44(3), 469–483. https://doi.org/10.1111/j.1467-8535.2012.01330.x
   Web of Science ®Google Scholar
• García-Martínez, I., Tadeu, P., Montenegro-Rueda, M., & Fernández-Batanero, J. M. (2022). Networking for online teacher collaboration. Interactive Learning Environments, 30(9), 1736–1750. https://doi.org/10.1080/10494820.2020.1764057
   Google Scholar
• Gašević, D., Joksimović, S., Eagan, B. R., & Shaffer, D. W. (2019). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92, 562–577. https://doi.org/10.1016/j.chb.2018.07.003
   Web of Science ®Google Scholar
• Graziano, K., & Bryans-Bongey, S. (2018). Surveying the national landscape of online teacher training in K-12 teacher preparation programs. Journal of Digital Learning in Teacher Education, 34(4), 259–277. https://doi.org/10.1080/21532974.2018.1498040
   Google Scholar
• Gregory, S. E. A., Winstone, N. E., Ridout, N., & Nash, R. A. (2020). Weak memory for future-oriented feedback: Investigating the roles of attention and improvement focus. Memory (Hove, England), 28(2), 216–236. https://doi.org/10.1080/09658211.2019.1709507
   PubMed Web of Science ®Google Scholar
• Hawkes, M. (2006). Linguistic discourse vairables as indicators of reflective online interaction. The American Journal of Distance Education, 20(4), 231–244. https://doi.org/10.1207/s15389286ajde2004_4
   Google Scholar
• Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye (Ed.), Collaborative learning through computer conferencing: The najaden papers (pp. 115–136). Springer. https://doi.org/10.1007/978-3-642-77684-7_8
   Google Scholar
• Hew, K. F., & Cheung, W. S. (2003). Evaluating the participation and quality of thinking of pre-service teachers in an asynchronous online discussion environment: Part II. International Journal of Instructional Media, 30(3), 118–123. https://doi.org/10.1177/0098628312475031
   Google Scholar
• Janelli, M., & Lipnevich, A. A. (2021). Effects of pre-tests and feedback on performance outcomes and persistence in massive open online courses. Computers & Education, 161, 104076. https://doi.org/10.1016/j.compedu.2020.104076
   Google Scholar
• Jensen, L. X, Bearman, M., & Boud D. (2021). Understanding feedback in online learning-A critical review and metaphor analysis. Computers & Education, 173, 104271. https://doi.org/10.1016/j.compedu.2021.104271
   Google Scholar
• Jin, C. (2020). MOOC student dropout prediction model based on learning behavior features and parameter optimization. Interactive Learning Environments. https://doi.org/10.1080/10494820.2020.1802300
   Web of Science ®Google Scholar
• Kautzmann, T. R., & Jaques, P. A. (2019). Effects of adaptive training on metacognitive knowledge monitoring ability in computer-based learning. Computers & Education, 129, 92–105. https://doi.org/10.1016/j.compedu.2018.10.017
   Web of Science ®Google Scholar
• Kim, J., Shaw, E., Wyner, S., Kim, T., & Li, J. (2010). Discerning affect in student discussions. Proceedings of the Annual Meeting of the Cognitive Science Society, 32, 2344–2349. Retrieved March 20, 2023, from https://escholarship.org/uc/item/9tg4362n
   Google Scholar
• Koszalka, T. A., Pavlov, Y., & Wu, Y. (2021). The informed use of pre-work activities in collaborative asynchronous online discussions: The exploration of idea exchange, content focus, and deep learning. Computers and Education, 161, 104067. https://doi.org/10.1016/j.compedu.2020.104067
   Google Scholar
• Kumi-Yeboah, A., Dogbey, J., & Yuan, G. (2017). Online collaborative learning activities: The perceptions of culturally diverse graduate students. Online Learning, 21(4), 5–28. https://doi.org/10.24059/olj.v21i4.1277
   Web of Science ®Google Scholar
• Lin, G. Y. (2018). Anonymous versus identified peer assessment via a Facebook-based learning application: Effects on quality of peer feedback, perceived learning, perceived fairness, and attitude toward the system. Computers & Education, 116(1), 81–92. https://doi.org/10.1016/j.compedu.2017.08.010
   Google Scholar
• Liu, M., Li, Y., Xu, W., & Liu, L. (2016). Automated essay feedback generation and Its impact in the revision. IEEE Transactions on Learning Technologies, 10(4), 502–513. https://doi.org/10.1109/TLT.2016.2612659
   Web of Science ®Google Scholar
• Liu, Q., Zhang, S., Wang, Q., & Chen, W. (2018a). Mining online discussion data for understanding teachers reflective thinking. IEEE Transactions on Learning Technologies, 11(2), 243–254. https://doi.org/10.1109/TLT.2017.2708115
   Web of Science ®Google Scholar
• Liu, S., Hu, T., Chai, H., Su, Z., & Peng, X. (2021). Learners’ interaction patterns in asynchronous online discussions: An integration of the social and cognitive interactions. British Journal of Educational Technology, 53(1), 23–40. https://doi.org/10.1111/bjet.13147
   Google Scholar
• Liu, S., Liu, S., Liu, Z., Peng, X., & Yang, Z. (2022). Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement. Computers and Education, 181, 104461. https://doi.org/10.1016/j.compedu.2022.104461
   Google Scholar
• Liu, Z., Pinkwart, N., Liu, H., Liu, S., & Zhang, G. (2018b). Exploring students’ engagement patterns in SPOC forums and their association with course performance. EURASIA Journal of Mathematics Science and Technology Education, 14(7), 3143–3158. https://doi.org/10.29333/ejmste/91667
   Google Scholar
• Ma, N., Du, L., Zhang, Y., Cui, Z., & Ma, R. (2020). The effect of interaction between knowledge map and collaborative learning strategies on teachers’ learning performance and self-efficacy of group learning. Interactive Learning Environments, 1–15. https://doi.org/10.1080/10494820.2020.1855204
   Google Scholar
• Ma, N., Li, Y. M., Guo, J. H., Laurillard, D., & Yang, M. (2022). A learning model for improving in-service teachers’ course completion in MOOCs. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.2025405
   Google Scholar
• McCoy, A., & McNaughton, D. (2021). Effects of online training on educators’ knowledge and use of system of least prompts to support augmentative and alternative communication. Journal of Behavioral Education, 30(3), 319–349. https://doi.org/10.1007/s10864-020-09374-6
   Google Scholar
• Molin, F., Haelermans, C., Cabus, S., & Groot, W. (2020). The effect of feedback on metacognition - A randomized experiment using polling technology. Computers & Education, 152, 103885. https://doi.org/10.1016/j.compedu.2020.103885
   Google Scholar
• Molin, F., Haelermans, C., Cabus, S., & Groot, W. (2020). The effect of feedback on metacognition - A randomized experiment using polling technology. Computers and Education, 152, 103885. https://doi.org/10.1016/j.compedu.2020.103885
   Web of Science ®Google Scholar
• Nandi, D., Hamilton, M., & Harland, J. (2012). Evaluating the quality of interaction in asynchronous discussion forums in fully online courses. Distance Education, 33(1), 5–30. https://doi.org/10.1080/01587919.2012.667957
   Web of Science ®Google Scholar
• Narciss, S., Hammer, E., Damnik, G., Kisielski, K., & Körndle, H. (2021). Promoting prospective teacher competencies for designing, implementing, evaluating, and adapting interactive formative feedback strategies. Psychology Learning & Teaching, 20(2), 261–278. https://doi.org/10.1177/1475725720971887
   Google Scholar
• Nash, R. A., Winstone, N. E., & Gregory, S. E. A. (2021). Selective memory searching does not explain the poor recall of future-oriented feedback. Journal of Applied Research in Memory and Cognition, 10(3), 467–478. https://doi.org/10.1016/j.jarmac.2021.01.003
   Google Scholar
• Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
   Web of Science ®Google Scholar
• Nicolescu, B. N., Macarie, T., & Petrescu, T. (2015). Some considerations on the online training programs for the teachers from the Romanian Pre-university educational system. Procedia, Social and Behavioral Sciences, 180, 878–884. https://doi.org/10.1016/j.sbspro.2015.02.233
   Google Scholar
• Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592
   Web of Science ®Google Scholar
• Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. https://doi.org/10.1007/s10648-006-9029-9
   Web of Science ®Google Scholar
• Puertas-Aguilar, M-Á, Álvarez-Otero, J., & de Lázaro-Torres, M.-L. (2021). The challenge of teacher training in the 2030 agenda framework using geotechnologies. Education Sciences, 11(8), 381. https://doi.org/10.3390/educsci11080381
   Web of Science ®Google Scholar
• Rivers, K., & Koedinger, K. R. (2017). Data-driven hint generation in vast solution spaces: A self-improving python programming tutor. International Journal of Artificial Intelligence in Education, 27(1), 37–64. https://doi.org/10.1007/s40593-015-0070-z
   Web of Science ®Google Scholar
• Ro, J. (2021). On the matter of teacher quality: Lessons from Singapore. Journal of Curriculum Studies, 53(4), 500–515. https://doi.org/10.1080/00220272.2020.1808903
   Web of Science ®Google Scholar
• Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280. https://doi.org/10.1016/j.learninstruc.2010.07.004
   Web of Science ®Google Scholar
• Sarrafzadeh, A., Alexander, S., Dadgostar, F., Fan, C., & Bigdeli, A. (2008). ‘How do you know that I don’t understand?’ A look at the future of intelligent tutoring systems. Computers in Human Behavior, 24(4), 1342–1363. https://doi.org/10.1016/j.chb.2007.07.008
   Web of Science ®Google Scholar
• Schrader, C., Grassinger, R. (2021). Tell me that I can do it better. The effect of attributional feedback from a learning technology on achievement emotions and performance and the moderating role of individual adaptive reactions to errors. Computers & Education, 161, 104028. https://doi.org/10.1016/j.compedu.2020.104028
   Web of Science ®Google Scholar
• Shermis, M. D., & Burstein, J. (2013). Handbook of automated essay evaluation: Current applications and new directions[M]. Routledge.
   Google Scholar
• Smith, J. A., & Sivo, S. A. (2012). Predicting continued use of online teacher professional development and the influence of social presence and sociability. British Journal of Educational Technology, 43(6), 871–882. https://doi.org/10.1111/j.1467-8535.2011.01223.x
   Web of Science ®Google Scholar
• Tang, H. (2021). Teaching teachers to use technology through massive open online course: Perspectives of interaction equivalency. Computers & Education, 174, 104307. https://doi.org/10.1016/j.compedu.2021.104307
   Web of Science ®Google Scholar
• Terzis, V., Moridis, C. N., & Economides, A. A. (2012). The effect of emotional feedback on behavioral intention to use computer based assessment. Computers & Education, 59(2), 710–721. https://doi.org/10.1016/j.compedu.2012.03.003
   Web of Science ®Google Scholar
• Thomas M.J.W. (2002). Learning within incoherent structures: The space of online discussion forums. Journal of Computer Assisted Learning, 18(3), 351–366. https://doi.org/10.1046/j.0266-4909.2002.03800.x
   Web of Science ®Google Scholar
• Topçu, A. (2008). ‘Intentional repetition’ and learning style: Increasing efficient and cohesive interaction in asynchronous online discussions. British Journal of Educational Technology, 39(5), 901–919. https://doi.org/10.1111/j.1467-8535.2007.00783.x
   Web of Science ®Google Scholar
• Tsai, Y., Lin, C., Hong, J., & Tai, K. (2018). The effects of metacognition on online learning interest and continuance to learn with MOOCs. Computers & Education, 121, 18–29. https://doi.org/10.1016/j.compedu.2018.02.011
   Web of Science ®Google Scholar
• Vanlehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369
   Web of Science ®Google Scholar
• Wang, C., Liu, X., Wang, L., Sun, Y., & Zhang, H. (2021). Automated scoring of Chinese grades 7–9 students’ competence in interpreting and arguing from evidence. Journal of Science Education and Technology, 30(2), 269–282. https://doi.org/10.1007/s10956-020-09859-z
   Web of Science ®Google Scholar
• Wu, Y., & Schunn, C. D. (2020). From feedback to revisions: Effects of feedback features and perceptions. Contemporary Educational Psychology, 60, 101826. https://doi.org/10.1016/j.cedpsych.2019.101826
   Google Scholar
• Yang, D., Richardson, J. C., French, B. F., Lehman, J. D., Richardson, J. C., French, B. F., & Lehman, J. D. (2011). The development of a content analysis model for assessing students’ cognitive learning in asynchronous online discussions. Educational Technology Research and Development, 59(1), 43–70. https://doi.org/10.1007/s11423-010-9166-1
   Web of Science ®Google Scholar
• Yang, Y. C. (2008). A catalyst for teaching critical thinking in a large university class in Taiwan: Asynchronous online discussions with the facilitation of teaching assistants. Educational Technology Research and Development, 56(3), 241–264. https://doi.org/10.1007/s11423-007-9054-5
   Web of Science ®Google Scholar
• Zhang, H., Apeanti, W. O., Georgescu, P., Harvim, P., Lu, D., Li, T., & Zhang, B. (2021). Sustainable teacher training via distance education: The effect of study centers, gender and economic demographics on academic performance. Sustainability (Basel, Switzerland), 13(14), 7965. https://doi.org/10.3390/su13147965
   Google Scholar
• Zheng, Y. F., Xu, C., & Li, Y. (2017). Study on An analysis model and Its visualization of computer-supported collaborative learning. E-education Research, 38(4), 47–52. https://www.cnki.net/kcms/doi/10.13811/j.cnki.eer.2017.04.007.html
   Google Scholar