Limitations of Disembodied Computer-Generated Voice to Convey Emotion in Multimedia Lessons

References

• Ambady, N., & Rosenthal, R. (1993). Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. Journal of Personality and Social Psychology, 64(3), 431–441. https://doi.org/10.1037/0022-3514.64.3.431
   Web of Science ®Google Scholar
• Atkinson, R. K., Mayer, R. E., & Merrill, M. M. (2005). Fostering social agency in multimedia learning: Examining the impact of an animated agent’s voice. Contemporary Educational Psychology, 30(1), 117–139. https://doi.org/10.1016/j.cedpsych.2004.07.001
   Web of Science ®Google Scholar
• Bower, G. H. (1992). How might emotions affect learning. In S. Christianson (Ed.), The handbook of emotion and memory: Research and theory (pp. 3–31). Psychology Press.
   Google Scholar
• Castro-Alonso, J. C., Wong, R. M., Adesope, O. O., & Paas, F. (2021). Effectiveness of multimedia pedagogical agents predicted by diverse theories: A meta-analysis. Educational Psychology Review, 33(3), 989–1015. https://doi.org/10.1007/s10648-020-09587-1
   Web of Science ®Google Scholar
• Chiou, E. K., Schroeder, N. L., & Craig, S. D. (2020). How we trust, perceive, and learn from virtual humans: The influence of voice quality. Computers & Education, 146, 103756. https://doi.org/10.1016/j.compedu.2019.103756
   Web of Science ®Google Scholar
• Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250. https://doi.org/10.1080/1358165042000283101
   Google Scholar
• Craig, S. D., & Schroeder, N. L. (2017). Reconsidering the voice effect when learning from a virtual human. Computers & Education, 114, 193–205. https://doi.org/10.1016/j.compedu.2017.07.003
   Web of Science ®Google Scholar
• Craig, S. D., & Schroeder, N. L. (2018). Text-to-speech software and learning: Investigating the relevancy of the voice effect. Journal of Educational Computing Research, 57(6), 1534–1548. https://doi.org/10.1177/0735633118802877
   Google Scholar
• Davis, R. O. (2018). The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis. Educational Research Review, 24, 193–209. https://doi.org/10.1016/j.edurev.2018.05.002
   Web of Science ®Google Scholar
• Davis, R. O., Vincent, J., & Park, T. (2019). Reconsidering the voice principle with non-native language speakers. Computers & Education, 140, 103605. https://doi.org/10.1016/j.compedu.2019.103605
   Web of Science ®Google Scholar
• D’Mello, S. K. (2017). Emotional learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (pp. 115–127). Society for Learning Analytics Research.
   Google Scholar
• D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. https://doi.org/10.1016/j.learninstruc.2011.10.001
   Web of Science ®Google Scholar
• Eagly, A. H., Wood, W., & Diekman, A. B. (2000). Social role theory of sex differences and similarities: A current appraisal. The Developmental Social Psychology of Gender, 12(174). https://doi.org/10.4324/9781410605245-7.
   Google Scholar
• Fiorella, L., & Mayer, R. E. (2022). Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 277–285). Cambridge University Press.
   Google Scholar
• Glanzer, M., & Cunitz, A. R. (1966). Two storage mechanisms in free recall. Journal of Verbal Learning and Verbal Behavior, 5(4), 351–360. https://doi.org/10.1016/S0022-5371(66)80044-0
   Google Scholar
• Graesser, A. C., Chipman, P., King, B., McDaniel, B., & ’D’Mello, S. K. (2007). Emotions and learning with auto tutor. Frontiers in Artificial Intelligence and Applications, 158, 569–571. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=4f34f76c94784e3b554f9f382b63510e50578c91
   Google Scholar
• Hillaire, G., Iniesto, F., & Rienties, B. (2019). Humanising text-to-speech through emotional expression in online courses. Journal of Interactive Media in Education, 2019(1), 1–9. https://doi.org/10.5334/jime.519
   Google Scholar
• Horovitz, T., & Mayer, R. E. (2021). Learning with human and virtual instructors who display happy or bored emotions in video lectures. Computers in Human Behavior, 119, 1–8. https://doi.org/10.1016/j.chb.2021.106724
   Google Scholar
• Hwang, N., & Fitzpatrick, B. (2021). Student–teacher gender matching and academic achievement. AERA Open, 7, 1–11. https://doi.org/10.1177/23328584211040058
   Google Scholar
• Jenkins, J. S., Bugeja, A. D., & Barber, L. K. (2014). More content or more policy? A closer look at syllabus detail, instructor gender, and perceptions of instructor effectiveness. College Teaching, 62(4), 129–135. https://doi.org/10.1080/87567555.2014.935700
   Google Scholar
• Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research, 68(3), 350–386. https://doi.org/10.3102/00346543068003350
   Web of Science ®Google Scholar
• Lawson, A. P., & Mayer, R. E. (2021). The power of voice to convey emotion in multimedia instructional messages. International Journal of Artificial Intelligence in Education, 32(4), 971–990. https://doi.org/10.1007/s40593-021-00282-y
   Google Scholar
• Lawson, A. P., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021a). Recognizing the emotional state of human and virtual instructors. Computers in Human Behavior, 114, 106554. https://doi.org/10.1016/j.chb.2020.106554
   Web of Science ®Google Scholar
• Lawson, A., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021b). The positivity principle: Do positive instructors improve learning from instruction video lectures? Educational Technology Research and Development: ETR & D, 69(6), 3101–3129. https://doi.org/10.1007/s11423-021-10057-w
   PubMedGoogle Scholar
• Lawson, A. P., Mayer, R. E., Adamo-Villani, N., Benes, B., Lei, X., & Cheng, J. (2021c). Do learners recognize and relate to the emotions displayed by virtual instructors? International Journal of Artificial Intelligence in Education, 31(1), 134–153. https://doi.org/10.1007/s40593-021-00238-2
   Web of Science ®Google Scholar
• Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997, March). The persona effect: Affective impact of animated pedagogical agents. In S. Pemberton (Ed.), Proceedings of the ACM SIGCHI conference on human factors in computing systems (pp. 359–366). ACM Press. https://doi.org/10.1145/258549.258797
   Google Scholar
• Liew, T. W., Tan, S. M., Pang, W. M., Khan, M. T. I., & Kew, S. N. (2023). I am Alexa, your virtual tutor!: The effects of Amazon Alexa’s text-to-speech voice enthusiasm in a multimedia learning environment. Education and Information Technologies, 28(2), 1455–1489. https://doi.org/10.1007/s10639-022-11255-6
   PubMed Web of Science ®Google Scholar
• Loderer, K., Pekrun, R., & Lester, J. C. (2020). Beyond cold technology: A systematic review and meta-analysis on emotions in technology-based learning environments. Learning and Instruction, 70, 101162. https://doi.org/10.1016/j.learninstruc.2018.08.002
   Web of Science ®Google Scholar
• Makransky, G., Wismer, P., & Mayer, R. E. (2019). A gender matching effect in learning with pedagogical agents in an immersive virtual reality science simulation. Journal of Computer Assisted Learning, 35(3), 349–358. https://doi.org/10.1111/jcal.12335
   Web of Science ®Google Scholar
• Mayer, R. E. (2021). Multimedia learning (3rd ed.). Cambridge University Press.
   Google Scholar
• Mayer, R. E. (2020). Searching for the role of emotions in e-learning. Learning and Instruction, 70, 101213. https://doi.org/10.1016/j.learninstruc.2019.05.010
   Web of Science ®Google Scholar
• Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93(2), 390–397. https://doi.org/10.1037/0022-0663.93.2.390
   Web of Science ®Google Scholar
• Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with animated pedagogical agents. Journal of Experimental Psychology. Applied, 18(3), 239–252. https://doi.org/10.1037/a0028616
   PubMed Web of Science ®Google Scholar
• Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multimedia learning: When presenting more material results in less understanding. Journal of Educational Psychology, 93(1), 187–198. https://doi.org/10.1037/0022-0663.93.1.187
   Web of Science ®Google Scholar
• Mayer, R. E., & Johnson, C. I. (2008). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100(2), 380–386. https://doi.org/10.1037/0022-0663.100.2.380
   Web of Science ®Google Scholar
• Mayer, R. E., Moreno, R., Boire, M., & Vagge, S. (1999). Maximizing constructivist learning from multimedia communications by minimizing cognitive load. Journal of Educational Psychology, 91(4), 638–643. https://doi.org/10.1037/0022-0663.91.4.638
   Web of Science ®Google Scholar
• Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning: Role of ’speaker’s voice. Journal of Educational Psychology, 95(2), 419–425. https://doi.org/10.1037/0022-0663.95.2.419
   Web of Science ®Google Scholar
• McGaugh, J. L. (2018). Emotional arousal regulation of memory consolidation. Current Opinion in Behavioral Sciences, 19, 55–60. https://doi.org/10.1016/j.cobeha.2017.10.003
   Web of Science ®Google Scholar
• Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358–368. https://doi.org/10.1037/0022-0663.91.2.358
   Web of Science ®Google Scholar
• Moreno, R., & Mayer, R. E. (2000). Engaging students in active learning: The case for personalized multimedia messages. Journal of Educational Psychology, 92(4), 724–733. https://doi.org/10.1037/0022-0663.92.4.724
   Web of Science ®Google Scholar
• Moreno, R., & Mayer, R. E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94(1), 156–163. https://doi.org/10.1037/0022-0663.94.1.156
   Web of Science ®Google Scholar
• Moreno, R., & Mayer, R. E. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326. https://doi.org/10.1007/s10648-007-9047-2
   Web of Science ®Google Scholar
• Nass, C., & Brave, S. (2005). Wired for speech: How voice activates and advances the human-computer relationship. MIT Press.
   Google Scholar
• Pekrun, R. (2016). Academic emotions. In Handbook of motivation at school (pp. 120–144). Routledge.
   Google Scholar
• Pekrun, R. (2017). Achievement emotions. In A. J. Elliot, C. S. Dweck, & D. S. Yeager (Eds.), Handbook of competence and motivation: Theory and application (pp. 251–271). The Guilford Press.
   Google Scholar
• Pekrun, R., & Perry, R. P. (2014). Control-value theory of achievement emotions. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 120–141). Taylor and Francis.
   Google Scholar
• Plass, J. L., & Kaplan, U. (2016). Emotional design in digital media for learning. In S. Y. Tettegah & M. P. McCreery (Eds.), Emotions, technology, and learning (pp. 131–161). Academic Press.
   Google Scholar
• Reeves, B., & Nass, C. (1996). The media equation. Cambridge University Press.
   Google Scholar
• Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714
   Web of Science ®Google Scholar
• Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110(1), 145–172. https://doi.org/10.1037/0033-295X.110.1.145
   PubMed Web of Science ®Google Scholar
• Ryu, J. E. E. H. E. O. N., & Baylor, A. L. (2005). The psychometric structure of pedagogical agent persona. Technology Instruction Cognition and Learning, 2(4), 291–314. https://www.researchgate.net/profile/Amy-Baylor/publication/237627605_The_API_Agent_Persona_Instrument_for_Assessing_Pedagogical_Agent_Persona/links/57336bdb08aea45ee838f5ee/The-API-Agent-Persona-Instrument-for-Assessing-Pedagogical-Agent-Persona.pdf
   Google Scholar
• Schroeder, N. L., & Adesope, O. O. (2014). A systematic review of pedagogical agents’ persona, motivation, and cognitive load implications for learners. Journal of Research on Technology in Education, 46(3), 229–251. https://doi.org/10.1080/15391523.2014.888265
   Google Scholar
• Schroeder, N. L., Adesope, O. O., & Gilbert, R. B. (2013). How effective are pedagogical agents for learning? A meta-analytic review. Journal of Educational Computing Research, 49(1), 1–39. https://doi.org/10.2190/EC.49.1.a
   Web of Science ®Google Scholar
• Wang, F., Li, W., & Zhao, T. (2022). Multimedia learning with animated pedagogical agents. In R. E. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (3rd ed., pp. 450–460). Cambridge University Press.
   Google Scholar
• Wixted, J. T. (2004). The psychology and neuroscience of forgetting. Annual Review of Psychology, 55(1), 235–269. https://doi.org/10.1146/annurev.psych.55.090902.141555
   PubMed Web of Science ®Google Scholar
• Zhao, F., & Mayer, R. E. (2023). Role of emotional tone and gender of computer-generated voices in multimedia lessons. Educational Technology Research and Development, 71(4), 1449–1469. https://doi.org/10.1007/s11423-023-10228-x
   Google Scholar