Limitations of Disembodied Computer-Generated Voice to Convey Emotion in Multimedia Lessons

Abstract

The goal of this study was to determine whether learning processes and outcomes are affected by the emotional tone of the computer-generated voice in a narrated slideshow. In a between-subjects design, participants viewed a narrated slideshow on lightning formation involving a computer-generated female (Experiment 1) or male voice (Experiment 2) that displayed happy, content, angry, or sad emotion. On subsequent surveys and post-tests, students could recognize positive or negative emotions conveyed by the female voice but not the male voice. The emotional tone of the instructor’s voices had minimal impact on students’ ratings of felt emotion during learning, ratings of social connection with the instructor, or scores on retention and transfer tests of learning outcome. The findings highlight the limitations of computer-generated voices to convey emotions that trigger affective, social, and cognitive processes when an onscreen instructor is absent, thereby suggesting a boundary condition for the cognitive-affective model of e-learning.

Keywords:

• Text-to-speech
• computer-generated voice
• narrated slideshow
• emotion
• multimedia learning

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the National Science Foundation under Grant 2201020.

Notes on contributors

Fangzheng Zhao

Fangzheng Zhao is a PhD Candidate in the Department of Psychological and Brain Sciences at the University of California, Santa Barbara.

Richard E. Mayer

Richard E. Mayer is Distinguished Professor in the Department of Psychological and Brain Sciences at the University of California, Santa Barbara.