A virtual speaker in noisy classroom conditions: supporting or disrupting children’s listening comprehension?

Abstract

Aim: Seeing a speaker’s face facilitates speech recognition, particularly under noisy conditions. Evidence for how it might affect comprehension of the content of the speech is more sparse. We investigated how children’s listening comprehension is affected by multi-talker babble noise, with or without presentation of a digitally animated virtual speaker, and whether successful comprehension is related to performance on a test of executive functioning.

Materials and Methods: We performed a mixed-design experiment with 55 (34 female) participants (8- to 9-year-olds), recruited from Swedish elementary schools. The children were presented with four different narratives, each in one of four conditions: audio-only presentation in a quiet setting, audio-only presentation in noisy setting, audio-visual presentation in a quiet setting, and audio-visual presentation in a noisy setting. After each narrative, the children answered questions on the content and rated their perceived listening effort. Finally, they performed a test of executive functioning.

Results: We found significantly fewer correct answers to explicit content questions after listening in noise. This negative effect was only mitigated to a marginally significant degree by audio-visual presentation. Strong executive function only predicted more correct answers in quiet settings.

Conclusions: Altogether, our results are inconclusive regarding how seeing a virtual speaker affects listening comprehension. We discuss how methodological adjustments, including modifications to our virtual speaker, can be used to discriminate between possible explanations to our results and contribute to understanding the listening conditions children face in a typical classroom.

Keywords:

• Listening
• comprehension
• noise
• education
• multimodality
• virtual humans
• education
• classroom

Acknowledgements

The authors gratefully acknowledge the master students, Sonny Aldenklint and Stephanie Meier, who collected the data; the Linneaus’ environment Cognition, Communication and Learning at Lund University for financial support; Professor Agneta Gulz at Lund University Cognitive Science division for valuable comments on the proof, Marianne Gullberg and Joost van de Weijer at Lund University Humanities Lab for advice on stimulus production and statistical analyses.

Disclosure statement

The authors report no declarations of interest.

Additional information

Funding

This work was supported by Vetenskapsrådet (Swedish Research Council) [grant no. 349-2007-8695].

Notes on contributors

Jens Nirme

Jens Nirme, PhD student with the Educational Technology Group (ETG), div. of Cognitive Science (LUCS) at Lund University since 2014, funded by the multidisciplinary research environment: Thinking in Time: Cognition, Communication and Learning (CCL). Main research interest in multimodal verbal communication. Also working with motion capture at the Lund University Humanities Lab.

Magnus Haake

Magnus Haake, Associate Professor in Cognitive Science. Part of the ETG (The Educational Technology Group) at the div. of Cognitive Science, Lund University, and the Noice & Voice-group at the div. of Logopedics, Phoniatrics, and Audiology, Lund University. Research areas: Cognitive Science, Learning Sciences, Educational Technology, and Social and Behavioral Psychology.

Viveka Lyberg Åhlander

Viveka Lyberg Åhlander, Speech Pathologist since 1999, PhD (Lund University) in 2011. Researching teachers’ voice health in classroom sound environments, with current focus on children’s perception, comprehension and cognitive capacities. Board member of the Sound Environment Centre at Lund University and the advanced study group AVaCO at the Pufendorf Institute.

Jonas Brännström

Jonas Brännström, DMSc., Associate professor, Dept. of Logopedics, Phoniatrics, and Audiology, Clinical Sciences Lund. Major research interests: Acceptance of background noise, Psychoacoustics and dichotic listening, Audiological rehabilitation.

Birgitta Sahlén

Birgitta Sahlén, Professor and research group leader at Dept. of Speech Language Pathology, Lund University. Heading several language intervention projects aimed at improving language learning environments for children with weak or vulnerable language (language disorder, with hearing loss) and multilingual children.