Speech intelligibility of virtual humans

Abstract

The speech intelligibility benefit of visual speech cues during oral communication is well-established. Therefore, an ecologically valid approach of auditory assessment should include the processing of both auditory and visual speech cues. This study describes the development and evaluation of a virtual human speaker designed to present speech auditory-visually. A male and female virtual human speaker were created and evaluated in two experiments: a visual-only speech reading test of words and sentences and an auditory-visual speech intelligibility sentence test. A group of five hearing, skilled speech reading adults participated in the speech reading test whereas a group of young normal-hearing participants (N = 35) was recruited for the intelligibility test. Skilled speech readers correctly identified 57 to 67% of the words and sentences uttered by the virtual speakers. The presence of the virtual speaker improved the speech intelligibility of sentences in noise by 1.5 to 2 dB. These results demonstrate the potential applicability of virtual humans in future auditory-visual speech assessment paradigms.

Keywords:

• Speech perception
• behavioural measures
• psychoacoustics/hearing science
• instrumentation
• virtual humans
• auditory-visual speech

Acknowledgements

The authors would like to thank all the participants who took part in this study, as well as the students from the Speech Language Pathology and Audiology Sciences educational programme for their help with data collection. Sam Denys is gratefully acknowledged for his help with the speech reading video recordings.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Oticon Fonden and a TBM-FWO grant from the Research Foundation-Flanders under Grant [T002216N].