Sub-visemic discrimination and the effect of visual resemblance on silent lip-reading

ABSTRACT

Relevant visual information is available in speakers’ faces during face-to-face interactions that improve speech perception. There is an ongoing debate, however, about how phonemes and their visual counterparts, visemes are mapped. An influential hypothesis claims that several phonemes can be mapped into a single visemic category (many-to-one phoneme-viseme mapping). In contrast, recent findings have challenged this view, reporting evidence for sub-visemic syllable discrimination. We aim to investigate whether Spanish words from the same visemic category can be identified or not. We designed a lip-reading task in which participants had to identify target words presented in silent video clips among 3 distractors differing in their visual resemblance from the target. Target words were identified above chance and significantly more than distractors from the same visemic category. Moreover, the error rate for distractors significantly decreased with decreasing visemic resemblance to the target. These results challenge the many-to-one phoneme-viseme mapping hypothesis.

KEYWORDS:

• Lip-reading
• visemic discrimination
• phoneme-viseme mapping
• visual speech
• orofacial movements

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Data are publicly available on Open Science Framework (OSF) at DOI 10.17605/OSF.IO/KJA6H

 Google Scholar

.

Institutional review board statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Ethics Committee of the Faculty of Medicine, Pontificia Católica Universidad de Chile (protocol code 190912007, date of approval March 20th 2021).

Additional information

Funding

This work was supported by the ANID Fondecyt Postdoctoral under Grant Folio 3201057; the ANID Fondecyt Regular under Grant Folio 1210659; and the ANID Fondecyt Regular under Grant Folio 1211227