Tongue reading: Comparing the interpretation of visual information from inside the mouth, from electropalatographic and ultrasound displays of speech sounds

Abstract

Speakers possess a natural capacity for lip reading; analogous to this, there may be an intuitive ability to “tongue-read.” Although the ability of untrained participants to perceive aspects of the speech signal has been explored for some visual representations of the vocal tract (e.g. talking heads), it is not yet known to what extent there is a natural ability to interpret speech information presented through two clinical phonetic tools: EPG and ultrasound. This study aimed to determine whether there is any intuitive ability to interpret the images produced by these systems, and to determine whether one tool is more conducive to this than the other. Twenty adults viewed real-time and slow motion EPG and ultrasound silent movies of 10 different linguo-palatal consonants and 4 vowels. Participants selected which segment they perceived from four forced-choice options. Overall, participants scored above chance in the EPG and ultrasound conditions, suggesting that these images can be interpreted intuitively to some degree. This was the case for consonants in both the conditions and for vowels in the EPG condition.

Keywords:

• electropalatography
• ultrasound
• speech reading
• visual feedback

Acknowledgments

We thank all the participants who gave up their time to complete the experiment and Steve Cowen for technical assistance.

Declaration of Interest: The first author is supported by an EPSRC grant (Grant No.: EP/I027696/1). The authors report no conflict of interest.