Suprasegmental lexical stress cues in visual speech can guide spoken-word recognition

REFERENCES

• Baayen, R. H., Piepenbrock, R., & van Rijn, H. (1995). The CELEX lexical database [CD-ROM]. Philadelphia, PA: University of Pennsylvania, Linguistic Data Consortium.
   Google Scholar
• Barkhuysen, P., Krahmer, E., & Swerts, M. (2005). The interplay between the auditory and visual modality for end-of-utterance detection. Journal of the Acoustical Society of America, 123, 354–365. doi:10.1121/1.2816561
   Web of Science ®Google Scholar
• Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278.
   Google Scholar
• Bates, D. M., & Sarkar, D. (2009). lme4: Linear mixed-effects models using s4 classes (R package version 0.999375-27) [Computer software]. Retrieved from http://cran.r-project.org/web/packages/lme4/index.html
   Google Scholar
• Beckman, M. E. (1986). Stress and non-stress accent. Dordrecht, The Netherlands: Foris. doi:10.1515/9783110874020
   Google Scholar
• Bernstein, L. E., Eberhard, S. P., & Demorest, M. E. (1989). Single-channel vibrotactile supplements to visual perception of intonation and stress. Journal of the Acoustical Society of America, 85, 397–405. doi: 10.1121/1.397690
   PubMed Web of Science ®Google Scholar
• Bolinger, D. (1958). A theory of pitch accent in English. Word, 14, 109–149.
   Web of Science ®Google Scholar
• Bond, Z. S. (1971). Units in speech perception. Ohio State University Working Papers in Linguistics, 9, 1–112.
   Google Scholar
• Booij, G. (1995). The phonology of Dutch. Oxford: Oxford University Press.
   Google Scholar
• Brown, G. (1977). Listening to spoken English. London: Longman.
   Google Scholar
• Cambier-Langeveld, T., & Turk, A. E. (1999). A cross-linguistic study of accentual lengthening: Dutch vs. English. Journal of Phonetics, 27, 255–280. doi: 10.1006/jpho.1999.0096
   Web of Science ®Google Scholar
• Chen, T., & Massaro, D. W. (2008). Seeing pitch: Visual information for lexical tones of Mandarin-Chinese. Journal of the Acoustical Society of America, 123, 2356–2366. doi: 10.1121/1.2839004
   PubMed Web of Science ®Google Scholar
• Cho, T. (2005). Prosodic strengthening and featural enhancement: Evidence from acoustic and articulatory realizations of /a,i/ in English. Journal of the Acoustical Society of America, 117, 3867–3878. doi: 10.1121/1.1861893
   PubMed Web of Science ®Google Scholar
• Cho, T., McQueen, J. M., & Cox, E. A. (2007). Prosodically-driven detail in speech processing: The case of domain-initial strengthening in English. Journal of Phonetics, 35, 210–243. doi: 10.1016/j.wocn.2006.03.003
   Web of Science ®Google Scholar
• Christophe, A., Peperkamp, S., Pallier, C., Block, E., & Mehler, J. (2004). Phonological phrase boundaries constrain lexical access. I. Adult data. Journal of Memory and Language, 51, 523–547. doi: 10.1016/j.jml.2004.07.001
   Web of Science ®Google Scholar
• Cohen, A., & 't Hart, J. (1967). On the anatomy of intonation. Lingua, 19, 177–192. doi: 10.1016/0024-3841(69)90118-1
   Web of Science ®Google Scholar
• Cole, R., Jakimik, J., & Cooper, W. E. (1978). Perceptibility of phonetic features in fluent speech. Journal of the Acoustical Society of America, 64, 44–56. doi: 10.1121/1.381955
   PubMed Web of Science ®Google Scholar
• Cooper, N., Cutler, A., & Wales, R. (2002). Constraints of lexical stress on lexical access in English: Evidence from native and non-native listeners. Language and Speech, 45, 207–228. doi: 10.1177/00238309020450030101
   PubMed Web of Science ®Google Scholar
• Cutler, A. (1986). Forbear is a homophone: Lexical prosody does not constrain lexical access. Language and Speech, 29, 201–220.
   Web of Science ®Google Scholar
• Cutler, A., & Clifton, C. E. (1984). The use of prosodic information in word recognition. In H. Bouma & D. G. Bouwhuis (Eds.), Attention and performance X: Control of language processes (pp. 183–196). Hillsdale, NJ: Erlbaum.
   Google Scholar
• Cutler, A., & van Donselaar, W. (2001). Voornaam is not a homophone: Lexical prosody and lexical access in Dutch. Language and Speech, 44, 171–195. doi: 10.1177/00238309010440020301
   PubMed Web of Science ®Google Scholar
• Cutler, A., & Pasveer, D. (2006). Explaining cross-linguistic differences in effects of lexical stress on spoken-word recognition. In R. Hoffman & H. Mixdorff (Eds.), Proceedings of the Third International Conference on Speech Prosody (pp. 250–254). Dresden, Germany: TUD Press.
   Google Scholar
• Dohen, M., & Loevenbruck, H. (2009). Interaction of audition and vision for the perception of prosodic contrastive focus. Language and Speech, 52, 177–206. doi: 10.1177/0023830909103166
   PubMed Web of Science ®Google Scholar
• Dohen, M., Loevenbruck, H., Cathiard, M.-A., & Schwartz, J.-L. (2004). Visual perception of contrastive focus in reiterant French speech. Speech Communication, 44, 155–172. doi: 10.1016/j.specom.2004.10.009
   Web of Science ®Google Scholar
• van Donselaar, W., Koster, M., & Cutler, A. (2005). Exploring the role of lexical stress in lexical recognition. The Quarterly Journal of Experimental Psychology, 58A, 251–273.
   Google Scholar
• Eefting, W. (1991). The effect of “information value” and “accentuation” on the duration of Dutch words, syllables, and segments. Journal of the Acoustical Society of America, 89, 411–424. doi: 10.1121/1.400475
   Web of Science ®Google Scholar
• Erickson, D. (1998). Effects of contrastive emphasis on jaw opening. Phonetica, 55, 147–169. doi: 10.1159/000028429
   PubMed Web of Science ®Google Scholar
• Fear, B. D., Cutler, A., & Butterfield, S. (1995). The strong/weak syllable distinction in English. Journal of the Acoustical Society of America, 97, 1893–1904. doi: 10.1121/1.412063
   PubMed Web of Science ®Google Scholar
• Fisher, C. G. (1969). The visibility of terminal pitch contour. Journal of Speech and Hearing Research, 12, 379–382.
   PubMedGoogle Scholar
• Fowler, C. (1995). Acoustic and kinematic correlates of contrastive stress accent in spoken English. In F. Bell-Berti & L. J. Raphael (Eds.), Producing speech: Contemporary issues. For Katherine Safford Harris (pp. 355–373). New York, NY: AIP Press.
   Google Scholar
• Fry, D. B. (1958). Experiments in the perception of stress. Language & Speech, 1, 126–152.
   Web of Science ®Google Scholar
• Harrington, J., Fletcher, J., & Roberts, C. (1995). Coarticulation and the accented/unaccented distinction: Evidence from jaw movement data. Journal of Phonetics, 23, 305–322. doi: 10.1016/S0095-4470(95)80163-4
   Web of Science ®Google Scholar
• van Heuven, V. J. (1988). Effects of stress and accent on the human recognition of word fragments in spoken context: Gating and shadowing. In W. A. Ainsworth & J. N. Holmes (Eds.), Proceedings of Speech ’88, 7th FASE symposium (pp. 811–818). Edinburgh: Institute of Acoustics.
   Google Scholar
• van Heuven, V. J., & Hagman, P. J. (1988). Lexical statistics and spoken word recognition in Dutch. In P. Coopmans & A. Hulk (Eds.), Linguistics in the Netherlands 1988 (pp. 59–69). Dordrecht, the Netherlands: Foris.
   Google Scholar
• Jesse, A. (2005). Towards a lexical fuzzy logical model of perception: The time-course of information in lexical identification of face-to-face speech (Unpublished doctoral dissertation). University of California, Santa Cruz, CA.
   Google Scholar
• Jesse, A., & Massaro, D. W. (2010). The temporal distribution of information in audiovisual spoken-word identification. Attention, Perception, & Psychophysics, 72, 209–225. doi: 10.3758/APP.72.1.209
   PubMed Web of Science ®Google Scholar
• de Jong, K. J. (1995). The supraglottal articulation of prominence in English: Linguistic stress as localized hyperarticulation. Journal of the Acoustical Society of America, 97, 491–504. doi: 10.1121/1.412275
   PubMed Web of Science ®Google Scholar
• Kelso, J. A. S., Vatikiotis-Bateson, E., Saltzman, E., & Kay, B. A. (1985). A qualitative dynamic analysis of reiterant speech production: Phase portraits, kinematics, and dynamic modeling. Journal of the Acoustical Society of America, 77, 266–280. doi: 10.1121/1.392268
   PubMed Web of Science ®Google Scholar
• Krahmer, E. J., & Swerts, M. (2007). The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception. Journal of Memory and Language, 57, 396–414. doi: 10.1016/j.jml.2007.06.005
   Web of Science ®Google Scholar
• Lansing, C. R., & McConkie, G. W. (1999). Attention to facial regions in the segmental and prosodic visual speech perception tasks. Journal of Speech, Language, and Hearing Research, 42, 526–539.
   PubMed Web of Science ®Google Scholar
• Lieberman, P. (1960). Some acoustic correlates of word stress in American English. Journal of the Acoustical Society of America, 33, 451–454. doi: 10.1121/1.1908095
   Web of Science ®Google Scholar
• Lieberman, P. (1963). Some effects of semantic and grammatical context on the production and perception of speech. Language and Speech, 6, 172–187.
   Web of Science ®Google Scholar
• Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioural principle. Cambridge, MA: MIT Press.
   Google Scholar
• Massaro, D. W., & Jesse, A. (2008). Audiovisual speech perception and word recognition. In G. Gaskell (Ed.), The Oxford handbook of psycholinguistics (pp. 19–35). Oxford: Oxford University Press.
   Google Scholar
• Mattys, S. L. (2000). The perception of primary and secondary stress in English. Perception & Psychophysics, 62(2), 253–265. doi: 10.3758/BF03205547
   PubMed Web of Science ®Google Scholar
• Mattys, S. L., & Samuel, A. G. (1997). How lexical stress affects speech segmentation and interactivity: Evidence from the migration paradigm. Journal of Memory and Language, 36, 87–116. doi: 10.1006/jmla.1996.2472
   Web of Science ®Google Scholar
• McAllister, J. (1991). The processing of lexically stressed syllables in read and spontaneous speech. Language and Speech, 34, 1–26.
   Web of Science ®Google Scholar
• McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748. doi: 10.1038/264746a0
   PubMed Web of Science ®Google Scholar
• Miller, G. A., & Nicely, P. (1955). An analysis of perceptual confusions among some English consonants. Journal of the Acoustical Society of America, 27, 338–352. doi: 10.1121/1.1907526
   Web of Science ®Google Scholar
• NESU [Computer software]. Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics. Retrieved from http://www.mpi.nl/world/tools/nesu.html
   Google Scholar
• Nicholson, K. G., Baum, S., Cuddy, L . L., & Munhall, K. G. (2002). A case of impaired auditory and visual speech prosody perception after right hemisphere damage. Neurocase, 8, 314–322. doi: 10.1076/neur.8.3.314.16195
   PubMed Web of Science ®Google Scholar
• Norris, D., & McQueen, J. M. (2008). Shortlist B: A Bayesian model of continuous speech recognition. Psychological Review, 115, 357–395. doi: 10.1037/0033-295X.115.2.357
   PubMed Web of Science ®Google Scholar
• Plag, I., Kunter, G., & Schramm, M. (2011). Acoustic correlates of primary and secondary stress in North American English. Journal of Phonetics, 39, 362–374. doi: 10.1016/j.wocn.2011.03.004
   Web of Science ®Google Scholar
• R Development Core Team. (2007). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org
   Google Scholar
• Reinisch, E., Jesse, A., & McQueen, J. M. (2010). Early use of phonetic information in spoken word recognition: Lexical stress drives eye movements immediately. The Quarterly Journal of Experimental Psychology, 64(4), 772–783. doi: 10.1080/17470210903104412
   Web of Science ®Google Scholar
• Reisberg, D., McLean, J. M., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97–113). Hillsdale, NJ: Lawrence Erlbaum Associates.
   Google Scholar
• Rietveld, T., Kerkhoff, J., & Gussenhoven, C. (2004). Word prosodic structure and vowel duration in Dutch. Journal of Phonetics, 32, 349–371. doi: 10.1016/j.wocn.2003.08.002
   Web of Science ®Google Scholar
• Risberg, A. M., & Lubker, J. (1978). Prosody and speech-reading. Speech Transmission Laboratory Quarterly Progress Status Report, 4, 1–16.
   Google Scholar
• Salverda, A.-P., Dahan, D., & McQueen, J. M. (2003). The role of prosodic boundaries in the resolution of lexical embedding in speech comprehension. Cognition, 90, 51–89. doi: 10.1016/S0010-0277(03)00139-2
   PubMed Web of Science ®Google Scholar
• Scarborough, R., Keating, P., Mattys, S. L., Cho, T., & Alwan, A. (2009). Optical phonetics and visual perception of lexical and phrasal stress in English. Language and Speech, 52, 135–175. doi: 10.1177/0023830909103165
   PubMed Web of Science ®Google Scholar
• Slootweg, A. (1988). Metrical prominence and syllable duration. In P. Coopmans & A. Hulk (Eds.), Linguistics in the Netherlands 1988 (pp. 139–148). Dordrecht: Foris.
   Google Scholar
• Sluijter, A. M., & van Heuven, V. J. (1995). Effects of focus distribution, pitch accent and lexical stress on the temporal organization of syllables in Dutch. Phonetica, 52, 71–89. doi: 10.1159/000262061
   Web of Science ®Google Scholar
• Sluijter, A. M., & van Heuven, V. J. (1996). Spectral balance as an acoustic correlate of linguistic stress. Journal of the Acoustical Society of America, 100, 2471–2485. doi: 10.1121/1.417955
   PubMed Web of Science ®Google Scholar
• van Son, N., Huiskamp, T. M. I., Bosman, A. J., & Smoorenburg, G. F. (1994). Viseme classifications of Dutch consonants and vowels. Journal of the Acoustical Society of America, 96, 1341–1355. doi:10.1121/1.411324
   Google Scholar
• Soto-Faraco, S., Sebastián-Gallés, N., & Cutler, A. (2001). Segmental and suprasegmental mismatch in lexical access. Journal of Memory and Language, 45, 412–432. doi: 10.1006/jmla.2000.2783
   Web of Science ®Google Scholar
• Srinivasan, R. J., & Massaro, D. W. (2003). Perceiving prosody from the face and voice: Distinguishing statements from echoic questions in English. Language and Speech, 46, 1–22. doi: 10.1177/00238309030460010201
   PubMed Web of Science ®Google Scholar
• Sulpizio, S., & McQueen, J. M. (2012). Italians use abstract knowledge about lexical stress during spoken-word recognition. Journal of Memory and Language, 66, 177–193. doi: 10.1016/j.jml.2011.08.001
   Web of Science ®Google Scholar
• Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215. doi: 10.1121/1.1907309
   Web of Science ®Google Scholar
• Summers, W. V. (1987). Effects of stress and final-consonant voicing on vowel production: Articulatory and acoustic analyses. Journal of the Acoustical Society of America, 82, 847–863. doi: 10.1121/1.395284
   PubMed Web of Science ®Google Scholar
• Swerts, M., & Krahmer, E. J. (2008). Facial expressions and prosodic prominence: Comparing modalities and facial areas. Journal of Phonetics, 36, 219–238. doi: 10.1016/j.wocn.2007.05.001
   Web of Science ®Google Scholar
• Thompson, D. M. (1934). On the detection of emphasis in spoken sentences by means of visual, tactual, and visual-tactual cues. Journal of General Psychology, 11, 160–172. doi: 10.1080/00221309.1934.9917823
   Google Scholar
• Walden, B. E., Prosek, R. A., & Worthington, D. W. (1974). Predicting audiovisual consonant recognition performance of hearing-impaired adults. Journal of Speech & Hearing Research, 17, 270–278.
   PubMedGoogle Scholar