Visual speech segmentation: using facial cues to locate word boundaries in continuous speech

References

• Aslin, R. N., Saffran, J. R., & Newport, E. L. (1998). Computation of conditional probability statistics by human infants. Psychological Science, 9, 321–324. doi:10.1111/1467-9280.00063
   Web of Science ®Google Scholar
• Bahrick, L. E., & Lickliter, R. (2000). Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Developmental Psychology, 36(2), 190–201. doi:10.1037/0012-1649.36.2.190
   PubMed Web of Science ®Google Scholar
• Blossom, M., & Morgan, J. L. (2006). Does the face say what the mouth says? A study of infants' sensitivity to visual prosody. In D. Bamman, T. Magnitskaia, & C. Zaller (Eds.), Proceedings of the 30th annual Boston University conference on language development, (pp. 24–35). Somerville, MA: Cascadilla Press.
   Google Scholar
• Brent, M., & Cartwright, T. (1996). Distributional regularity and phonotactic constraints are useful for segmentation. Cognition, 61(1–2), 93–125. doi:10.1016/S0010-0277(96)00719-6
   PubMed Web of Science ®Google Scholar
• Christophe, A., Gout, A., Peperkamp, S., & Morgan, J. (2003). Discovering words in the continuous speech stream: The role of prosody. Journal of Phonetics, 31, 585–598. doi:10.1016/S0095-4470(03)00040-8
   Web of Science ®Google Scholar
• Christophe, A., Peperkamp, S., Pallier, C., Block, E., & Mehler, J. (2004). Phonological phrase boundaries constrain lexical access: I. Adult data. Journal of Memory and Language, 51, 523–547. doi:10.1016/j.jml.2004.07.001
   Web of Science ®Google Scholar
• Cunillera, T., Càmara, E., Laine, M., & Rodríguez-Fornells, A. (2010). Speech segmentation is facilitated by visual cues. The Quarterly Journal of Experimental Psychology, 63, 260–274. doi:10.1080/17470210902888809
   Web of Science ®Google Scholar
• Emberson, L. L., Conway, C. M., & Christiansen, M. H. (2011). Timing is everything: Changes in presentation rate have opposite effects on auditory and visual implicit statistical learning. Quarterly Journal of Experimental Psychology, 64, 1021–1040. doi:10.1080/17470218.2010.538972
   Web of Science ®Google Scholar
• Endress, A. D., & Hauser, M. D. (2010). Word segmentation with universal prosodic cues. Cognitive Psychology, 61(2), 177–199. doi:10.1016/j.cogpsych.2010.05.001
   PubMed Web of Science ®Google Scholar
• Frank, M. C., Goldwater, S., Griffiths, T., & Tenenbaum, J. B. (2010). Modeling human performance in statistical word segmentation. Cognition, 117(2), 107–125. doi:10.1016/j.cognition.2010.07.005
   PubMed Web of Science ®Google Scholar
• Friederici, A. D., & Wessels, J. M. (1993). Phonotactic knowledge and its use in infant speech perception. Perception & Psychophysics, 54, 287–295. doi:10.3758/BF03205263
   PubMed Web of Science ®Google Scholar
• Goren, C., Sarty, M., & Wu, P. (1975). Visual following and pattern discrimination of face-like stimuli by newborn infants. Pediatrics, 56, 544–549.
   PubMed Web of Science ®Google Scholar
• Gout, A., Christophe, A., & Morgan, J. L. (2004). Phonological phrase boundaries constrain lexical access: II. Infant data. Journal of Memory and Language, 51, 548–567. doi:10.1016/j.jml.2004.07.002
   Web of Science ®Google Scholar
• Graf, P. H., Cosatto, E., Strom, V., & Huang, F. J. (2002). Visual prosody: Facial movements accompanying speech. In Proceedings of the fifth IEEE international conference on automatic face and gesture recognition, Washington DC.
   Google Scholar
• Grant, K. W., & Seitz, P. F. (2000). The use of visible speech cues for improving auditory detection of spoken sentences. The Journal of the Acoustical Society of America, 108, 1197–1208. doi:10.1121/1.1288668
   PubMed Web of Science ®Google Scholar
• Green, J. R., Nip, I. S. B., Wilson, E. M., Mefferd, A. S., & Yunusova, Y. (2010). Lip movement exaggerations during infant-directed speech. Journal of Speech, Language, and Hearing Research, 53, 1529–1542. doi:10.1044/1092-4388(2010/09-0005)
   PubMed Web of Science ®Google Scholar
• Hollich, G., Newman, R. S., & Jusczyk, P. W. (2005). Infants' use of synchronized visual information to separate streams of speech. Child Development, 76, 598–613. doi:10.1111/j.1467-8624.2005.00866.x
   PubMed Web of Science ®Google Scholar
• Houston, D. M., Jusczyk, P. W., Kuijpers, C., Coolen, R., & Cutler, A. (2000). Cross language word segmentation by 9-month-olds. Psychonomic Bulletin & Review, 7, 504–509. doi:10.3758/BF03214363
   PubMed Web of Science ®Google Scholar
• Houston, D. M., Pisoni, D. B., Kirk, K. I., Ying, E. A., & Miyamoto, R. T. (2003). Speech perception skills of deaf infants following cochlear implantation: A first report. International Journal of Pediatric Otorhinolaryngology, 67, 479–495. doi:10.1016/S0165-5876(03)00005-3
   PubMed Web of Science ®Google Scholar
• Johnson, E. K., & Jusczyk, P. W. (2001). Word segmentation by 8-month-olds: When speech cues count more than statistics. Journal of Memory and Language, 44, 548–567. doi:10.1006/jmla.2000.2755
   Web of Science ®Google Scholar
• Jusczyk, P. W., Houston, D. M., & Newsome, M. (1999). The beginnings of word segmentation in English-learning infants. Cognitive Psychology, 39, 159–207. doi:10.1006/jmla.2000.2755
   PubMed Web of Science ®Google Scholar
• Kuhl, P. K., & Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138–1141. doi:10.1126/science.7146899
   PubMed Web of Science ®Google Scholar
• Lea, R. B., & Cohen, B. H. (2004). Essentials of statistics for the social and behavioral sciences. Hoboken, NJ: John Wiley & Sons.
   Google Scholar
• Lewkowicz, D. J. (2010). Infant perception of audio-visual speech synchrony. Developmental Psychology, 46(1), 66–77. doi:10.1037/a0015579
   PubMed Web of Science ®Google Scholar
• Light, L. L., Kayra-Stuart, F., & Hollander, S. (1979). Recognition memory for typical and unusual faces. Journal of Experimental Psychology: Human Learning and Memory, 5, 212–228. doi:10.1037/0278-7393.5.3.212
   PubMedGoogle Scholar
• Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, MA: MIT Press.
   Google Scholar
• Massaro, D. W., & Light, J. (2003). Read my tongue movements: Bimodal learning to perceive and produce non-native speech /r/ and /l/. In Proceedings of Eurospeech (Interspeech), 8th European conference on speech communication and technology. Geneva, Switzerland, .
   Google Scholar
• Massaro, D. W., & Light, J. (2004). Using visible speech for training perception and production of speech for hard of hearing individuals. Journal of Speech, Language, and Hearing Research, 47, 304–320. doi:10.1044/1092-4388(2004/025)
   PubMed Web of Science ®Google Scholar
• McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748. doi:10.1038/264746a0
   PubMed Web of Science ®Google Scholar
• Mitchel, A. D., Christiansen, M. H., & Weiss, D. J. (in review). Cross-modal effects in statistical learning: Evidence from the McGurk illusion. Manuscript submitted for publication.
   Google Scholar
• Mitchel, A. D., & Weiss, D. J. (2010). What's in a face? Visual contributions to speech segmentation. Language and Cognitive Processes, 25, 456–482. doi:10.1080/01690960903209888
   Web of Science ®Google Scholar
• Mitchel, A. D., & Weiss, D. J. (2011). Learning across senses: Cross-modal effects in multisensory statistical learning. Journal of Experimental Psychology: Learning, Memory & Cognition, 37, 1081–1091. doi:10.1037/a0023700
   PubMedGoogle Scholar
• Morton, J., & Johnson, M. H. (1991). CONSPEC and CONLERN: A two-process theory of infant face recognition. Psychological Review, 98(2), 164–181. doi:10.1037/0033-295X.98.2.164
   PubMed Web of Science ®Google Scholar
• Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility. Psychological Science, 15(2), 133–137. doi:10.1111/j.0963-7214.2004.01502010.x
   PubMed Web of Science ®Google Scholar
• Patterson, M., & Werker, J. F. (1999). Matching phonetic information in lips and voice is robust in 4.5-month-old infants. Infant Behavior and Development, 22, 237–247. doi:10.1016/S0163-6383(99)00003-X
   Web of Science ®Google Scholar
• Patterson, M. L., & Werker, J. F. (2003). Two-month-old infants match phonetic information in lips and voice. Developmental Science, 6, 191–196. doi:10.1111/1467-7687.00271
   Web of Science ®Google Scholar
• Rosenblum, L. D., & Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception. Journal of Experimental Psychology: Human Perception and Performance, 22, 318–331. doi:10.1037/0096-1523.22.2.318
   PubMed Web of Science ®Google Scholar
• Saffran, J. R., Newport, E. L., & Aslin, R. N. (1996). Word segmentation: The role of distributional cues. Journal of Memory and Language, 35, 606–621. doi:10.1006/jmla.1996.0032
   Web of Science ®Google Scholar
• Seitz, A. R., Kim, R., van Wassenhove, V., & Shams, L. (2007). Simultaneous and independent acquisition of multisensory and unisensory associations. Perception, 36, 1445–1453. doi:10.1068/p5843
   PubMed Web of Science ®Google Scholar
• Sell, A. J., & Kaschak, M. P. (2009). Does visual speech information affect word segmentation? Memory and Cognition, 37, 889–894. doi:10.3758/MC.37.6.889
   PubMedGoogle Scholar
• Simion, F., Valenza, E., Macchi-Cassia, V., Turati, C., & Umilta, C. (2002). Newborns' preference for up-down asymmetrical configurations. Developmental Science, 5, 427–434. doi:10.1111/1467-7687.00237
   Web of Science ®Google Scholar
• Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215. doi:10.1121/1.1907309
   Web of Science ®Google Scholar
• Teinonen, T., Aslin, R. N., Alku, P., & Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition, 108, 850–855. doi:10.1016/j.cognition.2008.05.009
   PubMed Web of Science ®Google Scholar
• Thiessen, E. D. (2010). Effects of visual information on adults' and infants' auditory statistical learning. Cognitive Science, 34, 1093–1106. doi:10.1111/j.1551-6709.2010.01118.x
   PubMedGoogle Scholar
• Weikum, W. M., Vouloumanos, A., Navarra, J., Soto-Faraco, S., Sebastian-Galles, N., & Werker, J. F. (2007). Visual language discrimination in infancy. Science, 316, 1159.doi:10.1126/science.1137686
   PubMed Web of Science ®Google Scholar
• Weiss, D. J., Gerfen, C., & Mitchel, A. D. (2009). Speech segmentation in a simulated bilingual environment: A challenge for statistical learning? Language Learning and Development, 5(1), 30–49. doi:10.1080/15475440802340101
   PubMedGoogle Scholar
• Weiss, D. J., Gerfen, C., & Mitchel, A. D. (2010). Colliding cues in word segmentation: The role of cue strength and general cognitive processes. Language and Cognitive Processes, 25, 402–422. doi:10.1080/01690960903212254
   Web of Science ®Google Scholar
• Yehia, H. C., Kuratate, T., & Vaitikiotis-Bateson, E. (2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 30, 555–568. doi:10.1006/jpho.2002.0165
   Web of Science ®Google Scholar