Audiovisual integration in familiar person recognition

References

• van Atteveldt, N. M., Formisano, E., Blomert, L., & Goebel, R. (2007). The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cerebral Cortex, 17, 962–974. doi: 10.1093/cercor/bhl007
   PubMed Web of Science ®Google Scholar
• Belin, P., Bestelmeyer, P. E. G., Latinus, M., & Watson, R. (2011). Understanding voice perception. British Journal of Psychology, 102, 711–725. doi: 10.1111/j.2044-8295.2011.02041.x
   PubMed Web of Science ®Google Scholar
• Belin, P., Campanella, S., & Ethofer, T. (2013). Integrating face and voice in person perception. New York, Heidelberg: Springer.
   Google Scholar
• Bernstein, L. E., Auer, E. T., Wagner, M., & Ponton, C. W. (2008). Spatiotemporal dynamics of audiovisual speech processing. NeuroImage, 39, 423–435. doi: 10.1016/j.neuroimage.2007.08.035
   PubMed Web of Science ®Google Scholar
• Blank, H., Anwander, A., & von Kriegstein, K. (2011). Direct structural connections between Voice- and Face-recognition areas. Journal of Neuroscience, 31, 12906–12915. doi: 10.1523/JNEUROSCI.2091-11.2011
   PubMed Web of Science ®Google Scholar
• Bristow, D., Dehaene-Lambertz, G., Mattout, J., Soares, C., Gliga, T., Baillet, S., …  Mangin, J-F. (2009). Hearing faces: How the infant brain matches the face it sees with the speech it hears. Journal of Cognitive Neuroscience, 21, 905–921. doi: 10.1162/jocn.2009.21076
   PubMed Web of Science ®Google Scholar
• Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77, 305–327. doi: 10.1111/j.2044-8295.1986.tb02199.x
   PubMed Web of Science ®Google Scholar
• Burton, A. M., Bruce, V., & Hancock, P. J. B. (1999). From pixels to people: A model of familiar face recognition. Cognitive Science, 23, 1–31. doi: 10.1207/s15516709cog2301_1
   Web of Science ®Google Scholar
• Burton, A. M., Bruce, V., & Johnston, R. A. (1990). Understanding face recognition with an interactive activation model. British Journal of Psychology, 81, 361–380. doi: 10.1111/j.2044-8295.1990.tb02367.x
   PubMed Web of Science ®Google Scholar
• Burton, A. M., Jenkins, R., & Schweinberger, S. R. (2011). Mental representations of familiar faces. British Journal of Psychology, 102, 943–958. doi: 10.1111/j.2044-8295.2011.02039.x
   PubMed Web of Science ®Google Scholar
• Burton, A. M., Wilson, S., Cowan, M., & Bruce, V. (1999). Face recognition in poor-quality video: Evidence from security surveillance. Psychological Science, 10, 243–248. doi: 10.1111/1467-9280.00144
   Web of Science ®Google Scholar
• Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., et al. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593–596. doi: 10.1126/science.276.5312.593
   PubMed Web of Science ®Google Scholar
• Campanella, S., & Belin, P. (2007). Integrating face and voice in person perception. Trends in Cognitive Sciences, 11, 535–543. doi: 10.1016/j.tics.2007.10.001
   PubMed Web of Science ®Google Scholar
• Doesburg, S. M., Emberson, L. L., Rahi, A., Cameron, D., & Ward, L. M. (2008). Asynchrony from synchrony: Long-range gamma-band neural synchrony accompanies perception of audiovisual speech asynchrony. Experimental Brain Research, 185, 11–20. doi: 10.1007/s00221-007-1127-5
   PubMed Web of Science ®Google Scholar
• Ellis, H. D., Jones, D. M., & Mosdell, N. (1997). Intra- and inter-modal repetition priming of familiar faces and voices. British Journal of Psychology, 88, 143–156. doi: 10.1111/j.2044-8295.1997.tb02625.x
   PubMed Web of Science ®Google Scholar
• Föcker, J., Holig, C., Best, A., & Röder, B. (2011). Crossmodal interaction of facial and vocal person identity information: An event-related potential study. Brain Research, 1385, 229–245. doi: 10.1016/j.brainres.2011.02.021
   PubMed Web of Science ®Google Scholar
• Freeman, J. B., & Ambady, N. (2011). When two become one: Temporally dynamic integration of the face and voice. Journal of Experimental Social Psychology, 47, 259–263. doi: 10.1016/j.jesp.2010.08.018
   Web of Science ®Google Scholar
• Gainotti, G. (2007). Different patterns of famous people recognition disorders in patients with right and left anterior temporal lesions: A systematic review. Neuropsychologia, 45, 1591–1607. doi: 10.1016/j.neuropsychologia.2006.12.013
   PubMed Web of Science ®Google Scholar
• Gainotti, G. (2011). What the study of voice recognition in normal subjects and brain-damaged patients tells us about models of familiar people recognition. Neuropsychologia, 49, 2273–2282. doi: 10.1016/j.neuropsychologia.2011.04.027
   PubMed Web of Science ®Google Scholar
• Gainotti, G. (2014). Cognitive models of familiar people recognition and hemispheric asymmetries. Frontiers in Bioscience, E6, 159–174. doi: 10.2741/E698
   Google Scholar
• Gainotti, G., Barbier, A., & Marra, C. (2003). Slowly progressive defect in recognition of familiar people in a patient with right anterior temporal atrophy. Brain, 126, 792–803. doi: 10.1093/brain/awg092
   PubMed Web of Science ®Google Scholar
• Garrido, L., Eisner, F., McGettigan, C., Stewart, L., Sauter, D., Hanley, J. R., … Duchaine, B. (2009). Developmental phonagnosia: A selective deficit of vocal identity recognition. Neuropsychologia, 47, 123–131. doi: 10.1016/j.neuropsychologia.2008.08.003
   PubMed Web of Science ®Google Scholar
• de Gelder, B., & Vroomen, J. (2000). Bimodal emotion perception: Integration across separate modalities, cross-modal perceptual grouping or perception of multimodal events? Cognition & Emotion, 14, 321–324. doi: 10.1080/026999300378842
   Web of Science ®Google Scholar
• Gonzalez, I. Q., Leon, M. A. B., Belin, P., Martinez-Quintana, Y., Garcia, L. G., & Castillo, M. S. (2011). Person identification through faces and voices: An ERP study. Brain Research, 1407, 13–26. doi: 10.1016/j.brainres.2011.03.029
   PubMed Web of Science ®Google Scholar
• Hagan, C. C., Woods, W., Johnson, S., Calder, A. J., Green, G. G. R., & Young, A. W. (2009). MEG demonstrates a supra-additive response to facial and vocal emotion in the right superior temporal sulcus. Proceedings of the National Academy of Sciences of the United States of America, 106, 20010–20015. doi: 10.1073/pnas.0905792106
   PubMed Web of Science ®Google Scholar
• Hahn, C. A., O'Toole, A. J., & Phillips, P. J. (2016). Dissecting the time course of person recognition in natural viewing environments. British Journal of Psychology, 107(1), 117–134. doi: 10.1111/bjop.12125
   PubMed Web of Science ®Google Scholar
• Hancock, P. J. B., Bruce, V., & Burton, A. M. (2000). Recognition of unfamiliar faces. Trends in Cognitive Sciences, 4, 330–337. doi: 10.1016/S1364-6613(00)01519-9
   PubMed Web of Science ®Google Scholar
• Hanley, J. R. (2014). Accessing stored knowledge of familiar people from faces, names and voices: A review. Frontiers in Bioscience, E6, 198–207. doi: 10.2741/E702
   Google Scholar
• Hanley, J. R., & Damjanovic, L. (2009). It is more difficult to retrieve a familiar person's name and occupation from their voice than from their blurred face. Memory, 17, 830–839. doi: 10.1080/09658210903264175
   PubMed Web of Science ®Google Scholar
• Hanley, J. R., Smith, S. T., & Hadfield, J. (1998). I recognise you but I cańt place you: An investigation of familiar-only experiences during tests of voice and face recognition. Quarterly Journal of Experimental Psychology, 51A, 179–195. doi: 10.1080/713755751
   Web of Science ®Google Scholar
• Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4, 223–233. doi: 10.1016/S1364-6613(00)01482-0
   PubMed Web of Science ®Google Scholar
• Ho, H. T., Schroger, E., & Kotz, S. A. (2015). Selective attention modulates early human evoked potentials during emotional face-voice processing. Journal of Cognitive Neuroscience, 27(4), 798–818. doi: 10.1162/jocn_a_00734
   PubMed Web of Science ®Google Scholar
• Huynh, H., & Feldt, L. S. (1976). Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. Journal of Educational Statistics, 1, 69–82. doi: 10.2307/1164736
   Google Scholar
• Hyde, D. C., Jones, B. L., Flom, R., & Porter, C. L. (2011). Neural signatures of face-voice synchrony in 5-month-old human infants. Developmental Psychobiology, 53, 359–370. doi: 10.1002/dev.20525
   PubMed Web of Science ®Google Scholar
• Joassin, F., Maurage, P., Bruyer, R., Crommelinck, M., & Campanella, S. (2004). When audition alters vision: An event-related potential study of the cross-modal interactions between faces and voices. Neuroscience Letters, 369, 132–137. doi: 10.1016/j.neulet.2004.07.067
   PubMed Web of Science ®Google Scholar
• Joassin, F., Pesenti, M., Maurage, P., Verreckt, E., Bruyer, R., & Campanella, S. (2011). Cross-modal interactions between human faces and voices involved in person recognition. Cortex, 47, 367–376. doi: 10.1016/j.cortex.2010.03.003
   PubMed Web of Science ®Google Scholar
• Jordan, T. R., McCotter, M. V., & Thomas, S. M. (2000). Visual and audiovisual speech perception with color and gray-scale facial images. Perception & Psychophysics, 62, 1394–1404. doi: 10.3758/BF03212141
   PubMed Web of Science ®Google Scholar
• Kamachi, M., Hill, H., Lander, K., & Vatikiotis-Bateson, E. (2003). ‘Putting the face to the voice’: Matching identity across modality. Current Biology, 13, 1709–1714. doi: 10.1016/j.cub.2003.09.005
   PubMed Web of Science ®Google Scholar
• Kaufmann, J. M., & Schweinberger, S. R. (2004). Expression influences the recognition of familiar faces. Perception, 33, 399–408. doi: 10.1068/p5083
   PubMed Web of Science ®Google Scholar
• Knight, B., & Johnston, A. (1997). The role of movement in face recognition. Visual Cognition, 4, 265–273. doi: 10.1080/713756764
   Web of Science ®Google Scholar
• Kokinous, J., Kotz, S. A., Tavano, A., & Schroeger, E. (2015). The role of emotion in dynamic audiovisual integration of faces and voices. Social Cognitive and Affective Neuroscience, 10(5), 713–720. doi: 10.1093/scan/nsu105
   PubMed Web of Science ®Google Scholar
• Koppen, C., & Spence, C. (2007). Audiovisual asynchrony modulates the Colavita visual dominance effect. Brain Research, 1186, 224–232. doi: 10.1016/j.brainres.2007.09.076
   PubMed Web of Science ®Google Scholar
• von Kriegstein, K., Dogan, O., Gruter, M., Giraud, A. L., Kell, C. A., Gruter, T., … Kiebel, S. J. (2008). Simulation of talking faces in the human brain improves auditory speech recognition. Proceedings of the National Academy of Sciences of the United States of America, 105, 6747–6752. doi: 10.1073/pnas.0710826105
   PubMed Web of Science ®Google Scholar
• von Kriegstein, K., Kleinschmidt, A., Sterzer, P., & Giraud, A. L. (2005). Interaction of face and voice areas during speaker recognition. Journal of Cognitive Neuroscience, 17, 367–376. doi: 10.1162/0898929053279577
   PubMed Web of Science ®Google Scholar
• Lachs, L., & Pisoni, D. B. (2004a). Specification of cross-modal source information in isolated kinematic displays of speech. Journal of the Acoustical Society of America, 116, 507–518. doi: 10.1121/1.1757454
   PubMed Web of Science ®Google Scholar
• Lachs, L., & Pisoni, D. B. (2004b). Crossmodal source identification in speech perception. Ecological Psychology, 16, 159–187. doi: 10.1207/s15326969eco1603_1
   PubMed Web of Science ®Google Scholar
• Lander, K., & Bruce, V. (2000). Recognizing famous faces: Exploring the benefits of facial motion. Ecological Psychology, 12, 259–272. doi: 10.1207/S15326969ECO1204_01
   Web of Science ®Google Scholar
• Lander, K., & Bruce, V. (2003). The role of motion in learning new faces. Visual Cognition, 10, 897–912. doi: 10.1080/13506280344000149
   Web of Science ®Google Scholar
• Lander, K., & Chuang, L. (2005). Why. are moving faces easier to recognize? Visual Cognition, 12, 429–442.
   Web of Science ®Google Scholar
• Liegeois-Chauvel, C., de Graaf, J. B., Laguitton, V., & Chauvel, P. (1999). Specialization of left auditory cortex for speech perception in man depends on temporal coding. Cerebral Cortex, 9, 484–496. doi: 10.1093/cercor/9.5.484
   PubMed Web of Science ®Google Scholar
• Maravita, A., Bolognini, N., Bricolo, E., Marzi, C. A., & Savazzi, S. (2008). Is audiovisual integration subserved by the superior colliculus in humans? NeuroReport, 19, 271–275. doi: 10.1097/WNR.0b013e3282f4f04e
   PubMed Web of Science ®Google Scholar
• Mathias, S. R., & von Kriegstein, K. (2014). How do we recognize who is speaking? Frontiers in Bioscience, S6, 92–109. doi: 10.2741/S417
   Google Scholar
• McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748. doi: 10.1038/264746a0
   PubMed Web of Science ®Google Scholar
• Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996). Temporal constraints on the McGurk effect. Perception & Psychophysics, 58, 351–362. doi: 10.3758/BF03206811
   PubMed Web of Science ®Google Scholar
• Navarra, J., Vatakis, A., Zampini, M., Soto-Faraco, S., Humphreys, W., & Spence, C. (2005). Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Cognitive Brain Research, 25, 499–507. doi: 10.1016/j.cogbrainres.2005.07.009
   PubMedGoogle Scholar
• Neuner, F., & Schweinberger, S. R. (2000). Neuropsychological impairments in the recognition of faces, voices, and personal names. Brain and Cognition, 44, 342–366. doi: 10.1006/brcg.1999.1196
   PubMed Web of Science ®Google Scholar
• O'Mahony, C., & Newell, F. N. (2012). Integration of faces and voices, but not faces and names, in person recognition. British Journal of Psychology, 103, 73–82. doi: 10.1111/j.2044-8295.2011.02044.x
   PubMed Web of Science ®Google Scholar
• O'Toole, A. J., Roark, D. A., & Abdi, H. (2002). Recognizing moving faces: A psychological and neural synthesis. Trends in Cognitive Sciences, 6, 261–266. doi: 10.1016/S1364-6613(02)01908-3
   PubMed Web of Science ®Google Scholar
• Perrodin, C., Kayser, C., Abel, T. J., Logothetis, N. K., & Petkov, C. I. (2015). Who is that? Brain networks and mechanisms for identifying individuals. Trends in Cognitive Sciences, 19(12), 783–796. doi: 10.1016/j.tics.2015.09.002
   PubMed Web of Science ®Google Scholar
• Powers, A. R., Hillock, A. R., & Wallace, M. T. (2009). Perceptual training narrows the temporal window of multisensory binding. Journal of Neuroscience, 29, 12265–12274. doi: 10.1523/JNEUROSCI.3501-09.2009
   PubMed Web of Science ®Google Scholar
• Robertson, D. M. C., & Schweinberger, S. R. (2010). The role of audiovisual asynchrony in person recognition. Quarterly Journal of Experimental Psychology, 63, 23–30. doi: 10.1080/17470210903144376
   Web of Science ®Google Scholar
• Rosenblum, L. D., Johnson, J. A., & Saldana, H. M. (1996). Point-light facial displays enhance comprehension of speech in noise. Journal of Speech and Hearing Research, 39, 1159–1170. doi: 10.1044/jshr.3906.1159
   PubMedGoogle Scholar
• Rosenblum, L. D., Smith, N. M., Nichols, S. M., Hale, S., & Lee, J. (2006). Hearing a face: Cross-modal speaker matching using isolated visible speech. Perception & Psychophysics, 68, 84–93. doi: 10.3758/BF03193658
   PubMed Web of Science ®Google Scholar
• Sams, M., Aulanko, R., Hämalainen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., … Simola, J. (1991). Seeing speech: Visual information from lip movements modifies activity in the human auditory cortex. Neuroscience Letters, 127, 141–145. doi: 10.1016/0304-3940(91)90914-F
   PubMed Web of Science ®Google Scholar
• Schweinberger, S. R. (2001). Human brain potential correlates of voice priming and voice recognition. Neuropsychologia, 39, 921–936. doi: 10.1016/S0028-3932(01)00023-9
   PubMed Web of Science ®Google Scholar
• Schweinberger, S. R., Casper, C., Hauthal, N., Kaufmann, J. M., Kawahara, H., Kloth, N., … Zäske, R. (2008). Auditory adaptation in voice perception. Current Biology, 18, 684–688. doi: 10.1016/j.cub.2008.04.015
   PubMed Web of Science ®Google Scholar
• Schweinberger, S. R., Herholz, A., & Sommer, W. (1997). Recognizing famous voices: Influence of stimulus duration and different types of retrieval cues. Journal of Speech, Language, and Hearing Research, 40, 453–463. doi: 10.1044/jslhr.4002.453
   PubMed Web of Science ®Google Scholar
• Schweinberger, S. R., Herholz, A., & Stief, V. (1997). Auditory long-term memory: Repetition priming of voice recognition. Quarterly Journal of Experimental Psychology, 50A, 498–517. doi: 10.1080/027249897391991
   Web of Science ®Google Scholar
• Schweinberger, S. R., Kawahara, H., Simpson, A. P., Skuk, V. G., & Zäske, R. (2014). Speaker perception. Wiley Interdisciplinary Reviews: Cognitive Science, 5(1), 15–25. doi: 10.1002/wrna.1194
   PubMed Web of Science ®Google Scholar
• Schweinberger, S. R., Kloth, N., & Robertson, D. M. C. (2011). Hearing facial identities: Brain correlates of face-voice integration in person identification. Cortex, 47, 1026–1037. doi: 10.1016/j.cortex.2010.11.011
   PubMed Web of Science ®Google Scholar
• Schweinberger, S. R., Robertson, D., & Kaufmann, J. M. (2007). Hearing facial identities. Quarterly Journal of Experimental Psychology, 60, 1446–1456. doi: 10.1080/17470210601063589
   Web of Science ®Google Scholar
• Shah, N. J., Marshall, J. C., Zafiris, O., Schwab, A., Zilles, K., Markowitsch, H. J., et al. (2001). The neural correlates of person familiarity. A functional magnetic resonance imaging study with clinical implications. Brain, 124, 804–815. doi: 10.1093/brain/124.4.804
   PubMed Web of Science ®Google Scholar
• Shams, L., Kamitani, Y., & Shimojo, S. (2002). Visual illusion induced by sound. Cognitive Brain Research, 14, 147–152. doi: 10.1016/S0926-6410(02)00069-1
   PubMedGoogle Scholar
• Shams, L., Kamitani, Y., Thompson, S., & Shimojo, S. (2001). Sound alters visual evoked potentials in humans. NeuroReport, 12, 3849–3852. doi: 10.1097/00001756-200112040-00049
   PubMed Web of Science ®Google Scholar
• Sheffert, S. M., & Olson, E. (2004). Audiovisual speech facilitates voice learning. Perception & Psychophysics, 66, 352–362. doi: 10.3758/BF03194884
   PubMed Web of Science ®Google Scholar
• Smith, H. M. J., Dunn, A. K., Baguley, T., & Stacey, P. C. (2016). Matching novel face and voice identity using static and dynamic facial images. Attention, Perception & Psychophysics, 78(3), 868–879. doi: 10.3758/s13414-015-1045-8
   PubMed Web of Science ®Google Scholar
• Stevenson, R. A., & Wallace, M. T. (2013). Multisensory temporal integration: Task and stimulus dependencies. Experimental Brain Research, 227, 249–261. doi: 10.1007/s00221-013-3507-3
   PubMed Web of Science ®Google Scholar
• Terry, L. H., Livescu, K., Pierrehumbert, J. B., & Katsaggelos, A. K. (2010, September 26–30). Audio-Visual anticipatory coarticulation modeling by human and machine. Proceedings of 11th Annual Conference of the International Speech Communication Association 2010 (INTERSPEECH 2010), Makuhari, Chiba, Japan. pp. 2682–2685.
   Google Scholar
• VanLancker, D., & Kreiman, J. (1987). Voice discrimination and recognition are separate abilities. Neuropsychologia, 25, 829–834. doi: 10.1016/0028-3932(87)90120-5
   PubMed Web of Science ®Google Scholar
• VanLancker, D., Kreiman, J., & Emmorey, K. (1985). Familiar voice recognition: Patterns and parameters. Part I: Recognition of backward voices. Journal of Phonetics, 13, 19–38.
   Web of Science ®Google Scholar
• Walker, S., Bruce, V., & O'Malley, C. (1995). Facial identity and facial speech processing: Familiar faces and voices in the McGurk effect. Perception & Psychophysics, 57, 1124–1133. doi: 10.3758/BF03208369
   PubMed Web of Science ®Google Scholar
• Warren, D. H., Welch, R. B., & Mccarthy, T. J. (1981). The Role of Visual-Auditory Compellingness in the Ventriloquism Effect - Implications for Transitivity Among the Spatial Senses. Perception & Psychophysics, 30, 557–564. doi: 10.3758/BF03202010
   PubMed Web of Science ®Google Scholar
• van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, 102, 1181–1186. doi: 10.1073/pnas.0408949102
   PubMed Web of Science ®Google Scholar
• van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45, 598–607. doi: 10.1016/j.neuropsychologia.2006.01.001
   PubMed Web of Science ®Google Scholar
• Welch, R. B., & Warren, D. H. (1980). Immediate perceptual response to intersensory discrepancy. Psychological Bulletin, 88, 638–667. doi: 10.1037/0033-2909.88.3.638
   PubMed Web of Science ®Google Scholar
• Yip, A. W., & Sinha, P. (2002). Contribution of color to face recognition. Perception, 31, 995–1003. doi: 10.1068/p3376
   PubMed Web of Science ®Google Scholar
• Yovel, G., & Belin, P. (2013). A unified coding strategy for processing faces and voices. Trends in Cognitive Sciences, 17(6), 263–271. doi: 10.1016/j.tics.2013.04.004
   PubMed Web of Science ®Google Scholar
• Yovel, G., & O'Toole, A. J. (2016). Recognizing People in Motion. Trends in Cognitive Sciences, 20(5), 383–395. doi: 10.1016/j.tics.2016.02.005
   PubMed Web of Science ®Google Scholar
• Zäske, R., Schweinberger, S. R., & Kawahara, H. (2010). Voice aftereffects of adaptation to speaker identity. Hearing Research, 268, 38–45. doi: 10.1016/j.heares.2010.04.011
   PubMed Web of Science ®Google Scholar
• Zekveld, A. A., Kramer, S. E., Vlaming, M. S. M. G., & Houtgast, T. (2008). Audiovisual perception of speech in noise and masked written text. Ear and Hearing, 29, 99–111. doi: 10.1097/AUD.0b013e31818005bd
   PubMed Web of Science ®Google Scholar