Look who's talking? Sound changes gaze behaviour in a dynamic social scene

REFERENCES

• Birmingham, E., Bischof, W. F., & Kingstone, A. (2008). Gaze selection in complex social scenes. Visual Cognition, 16(2–3), 341–355. doi:10.1080/13506280701434532
   Web of Science ®Google Scholar
• Buchan, J. N., Pare, M., & Munhall, K. G. (2007). Spatial statistics of gaze fixations during dynamic face processing. Social Neuroscience, 2(1), 1–13. doi:10.1080/17470910601043644
   PubMed Web of Science ®Google Scholar
• Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. Journal of the Acoustical Society of America, 25, 975–979. doi:10.1121/1.1907229
   Web of Science ®Google Scholar
• Coutrot, A., Guyader, N., Ionescu, G., & Caplier, A. (2012). Influence of soundtrack on eye movements during video exploration. Journal of Eye Movement Research, 5(4), 1–10.
   Web of Science ®Google Scholar
• Emery, N. J. (2000). The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24(6), 581–604. doi:10.1016/S0149-7634(00)00025-7
   PubMed Web of Science ®Google Scholar
• Emmorey, K., Thompson, R., & Colvin, R. (2009). Eye gaze during comprehension of American sign language by native and beginning signers. Journal of Deaf Studies and Deaf Education, 14(2), 237–243. doi:10.1093/deafed/enn037
   PubMed Web of Science ®Google Scholar
• Foulsham, T., Cheng, J. T., Tracy, J. L., Henrich, J., & Kingstone, A. (2010). Gaze allocation in a dynamic situation: Effects of social status and speaking. Cognition, 117(3), 319–331. doi:10.1016/j.cognition.2010.09.003
   PubMed Web of Science ®Google Scholar
• Freeth, M., Foulsham, T., & Kingstone, A. (2013). What affects social attention? Social presence, eye contact and autistic traits. PLoS ONE, 8(1), e53286. doi:10.1371/journal.pone.0053286
   PubMed Web of Science ®Google Scholar
• Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin and Review, 5(3), 490–495. doi:10.3758/BF03208827
   Web of Science ®Google Scholar
• Gallup, A. C., Chong, A., & Couzin, I. D. (2012). The directional flow of visual information transfer between pedestrians. Biology Letters, 8(4), 520–522. doi:10.1098/rsbl.2012.0160
   PubMed Web of Science ®Google Scholar
• Ghazanfar, A. A., Nielsen, K., & Logothetis, N. K. (2006). Eye movements of monkey observers viewing vocalizing conspecifics. Cognition, 101(3), 515–529. doi:10.1016/j.cognition.2005.12.007
   PubMed Web of Science ®Google Scholar
• Goldstein, R. B., Woods, R. L., & Peli, E. (2007). Where people look when watching movies: Do all viewers look at the same place? Computers in Biology and Medicine, 37(7), 957–964. doi:10.1016/j.compbiomed.2006.08.018
   PubMed Web of Science ®Google Scholar
• Gullberg, M. (2002). Eye movements and gestures in human interaction. In J. Hyona, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 685–703). Amsterdam: Elsevier.
   Google Scholar
• Hanna, J. E., & Brennan, S. E. (2007). Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57, 596–615. doi:10.1016/j.jml.2007.01.008
   Web of Science ®Google Scholar
• Itier, R. J., & Batty, M. (2009). Neural bases of eye and gaze processing: The core of social cognition. Neuroscience and Biobehavioral Reviews, 33(6), 843–863. doi:10.1016/j.neubiorev.2009.02.004
   PubMed Web of Science ®Google Scholar
• Itti, L. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition, 12(6), 1093–1123. doi:10.1080/13506280444000661
   Web of Science ®Google Scholar
• Kendon, A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica, 26(1), 22–63. doi:10.1016/0001-6918(67)90005-4
   PubMed Web of Science ®Google Scholar
• Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002). Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Archives of General Psychiatry, 59(9), 809–816. doi:10.1001/archpsyc.59.9.809
   PubMed Web of Science ®Google Scholar
• Laidlaw, K. E., Foulsham, T., Kuhn, G., & Kingstone, A. (2011). Potential social interactions are important to social attention. Proceedings of the National Academy of Sciences, 108(14), 5548–5553. doi:10.1073/pnas.1017022108
   PubMed Web of Science ®Google Scholar
• Lansing, C. R., & McConkie, G. W. (1999). Attention to facial regions in segmental and prosodic visual speech perception tasks. Journal of Speech, Language and Hearing Research, 42(3), 526–539.
   PubMed Web of Science ®Google Scholar
• Lansing, C. R., & McConkie, G. W. (2003). Word identification and eye fixation locations in visual and visual-plus-auditory presentations of spoken sentences. Perception and Psychophysics, 65(4), 536–552. doi:10.3758/BF03194581
   PubMed Web of Science ®Google Scholar
• Levy, J., Foulsham, T., & Kingstone, A. (2013). Monsters are people too. Biology Letters, 9, 20120850.
   PubMed Web of Science ®Google Scholar
• Macdonald, R. G., & Tatler, B. W. (2013). Do as eye say: Gaze cueing and language in a real-world social interaction. Journal of Vision, 13(4). doi:10.1167/13.4.6
   PubMed Web of Science ®Google Scholar
• McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748. doi:10.1038/264746a0
   PubMed Web of Science ®Google Scholar
• Mital, P. K., Smith, T. J., Hill, R. L., & Henderson, J. M. (2011). Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation, 3(1), 5–24. doi:10.1007/s12559-010-9074-z
   Web of Science ®Google Scholar
• Nummenmaa, L., Hyona, J., & Hietanen, J. K. (2009). I'll walk this way: Eyes reveal the direction of locomotion and make passersby look and go the other way. Psychological Science, 20(12), 1454–1458. doi:10.1111/j.1467-9280.2009.02464.x
   PubMed Web of Science ®Google Scholar
• Pelphrey, K. A., Morris, J. P., & McCarthy, G. (2005). Neural basis of eye gaze processing deficits in autism. Brain, 128(5), 1038–1048. doi:10.1093/brain/awh404
   PubMed Web of Science ®Google Scholar
• Ricciardelli, P., Carcagno, S., Vallar, G., & Bricolo, E. (2013). Is gaze following purely reflexive or goal-directed instead? Revisiting the automaticity of orienting attention by gaze cues. Experimental Brain Research, 224(1), 93–106. doi:10.1007/s00221-012-3291-5
   PubMed Web of Science ®Google Scholar
• Risko, E. F., Laidlaw, K., Freeth, M., Foulsham, T., & Kingstone, A. (2012). Social attention with real versus reel stimuli: Toward an empirical approach to concerns about ecological validity. Frontiers in Human Neuroscience, 6. doi:10.3389/fnhum.2012.00143
   PubMed Web of Science ®Google Scholar
• Ross, N. M., & Kowler, E. (2013). Eye movements while viewing narrated, captioned, and silent videos. Journal of Vision, 13(4). doi:10.1167/13.4.1
   PubMed Web of Science ®Google Scholar
• Shen, J., & Itti, L. (2012). Top-down influences on visual attention during listening are modulated by observer sex. Vision Research, 65, 62–76. doi:10.1016/j.visres.2012.06.001
   PubMed Web of Science ®Google Scholar
• Shepherd, S. V., Steckenfinger, S. A., Hasson, U., & Ghazanfar, A. A. (2010). Human-monkey gaze correlations reveal convergent and divergent patterns of movie viewing. Current Biology, 20(7), 649–656. doi:10.1016/j.cub.2010.02.032
   PubMed Web of Science ®Google Scholar
• Smith, T. J., & Henderson, J. M. (2008). Edit blindness: The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research, 2, 1–17.
   Web of Science ®Google Scholar
• Spezio, M. L., Huang, P. Y. S., Castelli, F., & Adolphs, R. (2007). Amygdala damage impairs eye contact during conversations with real people. Journal of Neuroscience, 27(15), 3994–3997. doi:10.1523/JNEUROSCI.3789-06.2007
   PubMed Web of Science ®Google Scholar
• Spivey, M. J., & Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65(4), 235–241. doi:10.1007/s004260100059
   PubMed Web of Science ®Google Scholar
• Steinman, R. M. (1965). Effect of target size, luminance, and color on monocular fixation. Journal of the Optical Society of America, 55, 1158–1165. doi:10.1364/JOSA.55.001158
   Web of Science ®Google Scholar
• Stivers, T., Enfield, N. J., Brown, P., Englert, C., Hayashi, M., Heinemann, T., & Levinson, S. C. (2009). Universals and cultural variation in turn-taking in conversation. Proceedings of the National Academy of Sciences, 106(26), 10587–10592. doi:10.1073/pnas.0903616106
   PubMed Web of Science ®Google Scholar
• Tice, M., & Henetz, T. (2011). The eye gaze of 3rd party observers reflects turn-end boundary projection. In Proceedings of the 15th workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL 2011) (pp. 204–205). Los Angeles, CA.
   Google Scholar
• Van der Burg, E., Olivers, C. N. L., Bronkhorst, A. W., & Theeuwes, J. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34(5), 1053–1065. doi:10.1037/0096-1523.34.5.1053
   PubMed Web of Science ®Google Scholar
• Vatikiotis-Bateson, E., Eigsti, I. M., Yano, S., & Munhall, K. G. (1998). Eye movement of perceivers during audiovisual speech perception. Perception and Psychophysics, 60(6), 926–940. doi:10.3758/BF03211929
   PubMed Web of Science ®Google Scholar
• Vertegaal, R., Slagter, R., van der Veer, G., & Nijholt, A. (2001). Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 301–308). ACM.
   Google Scholar
• Võ, M. L. H., Smith, T. J., Mital, P. K., & Henderson, J. M. (2012). Do the eyes really have it? Dynamic allocation of attention when viewing moving faces. Journal of Vision, 12(13).
   PubMed Web of Science ®Google Scholar
• Yarbus, A. L. (1967). Eye movements and vision. New York, NY: Plenum.
   Google Scholar