Gestures and Phases: The Dynamics of Speech-Hand Communication

REFERENCES

• Abry , C. , Lallouache , M.-T. and Cathiard , M.-A. 1996 . “ How can coarticulation models account for speech sensitivity to audio-visual desynchronization? ” . In Speechreading by humans and machines , Edited by: Stork , D. and Hennecke , M. Vol. 150 , 247 – 256 . Berlin : Springer-Verlag. .
   Google Scholar
• Alibali , M. W. and DiRusso , A. A. 1999 . The function of gesture in learning to count: More than keeping track . Cognitive Development , 14 : 37 – 56 .
   Web of Science ®Google Scholar
• Arbib , M. A. 2005 . From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics . Behavioral and Brain Sciences , 28 : 105 – 124 .
   PubMed Web of Science ®Google Scholar
• Blake , J. , Olshansky , E. , Vitale , G. and Macdonald , S. 1997 . Are communicative gestures the substrate of language? . Evolution of Communication , 1 : 261 – 282 .
   Google Scholar
• Borghi , A. M. 2004 . Object concepts and action: Extracting affordances from objects parts . Acta Psychologica , 115 : 69 – 96 .
   PubMed Web of Science ®Google Scholar
• Browman , C. P. and Goldstein , L. 1992 . Articulatory phonology: An overview . Phonetica , 49 : 155 – 180 .
   PubMed Web of Science ®Google Scholar
• Buccino , G. , Binkfski , F. and Riggio , L. 2004 . The mirror neuron system and action recognition . Brain and Language , 89 : 370 – 376 .
   PubMed Web of Science ®Google Scholar
• Corballis , M. 2002 . From hand to mouth: The origins of language , Princeton, NJ : Princeton University Press .
   Google Scholar
• Corballis , M. 2003 . Laterality and human speciation Crow, T. J. . Speciation of modern Homo sapiens. Proceedings of the British Academy , 106 : 137 – 152 .
   Google Scholar
• Fadiga , L. , Craighero , L. , Buccino , G. and Rizzolatti , G. 2002 . Speech listening specifically modulates the excitability of tongue muscles: a TMS study . European Journal of Neuroscience , 15 : 399 – 402 .
   PubMed Web of Science ®Google Scholar
• Ferrari , P. F. , Gallese , V. , Rizzolatti , G. and Fogassi , L. 2003 . Mirror neurons responding to the observation of ingestive and communicative mouth actions . European Journal of Neuroscience , 17 : 1703 – 1714 .
   PubMed Web of Science ®Google Scholar
• Floel , A. , Ellger , T. , Breitenstein , C. and Knecht , S. 2003 . Language perception activates the hand motor cortex: Implications for motor theories of speech perception . European Journal of Neuroscience , 18 : 704 – 708 .
   PubMed Web of Science ®Google Scholar
• Fowler , C. A. 1986 . An event approach to the study of speech perception from a direct-realist perspective . Journal of Phonetics , 14 : 3 – 28 .
   Web of Science ®Google Scholar
• Furuyama , N. 2002 . Prolegomena of a theory of between-person coordination of speech and gesture . International Journal of Human-Computer Studies , 57 : 347 – 374 .
   Web of Science ®Google Scholar
• Gibson , J. J. 1975 . “ Events are perceivable but time is not ” . In The study of time II , Edited by: Fraser , J. T. and Lawrence , N. 295 – 301 . Berlin : Springer-Verlag. .
   Google Scholar
• Gibson , J. J. 1986 . The ecological approach to visual perception , Hillsdale, NJ : Lawrence Erlbaum Associates, Inc. . Original work published 1979
   Google Scholar
• Goldin-Meadow , S. 1999 . The role of gesture in communication and thinking . Trends in Cognitive Sciences , 3 : 419 – 429 .
   PubMed Web of Science ®Google Scholar
• Gullberg , M. and Holmquist , K. 1999 . Keeping an eye on gestures: Visual perception of gestures in face-to-face communication . Pragmatics and Cognition , 7 : 35 – 63 .
   Google Scholar
• Gullberg , M. and Holmquist , K. 2002 . “ Visual attention towards gestures in face-to-face interaction vs. on screen ” . In Gesture and sign language based human-computer interaction , Edited by: Wachsmuth , I. and Sowa , T. 206 – 214 . Berlin : Springer. .
   Google Scholar
• Haueisen , J. and Knosche , T. R. 2001 . Involuntary motor activity in pianists evoked by music perception . Journal of Cognitive Neuroscience , 13 : 786 – 792 .
   PubMed Web of Science ®Google Scholar
• Hodges , B. H. 2007 . Values define fields: The intentional dynamics of driving, carrying, leading, negotiating, and conversing . Ecological Psychology , 19 : 153 – 178 .
   Web of Science ®Google Scholar
• Iverson , J. M. and Goldin-Meadow , S. 1997 . What's communication got to do with it? Gesture in congenitally blind children . Developmental Psychology , 33 : 453 – 467 .
   PubMed Web of Science ®Google Scholar
• Kelly , S. D. , Kravitz , C. and Hopkins , M. 2004 . Neural correlates of bimodal speech and gesture comprehension . Brain and Language , 89 : 253 – 260 .
   PubMed Web of Science ®Google Scholar
• Kelso , J. A. S. 1995 . Dynamic patterns: The self-organization of brain and behaviour , Boston : MIT Press .
   Google Scholar
• Kendon , A. 1972 . “ Some relationships between body motion and speech ” . In Studies in dyadic communication , Edited by: Siegman , A. and Pope , B. New York : Pergamon Press. .
   Google Scholar
• Kendon , A. 1974 . “ Movement coordination in social interaction: Some examples described ” . In Nonverbal communication , Edited by: Weitz , S. New York : Oxford University Press. .
   Google Scholar
• Kendon , A. 2004 . Review of Susan Goldin-Meadow (2003), Hearing gesture: How our hands help us think . Gesture , 4 : 91 – 107 .
   Google Scholar
• Kimura , D. 1973 . Manual activity during speaking–Right, I. handers . Neuropsychologia , 11 : 45 – 50 .
   PubMed Web of Science ®Google Scholar
• Kohler , E. , Keysers , C. , Umilta , M. A. , Fogassi , L. , Gallese , V. and Rizzolatti , G. 2002 . Hearing sounds, understanding actions: Action representation in mirror neurons . Science , 297 : 846 – 848 .
   PubMed Web of Science ®Google Scholar
• Krauss , R. M. 1998 . Why do we gesture when we speak? . Current Directions in Psychological Science , 7 : 54 – 59 .
   Web of Science ®Google Scholar
• Lausberg , H. and Kita , S. 2003 . The content of the message influences the hand choice in co-speech gestures and in gesturing without speaking . Brain and Language , 86 : 57 – 69 .
   PubMed Web of Science ®Google Scholar
• Marsh , K. L. , Richardson , M. J. , Barron , R. M. and Schmidt , R. C. 2006 . Contrasting approaches to perceiving and acting with others . Ecological Psychology , 18 : 1 – 38 .
   Web of Science ®Google Scholar
• Mayberry , R. and Jaques , J. 2000 . “ Gesture production during stuttered speech: Insights into the nature of gesture-speech integration ” . In Gesture and language , Edited by: McNeill , D. 199 – 213 . Cambridge, , UK : Cambridge University Press .
   Google Scholar
• Mayberry , R. I. and Nicholadis , E. 2000 . Gesture reflects language development: Evidence from bilingual children . Current Directions in Psychological Science , 9 : 192 – 196 .
   Web of Science ®Google Scholar
• Mayberry , R. I. and Shenker , R. C. 1997 . “ Gesture mirrors speech motor control in stutterers ” . In Speech motor production and fluency disorders , Edited by: Hulstijn , W. , Peters , H. and van Lieshout , P. 183 – 190 . Amsterdam : Elsevier Science. .
   Google Scholar
• McClave , E. 1994 . Gestural beats: The rhythm hypothesis . Journal of Psycholinguistic Research , 23 : 45 – 66 .
   Web of Science ®Google Scholar
• McClave , E. 1997 . Pitch and manual gestures . Journal of Psycholinguistic Research , 27 : 69 – 89 .
   Google Scholar
• McNeill , D. 1985 . So you think gestures are nonverbal? . Psychological Review , 92 : 350 – 371 .
   Web of Science ®Google Scholar
• McNeill , D. 1992 . Hand and mind: What gestures reveal about thought , Chicago : University of Chicago Press .
   Google Scholar
• McNeill , D. , ed. 2000 . Language and gesture , Cambridge, , UK : Cambridge University Press .
   Google Scholar
• McNeill , D. 2005 . Gesture-first, but no gestures? . Behavioral and Brain Sciences , 28 : 138 – 139 .
   Google Scholar
• Meister , I. G. , Boroojerdi , B. , Foltys , H. , Sparing , R. , Huber , W. and Töpper , R. 2003 . Motor cortext hand area and speech: Implications for the development of language . Neuropsychologia , 41 : 401 – 406 .
   PubMed Web of Science ®Google Scholar
• Millikan , R. G. 1984 . Language, thought and other biological categories , Cambridge, MA : MIT Press .
   Google Scholar
• Morrel-Samuels , P. and Krauss , R. M. 1992 . Word familiarity predicts the temporal asynchrony of hand gestures and speech . Journal of Experimental Psychology: Learning, Memory and Cognition , 18 : 615 – 623 .
   Web of Science ®Google Scholar
• Munhall , K. G. , Gribble , P. , Sacco , L. and Ward , M. 1996 . Temporal constraints on the McGurk effect . Perception & Psychophysics , 58 : 351 – 362 .
   PubMed Web of Science ®Google Scholar
• Munhall , K. G. , Jones , J. A. , Callan , D. E. , Kuratate , T. and Vatikiotis-Bateson , E. 2004 . Visual prosody and speech intelligibility: Head movement improves auditory speech perception . Psychological Science , 15 : 133 – 137 .
   PubMed Web of Science ®Google Scholar
• Munhall , K. G. and Tohkura , Y. 1998 . Audiovisual gating and the time course of speech perception . Journal of the Acoustical Society of America , 104 : 530 – 539 .
   PubMed Web of Science ®Google Scholar
• Nobe , S. 2000 . “ Where do most spontaneous representational gestures actually occur with respect to speech? ” . In Language and gesture , Edited by: McNeill , D. 186 – 198 . Cambridge, , UK : Cambridge University Press .
   Google Scholar
• Pagano , C. C. and Turvey , M. T. 1995 . The inertia tensor as a basis for the perception of limb orientation . Journal of Experimental Psychology: Human Perception and Performance , 21 : 1070 – 1087 .
   PubMed Web of Science ®Google Scholar
• Pettito , L. A. , Holowka , S. , Sergio , L. E. , Levy , B. and Ostry , D. J. 2004 . Baby hands that move to the rhythm of language: Hearing babies acquiring sign languages babble silently on the hands . Cognition , 93 : 43 – 73 .
   Google Scholar
• Reed , E. S. 1996 . Encountering the world: Towards an ecological psychology , New York : Oxford University Press .
   Google Scholar
• Rogers , W. 1978 . The contribution of kinesic illustrators towards the comprehension of verbal behaviour within utterances . Human Communication Research , 5 : 54 – 62 .
   Google Scholar
• Saltzman , E. L. and Byrd , D. 2000 . Task-dynamics of gestural timing: Phase windows and multifrequency rhythms . Human Movement Science , 19 : 499 – 526 .
   Web of Science ®Google Scholar
• Santi , A. , Servos , P. , Vatikiotis-Bateson , E. , Kuratate , T. and Munhall , K. 2003 . Perceiving biological motion: Dissociating visible speech from walking . Journal of Cognitive Neuroscience , 15 : 800 – 809 .
   Google Scholar
• Schmidt , R. C. 2007 . Scaffolds for social meaning . Ecological Psychology , 19 : 137 – 151 .
   Web of Science ®Google Scholar
• Schmidt , R. C. and Turvey , M. T. 1994 . Phase-entrainment dynamics of visually coupled rhythmic movements . Biological Cybernetics , 70 : 369 – 376 .
   PubMed Web of Science ®Google Scholar
• Shaw , R. E. 2001 . Processes, acts, and experiences: Three stances on the problem of intentionality . Ecological Psychology , 13 : 275 – 314 .
   Web of Science ®Google Scholar
• Shaw , R. E. 2003 . The agent-environment interface: Simon's indirect or Gibson's direct coupling? . Ecological Psychology , 15 : 37 – 106 .
   Web of Science ®Google Scholar
• Shaw , R. E. , Turvey , M. T. and Mace , W. M. 1981 . “ Ecological psychology: The consequences of a commitment to realism ” . In Cognition and the symbolic processes: Vol. II , Edited by: Weimer , W. and Palermo , D. 159 – 226 . Hillsdale, NJ : Lawrence Erlbaum Associates, Inc. .
   Google Scholar
• Singer , M. A. and Goldin-Meadow , S. 2005 . Children learn when their teacher's gestures and speech differ . Psychological Science , 16 : 85 – 89 .
   PubMed Web of Science ®Google Scholar
• Sousa-Poza , J. F. , Rohrberg , R. and Mercure , A. 1979 . Effects of type of information (abstract-concrete) and field dependence on asymmetry of hand movements during speech . Perceptual and Motor Skills , 48 : 1323 – 1330 .
   PubMed Web of Science ®Google Scholar
• Stanney , K. 2002 . Handbook of virtual environments , Hillsdale, NJ : Lawrence Erlbaum Associates, Inc. .
   Google Scholar
• Taira , M. , Mine , S. , Georgopoulos , A. P. , Murata , A. and Sakata , H. 1990 . Parietal cortex neurons of the monkey related to the visual guidance of hand movement . Experimental Brain Research , 83 : 29 – 36 .
   PubMed Web of Science ®Google Scholar
• Treffner , P. J. 1999a . “ Resonance constraints on between-person polyrhythms ” . In Studies in perception and action V , Edited by: Grealy , M. A. and Thomson , J. A. 165 – 169 . Mahwah, NJ : Lawrence Erlbaum Associates, Inc. .
   Google Scholar
• Treffner , P. J. 1999b . The common structure of concepts is the affordance in the ecology. Commentary on Ruth Millikan's “A common structure of individuals, stuffs, and real kinds.” . Behavioral and Brain Sciences , 22 : 729 – 733 .
   Google Scholar
• Treffner , P. J. and Barrett , R. 2004 . Hands-free mobile phone speech while driving degrades coordination and control . Transportation Research, Part F: Traffic Psychology and Behaviour , 7 : 229 – 246 .
   Web of Science ®Google Scholar
• Treffner , P. J. , Barrett , R. and Petersen , A. J. 2002 . Stability and skill in driving . Human Movement Science , 21 : 749 – 784 .
   PubMed Web of Science ®Google Scholar
• Treffner , P. J. and Kelso , J. A. S. 1999 . Dynamic encounters: Long-memory during functional stabilization . Ecological Psychology , 11 : 103 – 137 .
   Web of Science ®Google Scholar
• Treffner , P. J. and Peter , M. 2002 . Intentional and attentional dynamics of speech-hand coordination . Human Movement Science , 21 : 641 – 697 .
   PubMed Web of Science ®Google Scholar
• Treffner , P. J. and Turvey , M. T. 1993 . Resonance constraints on rhythmic movement . Journal of Experimental Psychology: Human Perception and Performance , 19 : 1221 – 1237 .
   Web of Science ®Google Scholar
• Treffner , P. J. and Turvey , M. T. 1995 . Handedness and the asymmetric dynamics of bimanual rhythmic coordination . Journal of Experimental Psychology: Human Perception and Performance , 21 : 318 – 333 .
   Web of Science ®Google Scholar
• Treffner , P. J. and Turvey , M. T. 1996 . Symmetry, broken symmetry, and the dynamics of bimanual coordination . Experimental Brain Research , 107 : 463 – 478 .
   PubMed Web of Science ®Google Scholar
• Turvey , M. T. , Shaw , R. E. , Reed , E. and Mace , W. 1981 . Ecological laws of perceiving acting: In reply to Fodor and Pylyshyn (1981) . Cognition , 9 : 237 – 304 .
   PubMed Web of Science ®Google Scholar
• van Lieshout , P. H. H. M. 2004 . “ Dynamical systems theory and its application in speech ” . In Speech motor control in normal and disordered speech , Edited by: Maassen , B. , Kent , R. , Peters , H. , van Lieshout , P. and Hulstijn , W. 313 – 356 . Oxford, , UK : Oxford University Press .
   Google Scholar
• van Lieshout , P. H. H. M. , Hulstijn , W. and Peters , F. M. 1996 . From planning to articulation in speech production: What differentiates a person who stutters from a person who does not stutter? . Journal of Speech and Hearing Research , 39 : 546 – 564 .
   PubMedGoogle Scholar
• Watkins , K. E. , Strafella , A. P. and Paus , T. 2003 . Seeing and hearing speech excites the motor system involved in speech production . Neuropsychologia , 41 : 989 – 994 .
   PubMed Web of Science ®Google Scholar
• Wilson , A. D. , Bingham , G. P. and Craig , J. C. 2003 . Proprioceptive perception of phase variability . Journal of Experimental Psychology: Human Perception and Performance , 29 : 1179 – 1190 .
   PubMed Web of Science ®Google Scholar
• Yehia , H. C. , Kuratate , T. and Vatikiotis-Bateson , E. 2002 . Linking facial animation, head motion, and speech acoustics . Journal of Phonetics , 30 : 555 – 568 .
   Web of Science ®Google Scholar
• Yehia , H. , Rubin , P. and Vatikiotis-Bateson , E. 1998 . Quantitative association of vocal-tract and facial behaviour . Speech Communication , 26 : 23 – 43 .
   Web of Science ®Google Scholar