Emotional speech processing: Disentangling the effects of prosody and semantic cues

References

• Astesano , C. , Besson , M. and Alter , K. 2004 . Brain potentials during semantic and prosodic processing in French . Cognitive Brain Research , 18 : 172 – 184 .
   PubMedGoogle Scholar
• Attardo , S. , Eisterhold , J. , Hay , J. and Poggi , I. 2003 . Multimodal markers of irony and sarcasm . Humor—International Journal of Humor Research , 16 ( 2 ) : 243 – 260 .
   Web of Science ®Google Scholar
• Banse , R. and Scherer , K. R. 1996 . Acoustic profiles in vocal emotion expression . Journal of Personality and Social Psychology , 70 ( 3 ) : 614 – 636 .
   PubMed Web of Science ®Google Scholar
• Beaucousin , V. , Lacheret , A. , Turbelin , M.-R. , Morel , M. l. , Mazoyer , B. Tzourio-Mazoyer , N. 2007 . fMRI study of emotional speech comprehension . Cerebral Cortex , 17 2 , 339 352 .
   PubMed Web of Science ®Google Scholar
• Borod , J. C. , Cicero , B. A. , Obler , L. K. , Welkowitz , J. , Erhan , H. M. Santschi , C. 1998 . Right hemisphere emotional perception: Evidence across multiple channels . Neuropsychology , 12 ( 3 ) : 446 – 458 .
   PubMed Web of Science ®Google Scholar
• Borod , J. C. , Pick , L. H. , Hall , S. , Sliwinski , M. , Madigan , N. Obler , L. K. 2000 . Relationships among facial, prosodic, and lexical channels of emotional perceptual processing . Cognition and Emotion , 14 ( 2 ) : 193 – 211 .
   Web of Science ®Google Scholar
• Bower , G. H. 1981 . Mood and memory . American Psychologist , 36 ( 2 ) : 129 – 148 .
   PubMed Web of Science ®Google Scholar
• Bower , G. H. 1987 . Commentary on mood and memory . Behaviour Research and Therapy , 25 ( 6 ) : 443 – 455 .
   PubMed Web of Science ®Google Scholar
• Bowers , D. , Bauer , R. and Heilman , K. 1993 . The nonverbal affect lexicon: Theoretical perspectives from neuropsychological studies of affect perception . Neuropsychology , 7 : 433 – 444 .
   Google Scholar
• Bowers , D. , Coslett , H. B. , Bauer , R. M. , Speedie , L. J. and Heilman , K. M. 1987 . Comprehension of emotional prosody following unilateral hemispheric lesions: Processing defect versus distraction defect . Neuropsychologia , 25 ( 2 ) : 317 – 328 .
   PubMed Web of Science ®Google Scholar
• Breitenstein , C. , Daum , I. and Ackermann , H. 1998 . Emotional processing following cortical and subcortical brain damage: Contribution of the fronto-striatal circuitry . Behavioural Neurology , 11 : 29 – 42 .
   PubMed Web of Science ®Google Scholar
• Buchanan , T. W. , Lutz , K. , Mirzazade , S. , Specht , K. , Jon Shah , N. Zilles , K. 2000 . Recognition of emotional prosody and verbal components of spoken language: an fMRI study . Cognitive Brain Research , 9 : 227 – 238 .
   PubMedGoogle Scholar
• Caffi , C. and Janney , R. W. 1994 . Toward a pragmatics of emotive communication . Journal of Pragmatics , 22 : 325 – 373 .
   Web of Science ®Google Scholar
• Carroll , N. C. and Young , A. W. 2005 . Priming of emotion recognition . The Quarterly Journal of Experimental Psychology , 58A ( 7 ) : 1173 – 1197 .
   Web of Science ®Google Scholar
• Cheang , H. S. and Pell , M. D. 2008 . The sound of sarcasm . Speech Communication , 50 : 366 – 381 .
   Web of Science ®Google Scholar
• Dara , C. , Monetta , L. and Pell , M. D. 2008 . Vocal emotion processing in Parkinson's disease: reduced sensitivity to negative emotions . Brain Research , 1188 : 100 – 111 .
   PubMed Web of Science ®Google Scholar
• de Gelder , B. , Bocker , K. B. E. , Tuomainen , J. , Hensen , M. and Vroomen , J. 1999 . The combined perception of emotion from voice and face: Early interaction revealed by human brain responses . Neuroscience Letters , 260 : 133 – 136 .
   PubMed Web of Science ®Google Scholar
• de Gelder , B. and Vroomen , J. 2000 . The perception of emotions by ear and by eye . Cognition and Emotion , 14 ( 3 ) : 289 – 311 .
   Web of Science ®Google Scholar
• Dimberg , U. and Öhman , A. 1996 . Beyond the wrath: Psychophysiological responses to facial stimuli . Motivation and Emotion , 20 ( 2 ) : 149 – 182 .
   Web of Science ®Google Scholar
• Eckstein , K. and Friederici , A. D. 2006 . It's early: Event-related potential evidence for initial interaction of syntax and prosody in speech comprehension . Journal of Cognitive Neuroscience , 18 ( 10 ) : 1696 – 1711 .
   PubMed Web of Science ®Google Scholar
• Ekman , P. 1992 . An argument for basic emotions . Cognition and Emotion , 6 : 169 – 200 .
   Web of Science ®Google Scholar
• Fazio , R. H. 2001 . On the automatic activation of associated evaluations: An overview . Cognition and Emotion , 15 ( 2 ) : 115 – 141 .
   Web of Science ®Google Scholar
• Friederici , A. D. 2002 . Towards a neural basis of auditory sentence processing . Trends in Cognitive Sciences , 6 ( 2 ) : 78 – 84 .
   PubMed Web of Science ®Google Scholar
• Grandjean , D. , Banziger , T. and Scherer , K. R. 2006 . Intonation as an interface between language and affect . Progress in Brain Research , 156 : 235 – 268 .
   PubMed Web of Science ®Google Scholar
• Grimshaw , G. M. 1998 . Integration and interference in the cerebral hemispheres: Relations with hemispheric specialization . Brain and Cognition , 36 ( 2 ) : 108 – 127 .
   PubMed Web of Science ®Google Scholar
• Hagoort , P. , Hald , L. , Bastiaansen , M. and Petersson , K. M. 2004 . Integration of word meaning and world knowledge in language comprehension . Science , 304 : 438 – 441 .
   PubMed Web of Science ®Google Scholar
• Hansen , C. and Shantz , C. 1995 . Emotion-specific priming: Congruence effects on affect and recognition across negative emotions . Personality and Social Psychology Bulletin , 21 : 548 – 557 .
   Web of Science ®Google Scholar
• Hietanen , J. K. , Leppanen , J. M. , Illi , M. and Surakka , V. 2004 . Evidence for the integration of audiovisual emotional information at the perceptual level of processing . European Journal of Cognitive Psychology , 16 ( 6 ) : 769 – 790 .
   Web of Science ®Google Scholar
• Hsu , S.-M. , Hetrick , W. P. and Pessoa , L. 2008 . Depth of facial expression processing depends on stimulus visibility: Behavioral and electrophysiological evidence of priming effects. Cognitive, Affective . Behavioral Neuroscience , 8 ( 3 ) : 282 – 292 .
   Web of Science ®Google Scholar
• Innes-Ker , A. and Niedenthal , P. 2002 . Emotion concepts and emotional states in social judgment and categorization . Journal of Personality and Social Psychology , 83 : 804 – 816 .
   PubMed Web of Science ®Google Scholar
• Ishii , K. , Reyes , J. A. and Kitayama , S. 2003 . Spontaneous attention to word content versus emotional tone: Differences among three cultures . Psychological Science , 14 ( 1 ) : 39 – 46 .
   PubMed Web of Science ®Google Scholar
• Johnson , W. F. , Emde , R. N. , Scherer , K. R. and Klinnert , M. D. 1986 . Recognition of emotion from vocal cues . Archives of General Psychiatry , 43 : 280 – 283 .
   PubMed Web of Science ®Google Scholar
• Johnstone , T. , van Reekum , C. , Oakes , T. and Davidson , R. J. 2006 . The voice of emotion: An fMRI study of neural responses to angry and happy vocal expressions. Social, Cognitive . and Affective Neuroscience , 1 : 242 – 249 .
   PubMed Web of Science ®Google Scholar
• Kitayama , S. and Howard , S. 1994 . “ Affective regulation of perception and comprehension: Amplification and semantic priming ” . In The heart's eye: Emotional influences in perception and attention , Edited by: Niedenthal , P. M. and Kitayama , S. 41 – 65 . San Diego, CA : Academic Press .
   Google Scholar
• Kitayama , S. and Ishii , K. 2002 . Word and voice: Spontaneous attention to emotional utterances in two languages . Cognition and Emotion , 16 ( 1 ) : 29 – 59 .
   Web of Science ®Google Scholar
• Kotz , S. A. and Paulmann , S. 2007 . When emotional prosody and semantics dance cheek to cheek: ERP evidence . Brain Research , 115 : 107 – 118 .
   Web of Science ®Google Scholar
• Kreifelts , B. , Ethofer , T. , Grodd , W. , Erb , M. and Wildgruber , D. 2007 . Audiovisual integration of emotional signals in voice and face: An event-related fMRI study . NeuroImage , 37 : 1445 – 1456 .
   PubMed Web of Science ®Google Scholar
• Kreifelts , B. , Ethofer , T. , Shiozawa , T. , Grodd , W. and Wildgruber , D. 2009 . Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face-sensitive regions in the superior temporal sulcus . Neuropsychologia , 47 ( 14 ) : 3059 – 3066 .
   PubMed Web of Science ®Google Scholar
• Laukka , P. 2005 . Categorical perception of vocal emotion expressions . Emotion , 5 : 277 – 295 .
   PubMed Web of Science ®Google Scholar
• Leppänen , J. M. and Hietanen , J. K. 2004 . Positive facial expressions are recognized faster than negative facial expressions, but why? . Psychological Research , 69 : 22 – 29 .
   PubMed Web of Science ®Google Scholar
• Mahon , B. Z. and Caramazza , A. 2008 . A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content . Journal of Physiology – Paris , 102 : 59 – 70 .
   PubMed Web of Science ®Google Scholar
• Mahon , B. Z. and Caramazza , A. 2009 . Concepts and categories: A cognitive neuropsychological perspective . Annual Review of Psychology , 60 : 27 – 51 .
   PubMed Web of Science ®Google Scholar
• Massaro , D. and Egan , P. 1996 . Perceiving affect from the voice and the face . Psychonomic Bulletin and Review , 3 ( 2 ) : 215 – 221 .
   PubMed Web of Science ®Google Scholar
• Mitchell , R. L. C. , Elliott , R. , Barry , M. , Cruttenden , A. and Woodruff , P. W. R. 2003 . The neural response to emotional prosody, as revealed by functional magnetic resonance imaging . Neuropsychologia , 41 ( 10 ) : 1410 – 1421 .
   PubMed Web of Science ®Google Scholar
• Mogg , K. and Bradley , B. P. 1999 . Orienting of attention to threatening facial expressions presented under conditions of restricted awareness . Cognition and Emotion , 13 ( 6 ) : 713 – 740 .
   Web of Science ®Google Scholar
• Morais , J. and Ladavas , E. 1987 . Hemispheric interactions in the recognition of words and emotional intonations . Cognition and Emotion , 1 ( 1 ) : 89 – 100 .
   Google Scholar
• Morton , J. and Trehub , S. 2001 . Children's understanding of emotion in speech . Child Development , 72 : 834 – 843 .
   PubMed Web of Science ®Google Scholar
• Neely , J. H. , Keefe , D. E. and Ross , K. L. 1989 . Semantic priming in the lexical decision task: Roles of prospective prime-generated expectancies and retrospective semantic matching. Journal of Experimental Psychology: Learning, Memory . and Cognition , 15 ( 6 ) : 1003 – 1019 .
   Web of Science ®Google Scholar
• Niedenthal , P. M. 1990 . Implicit perception of affective information . Journal of Experimental Social Psychology , 26 : 505 – 527 .
   Web of Science ®Google Scholar
• Niedenthal , P. M. , Halberstadt , J. B. and Innes-Ker , A. H. 1999 . Emotional response categorization . Psychological Review , 106 ( 2 ) : 337 – 361 .
   Web of Science ®Google Scholar
• Niedenthal , P. M. , Halberstadt , J. B. and Setterlund , M. B. 1997 . Being happy and seeing “happy”: Emotional state mediates visual word recognition . Cognition and Emotion , 11 ( 4 ) : 403 – 432 .
   Web of Science ®Google Scholar
• Nygaard , L. and Lunders , E. 2002 . Resolution of lexical ambiguity by emotional tone of voice . Memory & Cognition , 30 ( 4 ) : 583 – 593 .
   PubMed Web of Science ®Google Scholar
• Nygaard , L. C. and Queen , J. S. 2008 . Communicating emotion: Linking affective prosody and word meaning . Journal of Experimental Psychology: Human Perception and Performance , 34 ( 4 ) : 1017 – 1030 .
   PubMed Web of Science ®Google Scholar
• Paulmann , S. , Jessen , S. and Kotz , S. A. 2009 . Investigating the multimodal nature of human communication: Insights from ERPs . Journal of Psychophysiology , 23 ( 2 ) : 63 – 76 .
   Web of Science ®Google Scholar
• Paulmann , S. and Kotz , S. A. 2008 . An ERP investigation on the temporal dynamics of emotional prosody and emotional semantics in pseudo- and lexical-sentence context . Brain and Language , 105 : 59 – 69 .
   PubMed Web of Science ®Google Scholar
• Paulmann , S. and Pell , M. D. 2009 . Decoding facial expressions as a function of emotional meaning status: ERP evidence . NeuroReport , 20 : 1603 – 1608 .
   PubMed Web of Science ®Google Scholar
• Paulmann , S. and Pell , M. D. 2010 . Contextual influences of emotional speech prosody on face processing: how much is enough? Cognitive . Affective and Behavioral Neuroscience , 10 ( 2 ) : 230 – 242 .
   PubMed Web of Science ®Google Scholar
• Pell , M. D. 2002 . Evaluation of nonverbal emotion in face and voice: Some preliminary findings on a new battery of tests . Brain and Cognition , 48 : 499 – 504 .
   PubMed Web of Science ®Google Scholar
• Pell , M. D. 2005a . Nonverbal emotion priming: evidence from the “facial affect decision task” . Journal of Nonverbal Behavior , 29 ( 1 ) : 45 – 73 .
   Web of Science ®Google Scholar
• Pell , M. D. 2005b . Prosody–face interactions in emotional processing as revealed by the facial affect decision task . Journal of Nonverbal Behavior , 29 ( 4 ) : 193 – 215 .
   Web of Science ®Google Scholar
• Pell , M. D. 2006 . Cerebral mechanisms for understanding emotional prosody in speech . Brain and Language , 96 ( 2 ) : 221 – 234 .
   PubMed Web of Science ®Google Scholar
• Pell , M. D. and Baum , S. R. 1997 . The ability to perceive and comprehend intonation in linguistic and affective contexts by brain-damaged adults . Brain and Language , 57 ( 1 ) : 80 – 99 .
   PubMed Web of Science ®Google Scholar
• Pell , M. D. , Monetta , L. , Paulmann , S. and Kotz , S. A. 2009a . Recognizing emotions in a foreign language . Journal of Nonverbal Behaviour , 33 ( 2 ) : 107 – 120 .
   Web of Science ®Google Scholar
• Pell , M. D. , Paulmann , S. , Dara , C. , Alasseri , A. and Kotz , S. A. 2009b . Factors in the recognition of vocally expressed emotions: A comparison of four languages . Journal of Phonetics , 37 : 417 – 435 .
   Web of Science ®Google Scholar
• Pell , M. D. and Skorup , V. 2008 . Implicit processing of emotional prosody in a foreign versus native language . Speech Communication , 50 : 519 – 530 .
   Web of Science ®Google Scholar
• Pourtois , G. , de Gelder , B. , Vroomen , J. , Rossion , B. and Crommelinck , M. 2000 . The time-course of intermodal binding between seeing and hearing affective information . NeuroReport , 11 ( 6 ) : 1329 – 1333 .
   PubMed Web of Science ®Google Scholar
• Preston , S. and Stansfield , R. B. 2008 . I know how you feel: Task-irrelevant facial expressions are spontaneously processed at a semantic level. Cognitive, Affective . Behavioral Neuroscience , 8 : 54 – 64 .
   Web of Science ®Google Scholar
• Rossell , S. L. and Nobre , A. 2004 . Semantic priming of different affective categories . Emotion , 4 ( 4 ) : 354 – 363 .
   PubMed Web of Science ®Google Scholar
• Scherer , K. R. 1986 . Vocal affect expression: A review and a model for future research . Psychological Bulletin , 99 ( 2 ) : 143 – 165 .
   PubMed Web of Science ®Google Scholar
• Scherer , K. R. , Banse , R. , Wallbott , H. G. and Goldbeck , T. 1991 . Vocal cues in emotion encoding and decoding . Motivation and Emotion , 15 ( 2 ) : 123 – 148 .
   Web of Science ®Google Scholar
• Schirmer , A. and Kotz , S. A. 2003 . ERP evidence for a sex-specific Stroop effect in emotional speech . Journal of Cognitive Neuroscience , 15 ( 8 ) : 1135 – 1148 .
   PubMed Web of Science ®Google Scholar
• Schirmer , A. and Kotz , S. A. 2006 . Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing . Trends in Cognitive Sciences , 10 : 24 – 30 .
   PubMed Web of Science ®Google Scholar
• Schirmer , A. , Kotz , S. A. and Friederici , A. D. 2002 . Sex differentiates the role of emotional prosody during word processing . Cognitive Brain Research , 14 : 228 – 233 .
   PubMedGoogle Scholar
• Schirmer , A. , Kotz , S. A. and Friederici , A. D. 2005 . On the role of attention for the processing of emotions in speech: sex differences revisited . Cognitive Brain Research , 24 : 442 – 452 .
   PubMedGoogle Scholar
• Shuster , L. I. 2009 . The effect of sublexical and lexical frequency on speech production: An fMRI investigation . Brain & Language , 111 ( 1 ) : 66 – 72 .
   PubMed Web of Science ®Google Scholar
• Tanenhaus , M. K. and Brown-Schmidt , S. 2008 . Language processing in the natural world . Philosophical Transactions of the Royal Society B: Biological Sciences , 363 : 1105 – 1122 .
   PubMed Web of Science ®Google Scholar
• Techentin , C. , Voyer , D. and Klein , R. M. 2009 . Between- and within-ear congruency and laterality effects in an auditory semantic/emotional prosody conflict task . Brain and Cognition , 70 : 201 – 208 .
   PubMed Web of Science ®Google Scholar
• Thompson , W. and Balkwill , L.-L. 2006 . Decoding speech prosody in five languages . Semiotica , 158 ( 1/4 ) : 407 – 424 .
   Web of Science ®Google Scholar
• Vingerhoets , G. , Berckmoes , C. and Stroobant , N. 2003 . Cerebral hemodynamics during discrimination of prosodic and semantic emotion in speech studied by transcranial Doppler ultrasonography . Neuropsychology , 17 ( 1 ) : 93 – 99 .
   PubMed Web of Science ®Google Scholar
• Vroomen , J. , Driver , J. and de Gelder , B. 2001 . Is cross-modal integration of emotional expressions independent of attentional resources? Cognitive, Affective . Behavioral Neuroscience , 1 : 382 – 387 .
   Web of Science ®Google Scholar
• Wambacq , I. J. A. and Jerger , J. F. 2004 . Processing of affective prosody and lexical-semantics in spoken utterances as differentiated by event-related potentials . Cognitive Brain Research , 20 ( 3 ) : 427 – 437 .
   PubMedGoogle Scholar
• Wilson , D. and Wharton , T. 2006 . Relevance and prosody . Journal of Pragmatics , 38 : 1559 – 1579 .
   Web of Science ®Google Scholar
• Wurm , L. H. , Vakoch , D. A. , Strasser , M. R. , Calin-Jageman , R. and Ross , S. E. 2001 . Speech perception and vocal expression of emotion . Cognition and Emotion , 15 ( 6 ) : 831 – 852 .
   Web of Science ®Google Scholar
• Zhang , Q. , Guo , C. , Lawson , A. and Jiang , Y. 2006 . Electrophysiological correlates of visual affective priming . Brain Research Bulletin , 71 : 312 – 323 .
   Web of Science ®Google Scholar