526
Views
1
CrossRef citations to date
0
Altmetric
Research Article

Detecting emotion in speech expressing incongruent emotional cues through voice and content: investigation on dominant modality and language

ORCID Icon &
Pages 492-511 | Received 01 Feb 2021, Accepted 16 Dec 2021, Published online: 03 Jan 2022
 

ABSTRACT

This research investigated how we detect emotion in speech when the emotional cues in the sound of voice do not match the semantic content. It examined the dominance of the voice or semantics in the perception of emotion from incongruent speech and the influence of language on the interaction between the two modalities. Japanese participants heard a voice emoting anger, happiness or sadness while saying “I’m angry”, “I’m pleased” or “I’m sad”, which were in their native language, in their second language (English) and in unfamiliar languages (Khmer and Swedish). They reported how much they agree that the speaker is expressing each of the three emotions. Two experiments were conducted with different number of voice stimuli, and both found consistent results. Strong reliance on the voice was found for the speech in participants’ second and unfamiliar languages but the dominance was weakened for the speech in their native language. Among the three emotions, voice was most important for perception of sadness. This research concludes that the impact of the emotional cues expressed by the voice and semantics varies depending on the expressed emotions and the language.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research was supported by Grants-in-Aid for Scientific Research [18K03021] by Japan Society for the Promotion of Science, awarded to the first and second authors.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.