https://orcid.org/0000-0003-1279-383X
![Sample our Language & Literature journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5941/TOC-Subject-Sample-2021-Language-Literature.jpg"})
ABSTRACT
Various types of arm-and-hand movements co-occurring with speech give rise to diverse cross-modal semantic relations. This study investigated how the brain processes self-adaptors, emblems, and iconic gestures with the same speech by using fMRI. Gestures with speech evoked bilateral fusiform gyrus and left supramarginal gyrus in visual and multisensory processing. These regions were involved in processing conventional forms and meanings in emblems with speech. Iconic gestures uniquely activated right supramarginal gyrus associated with spatial processing of unconventional configurations of event knowledge of lexical concepts. Self-adaptors without meaning particularly involved the left superior parietal lobule in directing spatial attention to hand movements and processing spatio-motoric attributes with motor representations. Without gestures, bilateral superior temporal gyrus and calcarine sulcus were engaged in processing sounds and meanings of audiovisual speech. The overall results demonstrate the brain’s sensitivity to gesture-speech semantic variation, and reveal the nature of knowledge in the involved neural regions.
Acknowledgements
This research was supported by grants from the Ministry of Science and Technology, Taiwan, ROC (MOST 103-2410-H-004 -180 -MY3).
Disclosure statement
No potential conflict of interest was reported by the authors.
Correction Statement
This article has been republished with minor changes. These changes do not impact the academic content of the article.
Notes
1 The data from The NCCU Corpus of Spoken Taiwan Mandarin can be accessed online at http://spokentaiwanmandarin.nccu.edu.tw/ or at TalkBank https://ca.talkbank.org/access/TaiwanMandarin.html