Analysing tension between language and images: a social semiotic view

ABSTRACT

This study explores tension between different modes from a social semiotic perspective. Tension and cohesion are viewed as two poles on a continuum of meaning relations ranging from oppositeness to similarity, and image-text interplay is analysed as a balance between tension and cohesion. Focusing on the device of evaluative dissonance, the multimodal analysis of two verbal-visual combinations from different genres (a procedure and a visual irony) demonstrates how attitudinal tension is created between language and images to make a deeper meaning such as foreshadowing the appraiser’s opposing feelings in the ensuing text and facilitating the construction of specific ideologies. The analysis also illustrates the multiplicative nature of meaning in the verbal-visual orchestration and uncovers that attitudinal meanings can be implicitly conveyed through image-text interaction and contextualising information. This study concludes that tension does not contradict, but closely cooperates with cohesion to produce a coherent message despite their semantic differences.

KEYWORDS:

• Tension
• cohesion
• evaluation
• Attitude
• multimodality

Acknowledgements

I am deeply grateful to Prof. Gwen Bouvier and the anonymous reviewers for their constructive comments on the earlier version of this article. I would also like to acknowledge the support received for this study from the Group for Linguistic Big Data Research, Sichuan International Studies University, China, where I have been a member.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 In the analytical framework of interpersonal intersemiotic complementarity, Royce (1998, 36) proposes “attitudinal dissonance” as a mechanism to relate the reader/viewer and the visual-verbal combinations through the relation of “an opposite or ironic attitude” regarding “views on possibility, probability, and certainty”. Therefore, “dissonance” is regarded in this study as a mechanism to arouse opposite meaning. By contrast, “ideational dissonance” in Yu’s (2019) research does not refer to a specific meaning construction device, but covers a range of meaning relations including semantic difference or variation apart from opposite meanings.

2 The examples of inscribed and provoked attitudes are adapted from Martin and White (2005, 67) and those of invited evaluation are taken from White (2014, 10, 16). Following Martin and Rose (2003, 31), this study indicates inscribed attitudes in bold as opposed to invoked attitudes with highlights.

3 For instance, it would be difficult to visualise the feelings of “love” or “hate” in an unambiguous way (Economou 2009, 112).

4 The verbal part of this documentary condemned the terrorists’ attack on the twin towers and thus inscribed a negative Judgement. However, the visual images, especially the shots which showed people jumping out of the tower and falling, invoked a positive Appreciation, as “they encourage a view of the scene as aesthetically pleasing” (Economou 2009, 143).

5 Excerpted from “Feeling homesick when studying abroad” (Archive). Lancaster University International Study Centre. 15 Aug. 2019. Retrieved on 10 Feb. 2020, from https://www.lancasterisc.com/news/feeling-homesick-studying-abroad.

6 The notion of “evaluative clash” (Partington 2017) can be regarded as “evaluative dissonance” in this study within the same mode of language.

7 “Engrish FAQ” (n.d.). Retrieved on 10 Feb. 2020, from http://www.engrish.com/engrish-faq/#Q15.

8 “Engrish FAQ” (n.d.). Retrieved on 10 Feb. 2020, from http://www.engrish.com/engrish-faq/#Q16.

9 “Engrish FAQ” (n.d.). Retrieved on 10 Feb. 2020, from http://www.engrish.com/engrish-faq/#Q14.

10 “Engrish FAQ” (n.d.). Retrieved on 10 Feb. 2020, from http://www.engrish.com/engrish-faq/#Q9.

11 The customer is assumed a Japanese or at least an Asian regarding ethnicity based on the physical attributes such as hair and skin colour.

12 For achieving gender-neutrality, this study uses the third person plural form (they, them, their) to refer to the customer in Figure 5 as a singular person.

13 “I am a little butch though … ”. (n.d.). Retrieved on 10 Feb. 2020, from http://www.engrish.com/2012/01/i-am-a-little-butch-though/.

Additional information

Notes on contributors

Yu Liu

Yu Liu holds a PhD in English Language from National University of Singapore, and is currently a professor at the College of International Education, Sichuan International Studies University, China. His research interests include multimodal discourse analysis, systemic functional linguistics and scientific literacy. He has published research articles in journals such as Semiotica, Social Semiotics, Language Sciences, and Chemistry Education Research and Practice.