Sensory threads of history: a dive into two historical documentaries

Abstract

Historical documentaries not only involve narrative indexicality and archives, but also dedicate to cultural landscape, distinctive perspectives and ethical enlightenment. So, does this legacy activate sensory experiences? Will it vary by culture? Based on previous research on multimodal discourse analysis (MDA), we hypothesize that it can evoke sensory memory in people, whose meaning-making may differ among cultures. To test the above hypothesis, we establish two multimodal corpora from historical documentaries Frozen Chosin and The Battle of Chosin depicting the same historical event from different cultural perspectives, using ELAN 6.4 to analyze sensory contours of coldness. The findings reveal that filmmakers actualize frostbite-laden experiences by adopting a structural adversity of iciness, with compositional meaning-making as the top, interactive as the second, while the last two are representational and linguistic. Behind this overall homogeneity, the lethal frostbite during engagement and post-battle are heterogeneously mediated by different ways of contacts, perspectives and salience, in which the Chinese film narrates in a slightly convergent multimodality while the American depicts a divergence-prone pattern. Finally, we justify that MDA can help scholars gain deeper insights into the sensory endeavor made by documentarists.

Impact Statement

In the current media-driven society, the power of documentaries transcends mere visual narratives; it encompasses a sensory trajectory that intertwines sound, visuals and emotions to create a deep empathy among viewers. This research delves into the intricate sphere of sensory meaning-making within historical documentaries, exploring how the convergence of sensory elements affects audience engagement and perception. By clearing up the ways in which sound, visuals and narrative structures interlink to shape viewer experiences, this article aims to illuminate the role of sensory meaning-making in cultivating empathy and viewer engagement. Through an interdisciplinary approach merging film studies and discourse analysis, this research seeks to unveil the potential for documentaries for sensory analysis in this increasingly multimodal world.

Keywords:

• Sensory meaning-making
• multimodality
• MDA
• Chosin
• frostbite

REVIEWING EDITOR:

• Guangchao Charles Feng, School of Communication, Hong Kong Baptist University, Hong Kong

SUBJECTS:

• Intercultural Communication; Media Communication; Art & Visual Culture; Visual Arts; Film Studies

1. Introduction

With increasing availability and robust demands in multisensory technologies, such as virtual reality (VR), augmented reality (AR), multi-user interface (MUI) on social media, the past decades have witnessed a burgeoning number of studies upon sensory meaning-making in a variety of social settings. Researchers have been making every endeavor to integrate emergent semiotic resources to explore sensory significations. (Howes, 2005: 400; Dicks et al., 2006: 77; MacDougall, 2006: 3; Hurdley & Dicks, 2011: 278; Low, 2019: 618).

The wealth of media platforms within reach has provided a valuable springboard for the investigation of a repertoire of sensory meaning-making on social interactions. In Empire of the Senses: The Sensual Culture Reader, Howes (2005: 3) illustrated the dominating characteristic of thermosensory meaning-making in a particular ethnic group, the Tzotzil of Mexico, to highlight the essential functions of all social artifacts of ‘heat’ in their everyday interactions.

Indeed, these ground-breaking works were conducive to interpretation in sensory meaning-making. Meanwhile, film premieres on coldness and winter weather, for example, Dead of Winter (Penn, 1987), The Last Winter (Fessenden, 2006), Whiteout (Sena, 2009), Frozen (Green, 2010), Let It Snow (Kapralov, 2020)¹ have echoed the point that filmmakers explore coldness in their semiotic undertakings to reveal the fact that it is more perilous than expected. However, the exploration of sensory elements within film storytelling and the examination of embedded meanings using multimodal discourse analysis (MDA) framework require further investigation. Considering the abovementioned fact, the article attempts to establish two multimodal corpora, centering specifically on the physical cold environment, between Frozen Chosin (Yan-pin et al., 2011) and The Battle of Chosin (MacLowry, 2016), explores multimodal revelations of frostbite past, and aims to answer the following research questions: (1) Can the two historical documentaries depicting the same historical event from different cultural perspectives reflect nuanced differences on sensory meaning-making of a physical cold environment? (2) How do they utilize multimodal synergy for this coldness? (3) Will this sensory meaning-making in the two documentaries vary by different cultural norms?

1.1. The use of multimodal theories to filmic materials

The structured MDA to filmic arts, at the outset, can be traced back to Rhetoric of the image (see e.g. White, 2012: 65; Luo, 2021: 105; Hu et al., 2022: 115), in which Barthes elucidated the ‘anchorage and relay’ roles of linguistic messages about ‘floating chain’ of advertising pictures (Barthes 1977 : 38–39). Later on, inspired by the social semiotic and functional grammar theories of Halliday (1978, 1994), Kress and Van Leeuwen led a new semiotic agenda by systematizing novel inventories of visual grammar and categorized three meanings, namely, representative, interactive and compositional (e.g. Forceville 1996: 164; Kress and Leeuwen 1996–2006: 42–43; Zhang, 2018: 736). This systemic pursuit of multimodality is followed through by Norris’s proposal of a modal density foreground-background continuum to qualitatively analyze participants’ continual interactions in constructing social worlds. In an ethnographic film study, Norris (2004: 11) recorded an interviewee’s behaviors in communicative settings and described self-perceived identities as an interactive and meaning-making event simultaneously engaging in ‘higher-level actions’, which can be dynamically shifted to sub-lower-level under hierarchically new foregrounded actions. Being the first to wander through a newly-trodden path on corpus-based multimodal video studies, Gu (2006: 127–167) outlined a corpus linguistic approach to multimodal text analysis starting from ‘a hierarchical configuration’ of analytical units, to ‘that of activity type, episode, and the participants’ behavior’. In view of interactions of modes, Zhang (2018: 740) built a synthetical MDA framework based on Halliday’s systemic-functional theory and Kress and Van Leeuwen’s visual grammar. The framework broke down modes (a controversial concept) into various levels, culture, context, content and expression, and illustrated complementary and non-complementary relations.

Subsequently, the development and application of computer-aided multimodal software have laid a solid foundation for analyzing multimodal corpora, including Multimodal Corpus Authoring (MCA) developed by Baldry and Thibault, ELAN developed by the Max Planck Institute for Psycholinguistics in Nijmegen and ANVIL, developed by Michael Kipp (Jewitt, 2014: 245). Comparatively, ELAN is highly flexible and user-friendly, facilitating researchers with customizable multimodal annotations, hierarchically interconnected tiers and comprehensive supporting features.

The establishment of cross disciplinary approaches to social semiotics in visual arts including cinema brings profound changes toward unconventional aesthetics and perspectivism. Film critics even contended that such a semiotic momentum was so prevailing that it ‘heralded the end of all traditional aesthetics’ (Lapsley & Westlake, 2006: 32).

Under systemic-functional framework, diverse multimodal researches have been undertaken to demonstrate the importance of semiotics in their analysis of films (see e.g. O’Halloran, 2004: 110; Baldry & Thibault, 2006: 3; Bateman & Schmidt, 2011: 1), such as the analysis of the detective film Chinatown 1974 by O’Halloran. The film won a great acclaim among public as ‘a complex detective thriller with dimensions which are political (about the nature of power), sexual (about the nature of gender), metaphysical (about the nature of the evil), psychological (about the nature of the self) and philosophical (about the nature of knowledge)’ (Eaton, 1997: 43). O’Halloran ranked this multi-constitute structure with ‘Film plot, Sequences, Scene, Mise-en-Scene and Frame’ for the systematic analysis of compositional and interpersonal meaning in two short scenes (O’Halloran, 2004: 114). Unfortunately, due to the issue of copyright permission at that time, O′Halloran also admitted the difficulty to replicate still frames in real time to conduct a much more productive analysis (Ibid: 113).

Further research on meanings in filmic materials is, characterized by granularity of two meta-functions, namely, ideational and interpersonal. The first is portrayed corresponding to the diegesis, while the second, the interpersonal, is mostly engaging in considerations of affect and has been observed, notably from a holistic effect. For example, continuous zoom-ins in Stanley Kubrick’s Eyes Wide Shut (1999) revealed the protagonist Tom Cruise’s worries and fear when he grasped a newspaper on the street (Bateman & Schmidt, 2011: 5–6). However, fewer empirical analysis on textual function have been attended to. In order to obtain a deeper understanding in filmic genres as well as to extend current systemic functional fieldwork, more empirical researches of interdisciplinary nature come to the fore. Bateman presented ‘a Hallidayan-inspired film montage’ concept in analyzing Jean-Luc Godard’s film Pierrot le fou (1965) in which he showed that the story world was horizontally hierarchized by spatiotemporal syntagma and any syntagmatic deviations beyond chronological order involved specific genres, medium and strategic purposes of the film (2013: 657–658). In the similar vein, Tseronis and Forceville studied Frederick Wiseman’s direct cinema documentaries (Titicut Follies, High School, Hospital, etc.) and argued that topical sequencing of shots, reciprocation of image, dialogue and sound could pose multimodal antithesis, which in turn, was conducive to the argument (Tseronis & Forceville, 2017: 165).

1.2. The sensory meaning-making basis of filmic multimodality

In order to gain insights into what sensory experiences are produced in material environments and digital world and what meanings are conveyed, over the last few decades scholars engaging in social science have integrated a sensory dimension (Farquhar, 2002: 57; McLuhan, 2003: 86; Dicks et al., 2006: 77; Hurdley & Dicks, 2011: 277; Pink 2015: 3–4). Researchers in anthropology have also pointed out the interwoven nature between sensory meaning-making and representation of images. For example, Farquhar (2002: 57) contended: ‘Direct sensory experience, the material attributes of concrete things and mundane activities, can be invoked, and thereby imagined, but only by way of language and images and only in the context of times, places, and habitus that impose constraints on what can be experienced or imagined’. Dicks et al. (2006: 83) went beyond visual methods and conducted a preliminary exploration of how multi-semiotic forms can account for different sensory meanings in an interactive science context. Noting ‘the distinctive semiotic affordances of different media’, he trialed a multimodal sensory meaning-making experiment and concluded that schoolchildren visit in the multi-semiotic exhibition can pick up different meanings than making sense of classroom textbooks.

It is fair to say ‘films appeal in an even more direct way to the human sensorium, in part because of the senses they address and the fact that they address them’ (MacDougall, 2006: 57). This argument resonated with McLuhan’s (2003: 85) view that all media are powerful translators which render immediate human sense experience into new forms so that ‘the entire world can be evoked and retrieved at any instant’. Moreover, in his book The Corporeal Image, MacDougall extended the unfolding quality of images and specifically highlighted the influential power of films on sensory expressions ‘through the different emphasis they place on foreground and background, the signified and the ambient, perception and thought’ (2006: 57). For instance, the remarkability of childhood was how complicated recasting of children’s images could be made by filmmakers. However, MacDougall pinpointed ‘the aesthetic dimension of social experience’ as ‘a relatively undeveloped area’ and ‘open to investigation in the visual media’ (Ibid: 59).

Hurdley and Dicks (2011: 277) approached this seemly contradictory, binary mechanism and underscored the co-existent methodological rationality between sensory study and linguistic field of multimodality, and called for more cross-disciplinary fieldwork in sensorial settings because ‘Eliciting the tensions between sensorial closeness and modal distances offers a new space for reflexive research practice and multiple ways of knowing social worlds’ (Hurdley & Dicks, 2011: 277). As other researches demonstrated, ample academic efforts have also been made to approach sensory memory with a specific focus on facial gestures in transgenerational interactions regarding memories of genocide. The sense is attended to as ‘tokens of meaning…allow people to infer horizon of expectations to the emotional trajectory’ (Mascia-Lees, 2011: 459).

Although existing studies have touched upon certain sensory categories in visual world and attention has been paid to this twin methodological feature, insufficient data may be obtained to explore what kinds of new sensory meaning are produced after multiple interconnections and how modes reconcile with each other in a holistic pursuit.

1.3. Multimodal researches on documentaries

The emergent approaches to multimodal analysis of documentaries mainly focuses on three aspects: first of all, academia is interested in how conceptual cognition can be metaphorized in multimodal semiotic discourses, especially in nature documentaries, and what are the driving contextual factors behind its image reconstruction (Vermenych, 2020: 115; Wu et al., 2021: 31). For example, Vermenych (2020: 115–122) investigated the synergy of verbal, visual and aural elements in an English documentary and explored how modes extended, elaborated and combined with each other in unconventional meaning construction and awareness-raising for urgent ecological issues. Recent years have also seen a vibrant national image discourse analysis, specifically on COVID prevention and control documentaries (Tang et al., 2020: n. p.; Li & Xing, 2022: 79–95; Fei & Zhang, 2022: 52). Li and Xing (2022: 95) paid a special attention to the attitudinal meaning in the verbal and visual sources of The Frontline: China’s Fight against COVID-19 (Ge 2020) and demonstrated ‘an aesthetic spirit of heroism, optimism, collectivism and idealism’ which echoes the important functions of documentaries, educational and persuasive. MDA also features an eclectic collection of intercommunication themes (Yuan & Nai, 2022: 23). For instance, Yuan and Nai conducted systematic research on a BBC documentary Chinese New Year, the biggest Celebration on Earth (Locke 2016) and analyzed how ‘Cultural China’ was represented in cultural, content, context and expressive planes and what orchestrated strategy could be applied to facilitate China ‘go global’ target (2022: 23).

Being part and parcel of this meaning-making in intercommunication, sensory experiences, social contexts and their relationship in documentaries have been underscored by more and more eastern Asian academia (see e.g. Fei & Zhang, 2022: 52). Although considerable endeavor on multimodal cinema discourse has been highlighted, a lack of corpus-based approach imposes restrictions on quantitative semiotic descriptions. Sensory meaning-making of historical documentaries is not immune to this challenge.

2. Analysis

Considering the research questions, audience ratings and cultural diversity, we established the multimodal corpora of the physical cold environment in the two documentaries, especially on visual and linguistic resources of the two films. In the light of MDA framework proposed by Zhang (2018: 740), we utilized ELAN 6.4 to annotate each discourse and transcribed its images and narrations. At last, we probed into cultural contexts to discuss this coldness and its synthetic effect in two culture-bound environments.

2.1. Corpora

The event of Chosin Reservoir in freezing weather has left an indelible impression on the public, inspiring many filmmakers to recreate the historical tragedy. The two films recalled a vivid and bone-chilling environment to the audience. Both depict the battle fighting during one of the harshest winters in war history and recorded a temperature plunge to as low as −36 °F (−38 °C) accompanied by frozen ground, huge frostbite casualties, icy roads, weaponry and maintenance supply malfunctions (Berquist 2014: 58). We select visual and linguistic resources of the physical cold environment for two reasons. First, we can probe extensively the interaction between two pivotal forces, namely, narrations and visuals and analyze how filmmakers represent them. Second, we can also bring the semiotic value and its historicity to the fore, being instrumental to assessing the senses and sensory output of documentaries.

The Chinese film was released by China August First Studio in 2011, presenting the event: The Ninth Corps of Chinese People’s Volunteer Army (hereinafter called ‘Chinese Army’) (160,000 troops) divided and encircled US 1st Marine and 7th Infantry Division (approx. 30,000 soldiers) in the Chosin area, the northeast of Korea. The 1st Marine was seriously attacked with 13,961 casualties, meanwhile Chinese Army also paid a heavy price. The number of combat casualties reached 14,000, and non-combat casualties by frostbite was even 30,732.² According to Douban Dataset, the most popular movie review website in China, the film was well-received by 9.3/10 positive feedback from 14,819 viewers.

The American documentary was premiered by PBS American Experience in 2016, which won 7.9/10 positive reflections from 1998 comments. The film was about the experience of the US military troops, and various winter adversity, miseries, especially the physical coldness they suffered in the battle.³ IMDb database also granted a top-notch rating 7.8/10⁴ and Mike D. explicated: ‘…an amazingly harrowing story of the 17-day engagement of bloody combat and heroic survival in subarctic temperatures’⁵.

2.2. Theoretical framework of multimodal sensory meaning-making

Systemic-functional linguists, Halliday and Matthiessen (2004: 24) contended that language serves a ‘meaning potential’ stratified by a given cultural and social network. Derived from Halliday’s systemic-functional grammar, Kress and Leeuwen (1996–2006: 2) extended the linguistic paradigm toward visual design and provide a brand-new visual solution for semiotic landscape which involves representational, interactive and compositional meanings.

In terms of representational meaning, narrative and conceptual processes are involved and the difference lies in vectors (Kress and Leeuwen, 1996–2006: 59). Based on Kress and Van Leeuwen’s clarification, narrative models ‘serve to present unfolding actions and events, processes of change, transitory spatial arrangements’ (Ibid.: 59). A diagonal line is its essential mark of conceptual pattern, representing participants in terms of class, structure and meaning (Ibid.: n. p.).

An alternative path to gauge interactions is to reflect the relative directionality between the viewer and the person depicted in the image, configuring contacts, distances and perspectives (Ibid.: n. p.). Finally, the third compositional element is divided into information value, salience and framing to describe pictorial representation as a total layout (Ibid.: 177).

Given realistic interplay of modes, Zhang developed a MDA framework based on Halliday’s systemic-functional theory and Kress and Van Leeuwen’s visual grammar (2018: 740). Inspired by the previous endeavors (Halliday, 1994; Kress and Leeuwen, 1996–2006; Zhang, 2018), this study attempts to examine how sensory meaning-making is established through two historical documentaries based on the framework of multimodal sensory meaning-making of the physical coldness (Figure 1).

Figure 1. Framework of multimodal sensory meaning-making of the physical coldness (revised from Zhang’s MDA framework 2018).

2.3. Methodology

By examining the structural cohesion between the two documentaries, we categorize the entire length of the films according to their narrative continuity, as illustrated in Tables 1 and 2. Subsequently, we employ ELAN 6.4 to annotate each discourse, transcribing both its visual images and narrations based on our established framework. Finally, we delve into cultural norms to explore the physical coldness and its synthetic impact within two distinct culture-bound environments.

Table 1. Narrative characteristics of Frozen Chosin.

Discourse	Content	Duration	Time ratio (%)
Prologue and before	96-Year-old veteran’s poor shape was shown with a strong will in hospital bed. Chinese 9th army advanced to North Korea and covertly encircled US-led UN troops in Chosin Reservoir	00:00–43:13	45.53
Engagement	A brutal battle in freezing weather broke out between Chinese 9th army and US X corps.	43:14–01:07:19	25.37
After	The extreme harsh winter frostbit Chinese army. US 1st marine division retreated in the aftermath.	01:07:20–01:34:55	29.06
Total		01:34:55	100

Table 2. Narrative characteristics of The Battle of Chosin.

Discourse	Content	Duration	Time ratio (%)
After	US marine was encircled and attacked by Chinese troops in wintry, poor conditions.	00:10–03:12	2.67
Before	Self-assured of rapid and decisive victory, General Douglas MacArthur ordered US Corps to proceed Chosin and conquer North Korea territory.	03:13–18:02	13.05
Engagement	Chinese 9th army surprised US X Corps and a fierce battle broke out in freezing winter weather.	18:03–46:53	25.4
After	US army was forced into full retreat to the port of Hungnam.	46:54–01:53:40	58.83
Total		01:53:30	100

2.4. Research procedure

This article used ELAN version 6.4 and the default coding scheme proposed by Wang and Wen (2008: 10) who designed a phonological coding scheme to study how English as second language (ESL) teachers apply phonological characteristics to facilitate their classroom teaching in a foreign context. The two researchers have trained themselves on the coding scheme and conducted a pilot test. Based on the measurement process and reliability, we have incorporated the following revised scheme consisting 7 tiers and 18 affiliated variables on the physical coldness as shown in Table 3. Furthermore, in order to explicate the interplay of modes, we picked up four shots to probe into the role that they play and how modal synergy can affect sensory meaning-making of coldness.

Table 3. ELAN coding scheme.

Annotation	Coding	Meaning	Tier	Type
Action	RNA	Narrative action	Narrative	Representational
Reaction	RNR	Narrative reaction
Mental	RNM	Narrative mental
Speech	RNS	Narrative speech
Classificational	RCC	Conceptual classificational	Conceptual
Analytical	RCA	Conceptual analytical
Symbolic	RCS	Conceptual symbolic
Offer	ICO	Contact offer	Contact	Interactive
Demand	ICD	Contact demand
Long	IDL	Distance long	Distance
Short	IDS	Distance short
medium	IDM	Distance medium
Horizontal	IPH	Perspective horizontal	Perspective
Vertical	IPV	Perspective vertical
Centre-margin	CC-M	Dominant and marginal	Information value	Compositional
Left-right	CL-R	Given and new
Top-down	CT-D	Ideal and real
Maximum	MAX	Salience	Salience

In the above scheme Perspective horizontal (IPH) is subdivided as IPHfront, IPHoblique and Perspective vertical (IPV) is partitioned to IPVhigh, IPVlow. ‘Information value’ includes: center to margin, left to right, top to down and vice versa. The salience is broken into maximum and low ranks, depending on color saturations, whether the element is shown in full colorful brightness or barely black and white. The last framing element is comprised of connectedness or disconnectedness according to the features whether there is a clear framing device, such as a framing line or not, continuity or discontinuity of color, shape and other features.

3. Results

First, the total distribution of annotated modes of the physical cold environment for each film was calculated in Figure 2. In the American documentary, compositional meaning holds the biggest total annotation duration, accounting for 11.49%, and the next is interactive with 2315.053 s, equivalent to 11.315% of total duration percentage. In order to maximize the bone-chilliness, the documentarist proposes diverse alignments and framings to emphasize or de-emphasize this physical coldness. Moreover, this sensory experience is also narrated in a salient interaction in which eye contact, distance and perspective play a crucial role within its empathetic communication. Representational and linguistic modes remain least. To our surprise, the Chinese adopts the same hierarchy of icy adversity, with compositional as the top, interactive as the second, while the last two are representational and linguistic. In the current dataset, we are also interested in observing how three discourses of physical coldness are systematized by different filmmakers (Tables 4, 5 and 7).

Figure 2. Multimodal sensory meaning-making visualization.

Table 4. Annotation data of ‘before the battle’ stage.

	Number of annotations		Total annotation duration		Annotation duration percentage
Film Tier	CH	US	CH	US	CH	US
Narrative	3	3	61.73	36.75	1.084	0.539
Conceptual	1	–	7.43	–	0.13	–
Contact	3	4	59.62	36.7	1.047	0.538
Distance	5	4	58.15	34.87	1.021	0.511
Perspective	5	6	58.08	36.683	1.02	0.538
Information value	4	3	58.64	32.91	1.03	0.483
Salience	4	3	58.64	32.89	1.03	0.482
Framing	4	3	58.66	32.91	1.03	0.483
Linguistic	15	4	27.863	19.66	0.489	0.288

Table 5. Annotation data of engagement.

	Number of annotations		Total annotation duration		Annotation duration percentage
Film Tier	CH	US	CH	US	CH	US
Narrative	13	7	198.09	99.93	1.976	6.359
Conceptual	8	3	91.13	75.81	2.184	2.177
Contact	18	9	200.619	125.33	4.166	8.461
Distance	34	15	280.55	175.67	4.121	8.463
Perspective	25	16	280.71	174.41	4.128	8.462
Information value	36	11	279.43	175.69	4.126	8.463
Salience	17	9	281.759	175.74	4.137	8.464
Framing	21	9	281.631	175.74	4.137	8.367
Linguistic	18	16	271.941	103.92	3.501	5.483

3.1. Characteristics of sensory meaning-making before the battle

During the total duration 43 m 13 s (see Table 1), the Chinese highlights an abrupt strategic maneuver of Chinese Army moving from Taiwan to North Korea, considering the fact that the Americans led force attacked North Korea and moved across 38 Parallel Line (Military Demarcation Line separating the South from the communist North in Korea), threatening Chinese borders. This important geographic change of field would result in inadequate winter-proof logistics and great losses in the harsh winter. However, the historical context must be taken into consideration that China was only founded on 1 October 1949, leaving extreme impoverishment after a long civil war, without experience to combat abroad. Witnessing this enemy crossing 38th Parallel Line, the newly born China might be in jeopardy and she was willing to be cold-resistant at her disposal. On the other hand, the Americans would possess a cutting edge than the Chinese in iciness protection since its military have experienced overseas combat in frozen weather, such as the battle of the Bulge (Sears, 2004: 62). It has possessed the only atomic bomb and a larger quantity of modern military technologies (Kirkland, 1995/96: 261).

The finding narrates a different actuality in Table 4: in the second column ‘Number of annotations’, instead of taking a dominate warmth for American troops and suffer a harrowing downside coldness, both are highly consistent to construct a similar sensory experience of coldness before the battle, focusing on interactive and compositional meaning-making of coldness, with a slightly more in Chinese. However, compared to under-developed logistics of Chinese regiments, ‘total annotation’ and ‘duration percentage’ explicitly unveil a modest rate of cold suffering in the American film and justify its cold resistant strength and military superiority.

This article selected four shots to probe into the multimodal sensory signification across cultures (as elaborated later): CH1 and CH2 (22:00.750–22:12.840), US1 and US2 (09:00.160–09:14.170), CH3 and CH4 (01:10:49.380–01:11:37.630) and US3 and US4 (01:16:54.740–01:17:04.610) for two reasons. First of all, these shots are framed in the same discourse describing a similar scene: the first two illustrate the mobilization of soldiers prior to the battle, while the latter two reveal the catastrophic frostbite experience after the battle. Second, these shots showcase the synergy between visual and linguistic elements in conveying the sensory experience of physical coldness, which can indicate nuanced cultural differences. In Figure 3, the linguistic modality elaborates Chinese soldiers from southern China only wore thin winter clothes and left to the frontline far north due to hasty strategic deployment. And the visual modality complements the fact that soldiers’ march toward the screen without thick clothes available. This synergy of multimodality is anchorage complementary, with the language ascertaining the visual. Here, the moving regiment is foregrounded as a non-transactional vector; therefore, it is a narrative RNA process.

Figure 3. CH1 and CH2 (source: pre-battle snapshots from Frozen Chosin, 2011).

Literary translation of the left caption: ‘The soldiers were only wearing thin winter clothing distributed in the Jiangnan region’. Right: ‘Hurriedly drove towards the front line’.

Filmmakers use ‘offer’ and ‘demand’ to show different engagements between participants and viewers: when a sense of connection and affective communication is required – the ‘demand’ picture is preferred; when realness or imagination is desired – the ‘offer’ is preferred (Kress and Leeuwen, 1996–2006: 120). Since no direct gaze from active participants and realness is prioritized, an offer ICO is justified. In terms of ‘distance’, shots vary from a long range IDL to relatively medium social distance IDM. The perspective also diverges between the shots. Beginning with a vertical view IPVlow, the shot unfolds the murky grey background with zigzag mountain path. Straw hats and grey uniforms were perfect camouflage protecting them from Americans. Then the shot fades into IPHfront, a horizontal and conspicuous close-up of the foreground – soldiers rolling up their sleeves and crossing over the river. This physical cold environment metaphorically echoes a Chinese catchphrase ‘roll up your sleeves, blaze new trails and work together to achieve the Chinese dream’.

In the compositional plane, the story reads from left to right; therefore, information value is CL-R, starting from given to new. The salience is relatively low, with low saturation consisting of solely white and black. There is no diagonal line or other barriers blocking soldiers, so ‘framing’ is characterized as ‘connected’. As Figure 2 illustrates, the findings validate the idea that in Chinese film, there is a strong emphasis on the interactive and compositional significance of coldness in meaning-making.

The American prior engagement accounts for 14 m 49 s, or 13.05% of the total (Table 2). The visual modality of Figure 4 is reactional processes RNR, because the vector is formed by the eyelines of US marine, or ‘by the direction of the glance of one or more of the represented participants’ (Kress and Leeuwen, 1996–2006: 67).

Figure 4. US1 and US2 (source: pre-battle snapshots from The Battle of Chosin, 2016).

Literary translation of the left caption: ‘At that time, they activated all the Marine Corps reserves’. Right: ‘I was too young to participate in World War II’.

With regard to its interactive meaning-making, young soldiers all mysteriously gaze at the camera, ICD, as if sharing reluctance to join the battle overseas, or an arrogance about enemies. The linguistic in US2 contrasts with the visual of embarking, explaining that some are too young to WWII and fear of being enrolled. The scene proceeds with low morale and heavy steps. Therefore, this synergy of multimodality is non-complementary interaction. In terms of ‘distance’ and ‘perspective’, the shots are taken similarly, shifting from public long distance IDL to a relatively medium social distance IDM. Beginning with a vertical IPVlow, it foregrounds with soldiers’ march while backgrounding the White House, exorbitant vehicles and rushing journalists. Then the shot fades into a horizontal IPHfront, close-up of the foreground – young soldiers getting aboard.

As for the compositional meaning of sensory marching, the visual reads from given to new, CL-R, which is a typical anglophone reading path. The background image of the White House, signifies UN peace-keeping troops are deployed there, a signature political power as well as a ‘global policeman’. The salience is also low, legitimizing a black and white historical memoir. There does not exist a clear discontinuity of color or shape between young soldiers so it is ‘connected’ in terms of framing. This support the notion that both filmmakers place significant emphasis on the use of interactive and compositional maneuvers in narrating the pre-battle while a lack of complementary synergy between linguistic and visual elements can be observed in the American film.

3.2. Characteristics of sensory meaning-making engagement

In both films, the engagement is portrayed around a quarter of the entire runtime (Tables 1 and 2). In Table 5, the portrait of its cold environment largely relies on multimodalities, especially on interactive and compositional meaning-making, to record the liveliness of cold encounters where frostbite could be extremely destructive. This resonates with Figure 3 justifying the fact that historical documentaries, also stress the importance of artistic arrangement, poetic combination of multiple images in partaking the revelation of collective experience, although traditional voice-over narrations are still a significant proportion (Nichols, 1991: 21).

Meanwhile, the data in Table 6 shows that the Chinese film is annotated most frequently for ‘information value’, while the American film is annotated equally for perspectives and linguistic modality. The Chinese navigates through manifold paths, left to right, top to down, center to margin and vice versa, presenting a ubiquitous devastating effect due to windiness, snowfall and temperature plunge. This finding demonstrated a holistic communication for information giving and challenged the notion that Asian filmmakers typically prioritize ‘centering a fundamental organizational principle in the visual semiotic of their culture’ (Kress & Leeuwen, 1996–2006: 195) The American invites its audience on more divergent points of view to witness such an actuality-based frozen misery and raise anti-war awareness. An intense linguistic modality throughout the film bolsters its argument. In other words, the American documentary’s storytelling approach is characterized by a relatively balanced distribution of annotated elements, which allows for a nuanced and multi-faceted portrayal of the event. The intensity of the documentary is conveyed implicitly through a carefully orchestrated combination of these elements.

Table 6. Annotation spreadsheets of coldness during engagement.

3.3. Characteristics of sensory meaning-making after the battle

It is noteworthy that the total number of annotated elements indicated in Tables 7 and 8 surged during the retreat and counter-retreat, pointing to the fact that both armies encountered even more severe adversity, including extreme coldness and frostbite. This is at odds with the common perception that frostbite during engagement may be the worst. Both employ visual and linguistic modalities to present this disastrous sensory experience (e.g. CH3 and US3). To our surprise, the number of linguistic modalities from American wins against the Chinese in a wide margin (Table 8). The gap might be caused by potential pauses in grammatical and rhythmic patterns. American English frequently incorporates adverbial phrases, parentheses and complementary elements that are used to signal pauses, whereas Chinese philosophy of Confucianism stresses a natural flow of forms. The second outstanding factor is ‘distance’ in interactive meaning-making (Table 8), American accounts for 51 while Chinese is 25 (Table 7). Among the total annotations, short distances of the films rank the first, medium social the second, and the last is long distance. In terms of perspective, both rely heavily on horizontal front in order to vivify coldness-stricken pains and disastrous memory. Another noticeable feature of interactive meaning-making in coldness is contact. Among 51 American contacts, 22 is ‘demand’ accounting for 43.14%, while among 17 Chinese contacts, 6 is ‘demand’ justifying 35.29%. Different from ‘offer’ stance, demand ‘requires an imaginary social response of some kind from the viewer’ (Kress and Leeuwen, 1996–2006: 119), expects audience to actively take part in meaning construction in the past, feel the iciness and raise empathic concerns. In Figure 5, the visual unfolds the survivor’s wailing grief and lamentation for the serious pain and suffering of Chinese soldiers, and the linguistic modality complements the catastrophic frostbite or loss of hands, fingers, even hundreds frozen to death. These shots justify Chinese army paying a heavy price in fighting and demand viewers to remember their sacrifice and bravery. In Figure 6, snapshots (US3 and US4) revealing the linguistic modality ‘all of the toes are gone… And two or three of them were found inside of that boot’ presents cold injuries lying in the lower body, solely toes, rather than hands. In fact, poor logistics and interrupted resupply directly took a toll on a huge number of Chinese wounds and death while adequate personnel equipment, resupply of airplane support of American-led troops could limit the detrimental damage to foot frostbite.

Figure 5. CH3 and CH4 (source: post-battle snapshots from Frozen Chosin, 2011).

Literary translation of the left caption: ‘Some hands are missing’ Right: ‘Some fingers are missing’

Figure 6. US3 and US4 (source: snapshots from The Battle of Chosin, 2016).

Literary translation of the left caption: ‘All of the toes are gone’ Right: ‘And two or three of them were found inside of that boot’.

Table 7. Detailed annotation data of ‘after the battle’ stage.

	Number of annotations		Total annotation duration		Annotation duration percentage
Film Tier	CH	US	CH	US	CH	US
Narrative	8	36	112.53	433.71	1.976	6.359
Conceptual	9	11	124.389	148.47	2.184	2.177
Contact	17	51	237.249	577.06	4.166	8.461
Distance	25	51	234.68	577.19	4.121	8.463
Perspective	20	49	235.079	577.14	4.128	8.462
Information value	19	48	235.009	577.2	4.126	8.463
Salience	19	48	235.609	577.23	4.137	8.464
Framing	17	46	235.639	570.63	4.137	8.367
Linguistic	28	111	199.408	373.948	3.501	5.483

Table 8. Annotation spreadsheets of physical coldness after the battle.

4. Discussion

ELAN version 6.4 (Nijmegen, The Netherlands) is a powerful software that allows annotating and investigating the sensory experience of physical coldness through MDA. In two documentaries, we have observed how sensory coldness is differently constructed and communicated by means of representational, interactive, compositional and linguistic meanings. We can also witness visual elements, such as vector, contact, distance, perspective, salience and framing, are culturally molded to scaffold the frostbite and winter experience. All these features are closely bounded and reshaped by cultural and historical landscapes.

For instance, both filmmakers in three discourses present different sensory experience of coldness in diverse synergies. This is largely due to the historical fact that America has participated in WWI and WWII, gaining much experience in battling overseas. Secondly, the American-led UN troops are well equipped with advanced artillery, tanks, machine guns, personnel facilities and aerial resupply (Kirkland, 1995/96: 261). Hence, it is understandable that the American documentary mainly focuses on capturing foot injuries during and after the battle, while relatively few cases of frostbite were documented among the casualties. By contrast, China was only founded on 1 October 1949, just after the end of civil war. Absent of marine, air forces and inexperience in fight overseas, Chinese army was ill-prepared, badly supplied with food, winter clothing and weapons, lacking transport, warmth and air protection. ‘Tens of thousands of Chinese died with frostbite as lethal as American guns’ (Ford, 2021: 248). The relatively more visual and linguistic modalities of Chinese film holistically present huge non-combat wounds of ears, arms, hands, legs and even hundreds frozen to death.

A correlation of annotation density plot that shares sensory frostbite experience can also be found in Figure 7, indicating a discursive frostbite pattern. For example, in the introductory part, both employ a low density to demonstrate initial cold-proof preparations and soldiers’ valor. They did not expect the extreme iciness in the battle, believing in a speedy victory without much cold impact. It is also noted that the overall density of Chinese (L) is more divergent whereas American (R) shows a relative convergency. However, the data of pre-battle does indicate that both filmmakers prioritize the use of interactive and compositional techniques to tell the story of the pre-battle events, while the American film exhibits a lack of consistent complementary synergy between its visual and linguistic elements.

Figure 7. Annotation density plots (before, engagement and after), Chinese (L) and American (R).

After observing density variations in a systemic way, we begin to question the notion of indexicality in the storytelling of historical documentaries. For instance, in the engagement and post-battle, Chinese documented a great quantity of frostbite and frozen injuries in forepart and middle part, with tendentious ante displacement. This artistic treatment conveys intensely frozen experience to be dramatized under the backdrop of American fierce artillery attack, chronicling a worst-case scenario being disadvantaged from winter cold-proof materials, to poor weaponry then to the exposure of fatal heavy bombs from UN task forces.

In fact, the audience also experiences and recalls the sensory meaning of coldness and harrowing memories through the films. Based on user reviews collected from the IMBd database, the findings confirm that the use of uniformly distributed multimodal elements has effectively evoked empathetic feelings of sadness, for example: ‘…the program is almost all in black and white and shows still crisp photos, combat footage, and survivors’ comments in almost equal measure. It’s one of the most evocative and saddening documentaries on the misery of battle that I’ve watched’.⁶ Furthermore, the extensive use of interactive and compositional presentations in two films helps to engage the audience in a more emotive and diverse frozenness. Viewers find the subtle multimodal difference in illustrating coldness between two films: ‘…when comparing these two documentaries, our film has a clear narrative that emphasizes the military aspects of strategy and tactics. In contrast, the American film focuses on the human aspect of the war, with interviews of veterans expressing sympathy for enemy soldiers, prisoners of war, and civilians caught in the conflict. The documentary also reflects on the distortion of human nature caused by the brutality of war, a perspective that was overlooked in our film. For example, the American film presents shocking footage and images of frostbitten Chinese soldiers taken from prisoners of war, in addition to interviews with American veterans, while our film only features oral accounts from Chinese veterans’.⁷

Interestingly, if we venture into a wider territory of media culture, we can notice that filmmakers are endeavoring to relate this sensory meaning-making to their cold weather-themed movies, for example: The Last Winter (Larry Fessenden 2006), Whiteout (Dominic Sena 2009), Frozen (Adam Green 2010), Let It Snow (Stanislav Kapralov 2020) to prove the peril of freezing coldness.

5. Conclusion

In this article, we aim to showcase a synergic MDA framework to explore how sensory meaning-making captures physical coldness in two documentaries. Our analysis opens up a new path for comprehending how the experience is semiotically constructed and interculturally interpreted in actualizing the event in the films. In order to present a coherent frozen experience of the past, an authentic subtitle transcription and multimodal images are collected and annotated via ELAN version 6.4 focusing on structures, before the battle, engagement and after the battle. First of all, the results refute the fallacy that a historical documentary is ‘piling up traditional data’, instead, it aims to make sense of the past experience ‘as a site of the sublime, something which may be experienced in flashes but never explained’ (Rosenstone, 2012: 96). However, our MDA discloses that, as the title depicts, Chinese documents the historical scene with a macroscopic priority on cold experience, making clear their fight as a historical and strategic commemoration of collective sacrifice to safeguard the country after the founding of China against a formidable enemy and unfavorable frozenness. The sensory perspective is reflected by the integration of multimodal synergy, mostly, in the interactive and compositional ways, such as veteran’s contacts, dynamic distancing, holistic information route and framing techniques.

On the other hand, the Chinese film is moderately overshadowed by American in terms of overall visualizing strategies of the battle, because the latter, with superior cold-proof facilities and modern weaponry, as the title indicates, addressed the battle as its pivot, creating an unexpectedly disaster with more interactive and compositional meaning-making in coldness. The aftermath misery and war-weary complex are reinforced by emotional interviews with people in different backgrounds: civilians, historians, and authors.

In essence, the sensory actuality of coldness in historical documentaries is semiotically expressed per se and culturally correlated with filmmakers’ reenactments and intentions on what kind of lessons to learn, and what thematic empathies should be evoked. Nevertheless, the research shows us how disparate multimodal coldness are, extending the notion of filmic indexicality in documentary studies. We are confident that future development of sensory meaning-making in 4D films and interactive media can be inspired by incorporating interdisciplinary research with disciplines, such as intercultural communications, sociology and multimodal discourse analysis.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Zhenhai Sun

Zhenhai Sun is a Visiting Research Fellow at University of Bristol and an associate professor in Shanxi Institute of Science and Technology. His research focuses on intercultural communication, documentary studies and systemic functional interdisciplinary paradigm. He has been actively partaking in intercultural communication among diverse cultures for years. Sponsored by Shanxi Scholarship Council of China, he now carries out an independent project in the University of Bristol investigating how historical documentaries are potentially restructured through corpus-based MDA framework.

Muye Ma

Muye Ma, a senior lecturer, in Shanxi Institute of Science and Technology, has been working on the cultivation of intercultural communicative competence of college students since the beginning of her career as an English teacher. She adopts the convergence strategy of communication accommodation theory in communicating with her students. While in the teaching process, she emphasizes on the contrast and induction of both foreign and Chinese cultures besides the basic language points.

Notes

1 A collection of typical winter/snow/cold movies can be found at Winter/Snow/Cold Horror Movies – IMDb.

2 This synopsis of Frozen Chosin is collected on the popular and authoritative movie review website at https://movie.douban.com/subject/6129689/.

3 This synopsis of The Battle of Chosin is collected on Chinese popular and authoritative movie review website at https://movie.douban.com/subject/26996628/.

4 The rating can be viewed via ‘American Experience’ The Battle of Chosin (TV Episode 2016) – IMDb.

5 The summary of The Battle of Chosin given by Mike D. can be found at ‘American Experience’ The Battle of Chosin (TV Episode 2016) – Plot – IMDb.

6 Reviews on coldness and its lasting effects are available at ‘American Experience’ The Battle of Chosin (TV Episode 2016) – ‘American Experience’ The Battle of Chosin (TV Episode 2016) – User Reviews – IMDb.

7 This review is accessible at https://movie.douban.com/subject/6129689/.