Towards a unified affordance approach: searching for congruent meaning making in COVID-19 warning designs

ABSTRACT

This case study on COVID-19 warning designs in a Swedish context illustrates how a unified affordance approach may contribute to an understanding of the meaning-making in reminders, instructions, cues and prompts that communicate the message “keep your distance.” The analysis combines semiotic and ecological affordance categories, taking both Gibson’s original theorizing on affordances and more recent affordance-informed research efforts into consideration. In so doing, the study aims to bridge a knowledge gap in the study of visual instructions and warning designs as well as in a more comprehensive way delineate the multimodal design strategies associated with COVID-19 warning designs. The analysis shows that Swedish COVID-19 warning designs of the keep-your-distance-kind belong to a non-standardized and emerging genre that is marked by great variation and ad-hoc design solutions, several of which combine physical blocking functions with verbally based messages. The analysis also highlights the tension between verbal and visual recourses, on one hand, and the signage placement and choice of materials, on the other hand. It is concluded that communication resources do not always appear to convey the same basic message, but in incongruent ways weaken what might be considered the intended main message.

KEYWORDS:

• Affordance
• warning design
• multimodality
• instructions
• COVID-19

1. Introduction

Today’s Sweden, which is the setting for the present study, may be described as a low-risk society. Even so, a great deal of discussion has recently been centred on risks: in relation to forest fires in 2018, and their connection to the ongoing climate crisis, and the COVID-19 pandemic, with its global repercussions. Sweden may be seen as an example of what Giddens describes as the risk society (1998, 27), where communication about risk takes different forms: casual conversation, political discussions, instructions and warnings.

In this paper, we focus on risk communication in the form of signage, and in doing so connect to an understanding of our present-day society as increasingly visual. Ledin and Machin (2018, 7) discuss this development as the evolution of a new design culture where communication relies less on writing (cf. Kress and van Leeuwen 2001), in favour of a greater use of “a variety of semiotic materials” (2018, 8). Mitchell argues that pictures have “always been with us” (2005, xiv) as ways of world-making and thus crucial for acquiring knowledge about the world. As a consequence, it is vital to pay as much attention to pictures as to writing in attempting to understand the world, which is one way of interpreting the impact of the “pictorial turn” (Mitchell 2005, xiv). The present study takes the understanding of our society as a place where risk communication is pivotal and the understanding of communication as highly dependent on visual resources as a starting point for a discussion on risk communication design, using the rapid production of new risk communication emerging from the COVID-19 pandemic as a case in point. In Sweden, visual prompts, instructions, cues and reminders have formed a popular means to affect people’s potentially risky behaviour in connection with the pandemic. This type of public signage is in line with the national policy of avoiding lockdown by encouraging citizens to act responsibly through following simple guidelines such as “keep your distance” or “wash your hands”: messages which are easily distributed using simple visual cues and reminders.

There is no agreement yet as to whether the types of widespread visual reminders are effective. In fact, behavioural scientists have argued, in a newspaper article (Kristensson et al. 2020), that such visual interventions, or “warning designs,” have very little effect since the virus has continued to spread and that a better way to affect behaviour would be to obstruct or prohibit risky behaviour. This critique, however, is based on the assumption that Swedish COVID-19 signage is largely standardized, of a two-dimensional (2D) and word-based kind, typically displaying the phrase “keep your distance.” The scientists refer to a uniform design solution, which often takes the form of a sticker. As it turns out, this is a faulty assumption. Instead, our case study of the multimodal landscape of COVID-19 warning design clearly shows that this is a genre marked by variation.¹ In fact, many types of non-standardized interventions are neither 2D, nor word-based, but more akin to nudge-type interventions (Perry et al. 2015) in the sense that they are based on the idea that many decisions and ensuing behaviours that are information-prompted are automatic and not made consciously.

The question then arises, how may we understand and analyze this diverse, multimodal signage landscape and its characteristics? In this study we use the concept affordance to analyze the warning designs and in doing so also bring our understanding of the concept affordance itself to the centre of attention. Currently, our understanding of affordances is coloured by the fact that different research strands focus on two distinct aspects of affordances. There is the socio-semiotic strand, focusing on indirect perception affordances, and there is the ecological strand, focusing on direct perception affordances. We will argue that there are benefits in considering these perspectives jointly. Such a unified approach will be illustrated through a case study of COVID-19 warning designs as multimodal compositions.

1.1. Data and research questions

Instantiations of this new type of warning design are posted in businesses (shops and restaurants) across Sweden and often involve an instruction to customers to maintain their physical distance to each other. From our initial data of 150 signs collected in the spring of 2020, we could identify around 50 cases which express the message “keep your distance” (Swedish “håll avstånd”).² As there is considerable variation in what the signs look like, our definition of what constitutes a warning design is, consequently, quite broad. It includes different kinds of signage, sometimes placed on the floor (as the examples in Figure 1), in which case they often form round stickers with limited verbal material, carrying the main message that we need to remember to keep our distance. Sometimes they do not include words and pictograms, but only symbols directly sprayed onto surfaces, or they consist solely of adhesive tape on floors and furniture. In such instances, we regard the tape, or the spray paint, as a type of sign in and of itself, although such examples lack what is normally regarded a prerequisite for a sign, namely framing. Hence, some of the included signs lack a compositional “directive” (Kress and van Leeuwen 2006, 176). There is also a subtype with limited pictorial and extensive verbal elements, where “keep your distance” is one of several messages (e.g. Figure 6 below). Regardless of design, all of the samples can be described as a type of visual instruction that “warn us of potential dangers and indicate proscribed behaviour” (Mautner 2012, 190).

Figure 1. Examples of warning designs placed on the floor/ground.

In the examples in Figure 1, we see, from left to right, a floor sticker in combination with a demarcation line in a grocery store; distance markings sprayed onto the pavement in front of a food truck; a floor sticker in a grocery store; and a demarcation line in combination with a square sign in a home improvement store. In our data, there are elements that need to be interpreted, including visual instructions and verbal reminders, since such warning designs “bring to awareness latent knowledge” (Wogalter and Mayhorn 2017, 331). There are also physical elements, which may function very directly. Under certain conditions, they may result in an onlooker’s bodily, automatic, response taking place within a timespan of less than 200 m.s., i.e. before interpretative processing has been activated (Eriksson 2018). We will argue that the presumed impact of these hold-your-distance warning designs may be discussed based on affordance categories that are distinctly separate but interrelated, namely affordances in the realm of direct versus indirect perception.

We suggest in this paper that an instructional design may be “congruent,” if it adheres to the “principle of congruence” as per Tversky, Morrison and Betrancourt’s definition geared towards instructional designs that tend to be highly stylized and abstract (2002): “The structure and content of the external representation should correspond to the desired structure and content of the internal representation” (2002, 249). Our definition of “congruent design” is thus an extension of Tversky’s original mind-based perspective in that it also considers the human body, since some bodily responses require no thought. For example, if a sign’s volume, perceived density, material and situational placement “convey” the same message as its other relatively external elements (colour, printed words, framing, lines, pictograms, etc.), we regard it as congruent.³

With regards to the concept of instructional designs’ congruence, it needs to be noted that the perhaps most well-known instructional design principle, Mayer’s Multimedia Principle (2005; Clark and Mayer 2016), in some ways is rather analogous to the principle of congruence and our notion of “congruent design” in that it states that it is more effective to combine and mix verbal resources with pictorial ones than keeping them separate and letting them “speak” for themselves, which is often confusing (cf. Royce 2007). By combining verbal and pictorial resources, more working-memory capacity may be used for interpretative processing, rendering instructional signage easier to comprehend and decode (Swenberg and Eriksson 2018; Sweller, Ayres, and Kalyuga 2011).

The research questions are related to the notion “congruent design” and read as follows:

• How can the different understandings of affordances (“semiotic” and “ecological”) be applied in the analysis of signs? More specifically, how might the affordances of keep-your-distance signs be mapped in ways that draw on both semiotic (based on indirect perception) and ecological (based on direct perception) understandings?

• To what extent is “congruent design” between different semiotic materials present in the material? If so, how is this achieved?

2. Previous research

This section presents previous research underpinning our study. First, we delve more deeply into the concept of affordance and make the case for a unified approach. Next, we place the study in the field of risk communication, from the specific perspective of previous work on public signs and warning design. The final subsection connects the previous two by covering previous studies on warning design which have employed the notion of affordance.

2.1. The concept of affordance

Affordance was originally introduced by Gibson (1977, 1979), as part of his ecological psychology, primarily referring to the meaningful possibilities and constraints that humans and other animals perceive in their environment, standing on their legs, with their eyes oriented towards the horizon. He proposes the following definition: “the affordance of anything is a specific combination of the properties of its substance and its surfaces taken with reference to an animal” (1977, 67). Put differently, it is about what the environment provides the animal. For Gibson, it is fundamental that affordances can be perceived directly and are specified in the, normally, reflected light that reaches the animal’s eyes (Goldstein 1981). The ecological approach to perception, where affordance is a key concept, is developed further by E.J. Gibson and Pick (2000). They argue that affordance exists whether perceived or not, since it is an objective property. A reciprocity of perception and action is tied to affordance, so that perception guides action in accord with environmental supports or impediments. According to Evans et al. (2017), this relational and action promoting aspect is key when defining what an affordance is and is not.

Gibson’s definition is well-known, although not always held in high regard. What is less known is the link to Gibson’s work as lieutenant colonel in the US Air force in WWII, responsible for evaluating visual instructions for training fighter pilots (Reed 1988, 114–180). In part, this experience as “designer of instructions” led him, later, to believe that simple and direct reactions, rather than interpretation-informed reactions, guide human actions. This helps explain Gibson’s insistence that reflected light, the horizon and movement are integral to the affordance concept: it is how fighter pilots see, and act in, the world.

Affordance in socio-semiotic studies has been tied to the concept of mode. The difference between the modes of language (spoken and written) and that of image has been discussed by e.g. Kress (2003, 2) who states, “whether I want to or not I have to use the possibilities given to me by a mode of representation to make my meaning. For Kress (2010), affordance is a useful starting point for discussing the semiotic implications of transitions between semiotic modes, such as writing and images. The affordance of a mode, then, is not just an objective property; it is something that evolves over time, as the mode is used and interpreted and optimized to fulfil specific communicative goals. As an example, Ledin (2015) has studied how lists and tables, as semiotic formats, have evolved over time to meet needs of categorization and classification. From the original visual perception of an animal, the concept has been extended to include anything that is inviting or obstructing when one specific mode is chosen over another.

Affordance today is a widely used theoretical concept in social semiotics (Kress 2010), literacy studies and critical multimodal discourse analysis (CMDA). Ledin and Machin (2016) use affordance as a concept in what they call a “discourse-design approach to multimodality” (2016, 1). Their understanding of affordance is influenced by Voloshinov (1973), who stressed the material basis of communication. Machin (2016) further points to the importance of affordance analysis in the development of the broader field of multimodal critical discourse studies (MCDA): For any semiotic resource, analysing affordance means asking what it has evolved to do.

As the warning designs studied in this paper make up multimodal compositions, affordance is a given analytical category. Our unified approach involves the identification of aspects of multimodal compositions that are typical examples of representational elements and “visual grammar,” such as framing, printed words, fonts, colour, texture, including a discussion on their ideological underpinnings, socio-cultural and situational implications. But it also involves aspects such as objects’ placement, anonymity, negative space, malleability, perceived weight/lightness, durability, visibility. The field of multimodality studies is increasingly showing an interest in such aspects, like the semiotics of space or place (cf Stenglin 2009; Björkvall, van Meerbergen, and Westberg 2020) and we wish to see our study as a continuation of that development. According to Kress and van Leeuwen, certain material properties and phenomena are not easily captured when using terminologies and concepts that are intimately connected to representation (2006, 168–171). Hence, similar to Evans et al. (2017), we suggest that important material properties of communication objects that are not intimately related to representational techniques may be illuminated and defined using criteria that are more closely aligned with Gibson’s original conception and application of affordance.

2.2. Public signs and warning design

The present object of study is public warning designs found in businesses. By highlighting the role of information design in pandemic times, we support the widely held view that the choice of information design in disease prevention contexts influences the degree to which information is engaging or persuasive (Perry et al. 2015). Warning designs reflect the ethos-driven concern to inform the public of risky conditions (e.g. Frascara 2015). A distinction often made in a linguistic landscaping framework is that between top-down and bottom-up signs, where the former refers to “official signs subject to government and policy regulations” and the latter to “unofficial signs posted by individuals” (Gorter 2006, 3). Our material does not fall neatly into either category, but is informed by top-down regulations (from the Public Health Agency of Sweden), while at the same time being produced and posted by business owners. These warning designs target the general public, but do not conform to typical “risk communication” in that they are not communicated by an expert (cf. definitions from WHO https://www.who.int/news-room/q-a-detail/risk-communication-frequently-asked-questions). Serving essentially as reminders of social distancing, they appear to be produced by non-experts on risk communication, and most typically by local business owners.

Signs in the sense of a genre of public notices have been categorized into types based on function. A widely cited functional taxonomy by Scollon and Wong Scollon (2003) includes informational (e.g. displaying the names of buildings and organizations housed in them), promotional (e.g. advertising billboards) and directive signs (e.g. no parking signs). In addition to these, the symbolic and commemorative functions of signs are also mentioned in the literature (e.g. Landry and Bourhis 1997). The primary function of the material collected for the present study is directive. Directive signs have been characterized as “warn[ing] us of potential dangers and indicat[ing] proscribed behaviour” (Mautner 2012, 190). The term “regulatory sign” is also used in the literature to foreground the function “direct[ing] the actions and behaviours of […] social actors” (Dressler 2015, 131). Functions are foregrounded also in the warning design tradition, where warnings are defined as “a type of risk communication intended to give people information about potential hazards and instructions to promote safe behaviour” (Wogalter and Mayhorn 2017, 331). What is especially relevant here is that warnings are said to also potentially “serve as a reminder to cue access to existing knowledge” (331).

The polysemy of the term “sign” in the study of language and semiotics sometimes makes it difficult to deal with. Given its different meanings, it is not difficult to see why the alternative term “signage” may be preferable. “Signage” has been widely adopted in linguistic landscaping, which more so than any other area of study has placed signage at centre stage. The linguistic landscaping approach is cent\red on multilingualism and language choice, and hence primarily based on verbal aspects of signage. More recently, however, there have been efforts to extend the object of study to other semiotic elements, such as “colours, images, other visuals and the materials of the signs […] or the characteristics of text type and font” (Gorter 2014, 11).

When we consider classifications of signs based on form, it becomes clear that signs are complex and highly varied in their realizations. For example, given the rate of technological innovation, the inventory of types of physical appearance of signs is expanding rapidly beyond static or fixed inscriptions. Among relatively recent types, we find electronic flat-panel displays, LED neon lights, foam boards, interactive touch screens and scrolling banners (e.g. Gorter 2013). Such variability contributes to the challenge of finding a suitable definition. We see the three terms – sign, signage and warning design – as forming a scale of general to specific labels for our object of study. We use them somewhat interchangeably in the paper, but see “warning design” as providing the most precise definition. We await clarifications regarding definitions of “sign” and “signage” in future research.

2.2.1. Warning designs and affordance

We have identified only a few studies on warning designs that employ affordance theorizing in some way. Forceville and Kjeldsen (2018) categorize traffic signs into warning signs, prohibitive signs, instructive signs and pretend traffic signs. The last category represents an unusual subgenre that is playful and/or ironic. They investigate the rhetorical and argumentative potentials of traffic signs and how the communication context conditions meaning. It is argued that, despite the fact that argumentation scholars doubt that visuals may be overtly argumentative (see Patterson 2010), the meaning potentials of visuals are strengthened if such multimodal texts adhere to specific genre conventions, are highly coded and, therefore, contain pictograms.

Warning designs from the perspective of pictograms are discussed by information design scholars Wogalter and Mayhorn (2017). They focus on types that are highly codified and adhere to a research-based standard (ANSI Z535). Their discussion about warning design as a method of hazard control revolves around the so-called Communication-Human Information Processing model (C-HIP). The model is essentially based on a sender-receiver communication model and may be used to assess how a multimodal warning is successful or not. The C-HIP model is said to be a useful tool in helping researchers and designers to identify processing bottlenecks, aiming to eliminate some of the onlooker’s cognitive load while trying to grasp the message.

In a study of evacuation signs, based on an affordance-informed questionnaire design, Olander et al. (2017) argue that warning designs that clearly display dissuasive messages through brightly coloured signalling (red X-marking) are most effective. To be effective, the signaling must “break the tendencies of normalcy” (2017, 84). Their take on affordance is that affordances are essentially outcomes. Based on Hartson (2003), they assume that the assessment of what warning designs offer people can be categorized into four main affordance types: (1) sensory, (2) cognitive, (3) physical and (4) functional. This approach, they argue, allows researchers to examine the success factors of dissuasive signage that are part of a highly standardized communication (and evacuation) system.

3. Data and method

The data collection forming the base for this paper consists of warning designs, collected in small towns and rural areas in Sweden during May and June 2020. The signs and their surrounding contexts were photographed using smartphones. When delimited to keep-your-distance types, the material comprises 51 signs.

The analysis involves two steps, focussing on indirect and direct affordances, respectively. In the analysis of the indirect affordances, the overarching question is to what extent the meaning-making is congruent. In this part of the analysis, we use categories from the Systemic Functional framework (Halliday and Matthiesen 2014; Kress and van Leeuwen 2006). A social semiotic perspective sees language as functional, semantic, contextual and semiotic, and the systemic functional framework specifies two basic functions of language: “making sense of our experience, and acting out our social relationships” (Halliday and Matthiesen 2014, 30). Further, a third function relates to the construction of text, and has an enabling or organizing function. Usually, these three functions are referred to as metafunctions: the ideational, the interpersonal and the textual. Originally, these metafunctions were developed in the study of verbal language, or the written and speaking modes, but as the field of multimodal studies has evolved, meaning-making using different semiotic resources has been studied from this perspective, first centring on pictures and images (Kress and van Leeuwen 2006) and later on phenomena such as buildings (Ravelli and McMurtrie 2016) and digital touch communication (Jewitt and Leder Mackley 2018).

The key categories for the semiotic analysis are the following:

• Ideational categories are processes, participants and circumstances which realize our experience of the world. In the verbal semiosis there is little variation in the data, since they have been delimited to keep-your-distance signs but processes of different types – such as actions and reactions – are also realized visually.

• Interpersonal categories are perhaps the most interesting in this set of data. They concern the way the signs form a relation to the reader/viewer. The signs produce a speech act, most often a directive or a statement, and this speech act may be emphasized or mitigated by means of visual resources, such as salience or framing.

• Textual categories have to do with the composition of the signs. Verbal and visual resources in interplay create an ensemble of connected or disconnected elements which is one aspect of what we refer to as congruent design. From the perspective of the congruence between verbal and visual resources, composition is important for the analysis, especially in relation to the visual category centre–margin, which indicates what ideational content is in focus in the signs.

The analysis focusing on direct affordances employs the conceptual framework for understanding affordances proposed by Evans et al. (2017, 39). Their threshold criteria for substantiating purported affordances stipulate that what may be defined as an affordance is not (1) an object or a feature of an object or (2) an outcome. In addition, it needs to have (3) variability. For example, visibility is an affordance since it is not a mere feature or an object, but something more fundamental and relationally construed. It is something that undoubtedly promotes and supports various actions. Nor is visibility a clear-cut outcome, but normally a prerequisite for information seeking and sharing. Moreover, visibility certainly has variability. To what degree something is visible is a critical topic in any research on designed objects’ visual properties, since the variability of visibility in unambiguous ways impacts user behaviour. Another affordance is anonymity; it is not a feature, nor an outcome, and it has variability. Low degrees of visibility in a warning design’s compositional elements, normally results in high degrees of anonymity with regards to the overall spatial setting.

4. A unified affordance approach to warning designs

In what follows, we present a selection of signs serving to illustrate the characteristics of the emerging COVID-19 warning design genre. We have selected six different examples that make it possible for us to illustrate the variability found in the data. In each instance, we show what a unified affordance approach may reveal about the warning design. Figure 2 shows the first example, involving a demarcation sign in a grocery store. It is placed at the end of the cashier, where customers pack their groceries. There is room for two customers simultaneously, and the demarcation is intended to separate the two adjoining areas. Two stretches of text are displayed. The upper text, in block letters, reads “Keep distance when you pack your purchases” (Håll avstånd när du packar dina varor), while the lower text, in mock handwriting, reads “Thanks for your concern!” (Tack för din omtanke). At the bottom of the board, the third stretch of text involves the logo of the supermarket.

Figure 2. Demarcation sign (grocery store).

If we start with the verbal components, we can note that the upper text (“Keep distance when you pack your purchases”) functions interpersonally as a command (Halliday and Matthiesen 2014, 136) with an undertaking of that command as the preferred response. The finite verb of the speech act, håll (“keep”), realizes a material process (2014, 214) connected to actions in the physical world, forming a strong expectation of physical action from the recipient. The command is presented in a congruent way.

The following text, “Thanks for your concern, constitutes not a clause, but a noun phrase. In terms of speech functions and turn-taking, this type of phrase is typical for a response to an offer or a service, which may be verbal or physical. The sign presupposes that the required action in the previous text has been performed and construes a thank you to the reader for having done so. Together, the two segments form part of a dialogue, where one turn–the undertaking of the command–is implicit. Interpersonally, the sign has a relatively uncommon feature in the use of the second-person singular du (“you”) in the first verbal segment. Since the pronoun in the direct address is tied to the circumstantial element (Halliday and Matthiesen 2014, 222) “when you pack your purchases,” it avoids making the command face-threatening.

Visually, the two segments are different in terms of not only speech acts or turntaking roles, but also writing. The first segment is in bold, round typeface, in line with the supermarket’s graphical profile, while the second may be seen as more salient, due to the typeface, chosen to resemble handwriting. The mock handwriting underlines the interpersonal function of this segment: the primary function is to establish a friendly tone in the communication. Verbal and visual resources work together here, with the initial “Thank you” and the softer design of the handwriting.

The third written segment is the supermarket’s logo, placed at the bottom of the sign. It creates a possibility to associate the concern and social responsibility expressed in the sign to the sender, which may increase the business’s social capital.

The composition can be seen as a flow from top to bottom, where the “dialogue” in the upper part can be seen as a more general message and the sender’s logo at the bottom as more specific (Kress and van Leeuwen 2006, 190). We may note that this does not necessarily apply universally, but there may be cultural variation in reading preferences being for example vertical or horizontal.

Concerning the sign’s ecological affordances, it is striking how visible it is. This is partly explained by the overall setting’s (floor, walls, ceilings and conveyor belts) low-contrast values and “greyness” that indirectly make the sign’s sharp outline and distinct green colour stand out (e.g. Olander et al. 2017). It is also explained by the cardboard sign’s flatness that, simply put, reflects light in completely different ways from the negative space immediately surrounding it and the objects far behind it. This also illustrates another ecological affordance: anonymity, which in this context refers to how easily this sign as a complete, visual, object is identifiable in the overall grocery store setting. It is important to note that this particular sign is different from most keep-your-distance signs as it is placed in a highly visible space: in front of, and very close to, a presumed customer’s torso and head. This strategic placement further leverages its visibility. This also makes it reachable (and touchable), which is yet another affordance (Carello, Wagman, and Turvey 2005). The “reachability” in combination with the sign’s solidity and firmness (characteristics that do not fully qualify as affordances) explain its blocking function and the way it obstructs passage. Consequently, this cardboard sign is essentially a screen that hinders movement. That said, depending on from what angle a customer interacts with it, it also affords some moveability. The fact that it is possible to perceive its edges, which renders the object 3-D, might reveal its true nature of being made of air-filled, lightweight cardboard. This potentially lessens the blocking function’s potency.

In summary, the Figure 2 sign offers a multimodal ensemble where verbal, visual and physical resources construct an instruction in the form of a sign that is a screen, which is probably effective in hindering close proximity between customers. This is, then, a good example of alignment between ecological affordances that directly hinder unwanted behaviour, and the semiotic affordances that aim to promote expected behaviour by engaging people mentally. There is only a slight mismatch in this example between semiotic and ecological affordances. The printed words stress keep-your-distance, but the sign’s ontology and placement imply something a little different, namely obstruction. Blocking people, and people’s unwanted viruses (on our bodies and in our exhaled air) is not quite the same thing as telling people to keep their distance. That said, in the COVID-19 disease prevention context, both objectives and their associated cautionary appeals seem to work in tandem.

Figure 3 shows a type of sign which is frequent in our data: a round sticker on the floor. The main message is “Think about the distance” (Tänk på avståndet), in bold at the top. The lexicogrammar, with the noun in the definite, clearly marks it as a reminder: the reader is expected to know what distance is referred to. Ideationally, then, the sign concerns distance, where the choice of verb underlines its character of reminder. The verb “keep” (håll) in Figure 2 above realizes a material process in the physical world. By contrast, Figure 3’s “think about” (tänk på) realizes a mental process, giving the sign a slightly different focus.⁴ Also, the fact that the reminder lacks circumstantial participants makes it less concrete compared to Figure 2.

Figure 3. Floor sticker serving as a reminder (grocery store).

Interpersonally, the Figure 3 sign is less face-threatening than Figure 2. While they both form commands, the undertaking of a command that asks you to think about something is less taxing than a directive to perform an action.

Like Figure 2, Figure 3 also includes a second stretch of text, which reads “Shop with consideration” (Handla med hänsyn) and functions grammatically as a command, involving a material process (“shop”), but with the circumstantial attribute as the most salient part, “with consideration.” Since shopping is a given activity in the shop, this instruction centres on behaviour. Not only does the word choice express consideration, but there is also a heart attached to the text. Through the heart, the socially facilitating text also contributes to branding the supermarket chain. The font and colouring are in line with the company’s graphical profile, where red heart shapes occur quite frequently, and the phrase “Shop with consideration” (Handla med hänsyn) is vaguely familiar as the company’s slogan.

Between the two texts, there is a pictogram of two feet. From the perspective of centre–margin composition, the pictogram could be considered the most prominent part of the sticker, contributing to the invitation to physical action which is not voiced in the verbal elements. The fact that the floor sticker is combined with a demarcation line, a wide type of coloured tape, possibly ensures a certain alignment between ecologic and semiotic affordance. The line potentially affords stopping or hindering movement while the round sticker underlines this message by inviting interpretation. This is also pointing out that the element in this composition that is most readily discussed in terms of direct perception is this line since it is high in visibility and has rather low anonymity considering this grocery store’s overall setting and colour scheme. The pictogram of feet represents a place where customers may place their feet and stand still, although it is unlikely that these feet-shapes afford actual feet (with shoes on) on top of them (waiting to move), since this is clearly just that: a representation. In other words, viewed through the ecological perception-lens, the feet-shapes are associated with low reliability. However, since humans stand on two legs, the line and sticker are easily missed, even if the line and sticker do not seem to suffer from low visibility.

In Figure 4, we see a subgenre of the keep-your-distance signs, where humour is used to soften the command expressed in the verbal elements. There are only two words: “Keep distance” (håll avstånd), forming a command realizing a material process, without any softening circumstantial element. Verbally, this is potentially quite face-threatening. Interpersonally, however, the command is mitigated by the pictogram, showing two human figures – a male and a female – and between them a moose. The moose is unrelated to the business domain (a coffee shop) and the area is not close to wildlife, so the pictogram’s relevance is unclear. One possible interpretation is that humans are invited to stand so far apart so that something quite large, such as a moose, may fit between them.

Figure 4. Mock road sign outside coffee shop.

The composition with the verbal elements as a headline above the pictogram presents the visual element as the main content. The use of a traffic sign, where the colours yellow and red in the Swedish context signal a prohibition, marks that the message should be taken seriously. And yet, it falls into the category of a playful “pretend traffic sign” (Forceville and Kjeldsen 2018). In the Swedish context, road signs and moose go together in the sense that road signs warning for moose are relatively common in some parts of the country; however, the moose pictogram is not the prototypical one used for authentic road signs (it appears to be an American and not a European moose).

This sign is associated with similar ecological affordances as that in Figure 2, therefore it is high in visibility. However, this sign is considerably more solid and heavy, and protrudes from a grey (concrete) stand that certainly has low moveability. The sign’s blocking and obstructing function is therefore quite pronounced. In comparison with Figure 2, this sign, as already noted, displays pictograms that are meant to communicate keep-your-distance. However, in terms of ecological affordances, the pictograms as such do not trigger automatic distance-promoting behaviour. However, the question whether pictograms have ecological validity or not is not easy to answer. On the one hand, the human shape pictogram obviously resembles human-like forms and human animals are presumably “pre-programmed” to automatically react to things that appear to be fundamentally human, including objects that are abstract (Anderson and Hodgins 2005). Gestures and facial expressions that are comic-strip-like are the most clear-cut examples of this (Tan 2005). On the other hand, here it seems unlikely that the pictograms contain enough information to trigger an automatic bodily response. If they did, the resulting (mirror-neuron promoted) action would most likely be “standing.” If so, the pictograms would be associated with the rudimentary affordance of comparability (Eriksson and Eriksson 2019). If the pictograms had been less standardized and featured a more distinct posture or implied movement, they may have been associated with the ecological affordances of reliability or immediacy and potentially triggered a more purposeful action (e.g. Anderson and Hodgins 2005; Eriksson 2018).

In summary, in terms of semiotic and ecological affordances, we suggest that this sign is an example of an incongruent design strategy since what is communicated verbally and visually is not emphasized or/and exemplified by the physical aspects of the sign and its situatedness. It is somewhat unclear what distance-keeping really has to do with obstructing movement outside a coffee shop, even if it is supposed to mark a boundary where customers are meant to queue.

Figures 5 and 6 show two very different warning designs aiming to make people keep their distance. Figure 5 shows an ordinary white A4 paper taped to a sign display easel, placed outside a flower shop entrance. The message is almost exclusively verbal, except the heart at the bottom, which is part of the ending, together with the phrase “Thank you for visiting us” (Tack för att du besöker oss). The keep-your-distance part of the message “Keep your distance to each other” (Håll avstånd till varandra) occurs in line 6 of 13, and is voiced in other words in line 5, “avoid queuing and crowding” (Undvik köbildning och trängsel), so approximately in the middle of several different types of information. At the other extreme, we find Figure 6 which does not draw on any verbal resources at all. Two pieces of thick adhesive tape have been placed across the table and sofas of a restaurant booth, preventing customers from sitting there. This particular adhesive tape is a clear-cut example of a resource, and material object, that definitely may be considered a “Gibsonian affordance.” The tape-design affords a kind of anti-action, that is the opposite of sitting down (perhaps standing up or passing by). Yet, the actual tape would be associated with the ecological affordance of malleability, since it is not rigid or/and stiff. In theory, then, this potentially weakens its function as a barrier and an obstacle that prevents access. Rather that voicing the message “keep the distance” the tape hinders sitting down, and in doing so exemplifies the type of obstruction of faulty behaviour that the previously mentioned researchers (Kristensson et al. 2020) would prefer to nudging and recommendations.

Figure 5. Signage with extreme emphasis on verbal communication.

Figure 6. Signage with extreme emphasis on non-verbal communication.

Our last example is the mannequin in Figure 7, wearing a yellow reflective vest, with two commands printed in grey reflective letters in front. The first command “Think about keeping the distance to each other” (Tänk på att hålla avståndet till varandra) includes the definite form of the noun distance, marking it a reminder, as in Figure 3. The second command “Show consideration” (Visa hänsyn) is potentially face-threatening in its directness. The choice of the greeting “Hello!” (Hej!) in the top sign, outside of the main text, may serve to mitigate the direct force of the commands. The verbal processes represented here are a mix of those in and Figure 3, with “Think about” (Tänk på) realizing a mental process and “show” (visa), realizing a material process. Seen from the perspective of turn-taking, the mock interaction opens with a greeting, setting up a strong expectation that customers approaching the human shape warning design will in some way respond to an interpersonal call, before the ensuing exchange involving face-threatening commands.

Figure 7. Mannequin in supermarket.

The mannequin is dressed in the staff uniform of the supermarket and is in this way presented as a representative of the store. Its lacking a head may be seen as unimportant from the perspective of representing a universal or anonymous employee or manager (even if it may be disconcerting with respect to direct perception). The connotations of the reflective vest involve danger and safety.

The bright-coloured vest and the greeting sign notwithstanding, the mannequin is not very high in visibility. Nor is it very low in anonymity. This is explained by its tendency to blend into the store’s overall setting and colour scheme (white, beige and red dominate), as in Figure 3. This is certainly the case when perceived from a distance. Moreover, the mannequin is low in reliability and a good example of when something is seemingly human-like, but really is not (e.g. Anderson and Hodgins 2005). However, it is not the missing head that makes it uncanny (it would have been eerier with a head), but, we believe, its unnatural, frozen gesture and stiff, thin, limbs. The ecological affordance associated with the displayed gesture possibly invites stopping or standing. Still, it seems that the gesture is not clearly articulated, but, instead, resembles some other nondescript gesture. The gesture in itself thus offers low comparability, not presenting one from the standard human repertoire. Finally, the warning design is low in recordability – yet another affordance – since it is highly unique and, therefore, forms a very complex communication object.

Overall, the complex and incongruent design can be said to background the main hold-your-distance message and cause confusion. The branding aspect is strong here, with a seemingly clear desire on the part of the supermarket to increase their social capital. Figure 7 may in this sense be said to show how companies may use the pandemic for marketing efforts. Several examples from our data include some type of marker of location, which can be in the form of a company logo or even a staff uniform. The inclusion of these illustrates the occasionally fine balance between signaling social responsibility and taking an opportunity for branding. There are parallels to the Corporate Social Responsibility Report (e.g. Jaworska and Nanda 2018), where research has shown the responsibility framing being part of the building of trust and a positive corporate identity, where the ultimate goal is increased market shares.

5. Discussion

Through a selection of COVID-19 warning designs, we have attempted to illustrate how a basic analysis may be carried out when using affordance as a primary analytical tool. We believe that warning design signage may be analyzed applying ecological affordance theory (Gibson 1977, 1979) in combination with affordance theorizing popularized by Norman (1988), Kress (2010) and other design and multimodality scholars (Machin 2016) in order to illuminate what constitutes congruent, instructional keep-your-distance signage. In so doing, this paper aims to bridge a gap in the study of instruction and warning design. Given the high-stakes, risk communication nature of the material, it could be argued that achieving “congruent design” is especially important.

In our case study, the focus has been on the message “keep the distance.” It is particularly important in relation to this message to observe the physical aspects of warning design: to what extent the signs actually obstruct physical contact. In exploring this aspect, it has been useful to apply an ecologically based concept of affordance, with its understanding of affordance as a phenomenon which can be experimentally tested and determined, as for example anonymity. This understanding of affordance differs from the understanding in multimodality and social semiotic studies where modes or semiotic resources are foregrounded, and where it is possible to talk about a specific mode as for instance “affording multidimensionality or discursivity.” In future studies, this opposition between two different understandings of the concept affordance needs to be explored further.

Affordance-informed research is steadily increasing (Evans et al. 2017; Rice et al. 2017). Still, different kinds of gap remain between multimodality and ecological, biological and cognitivist considerations with respect to how the term affordance is used by scholars to analyze and understand how multimodal expressions really function. Also, more to the point, in communication research in general there are ample inconsistencies regarding how the term is used and what it connotes (Evans et al. 2017). This has resulted in a situation where scholars to some extent use affordance theory in a different way from how it developed originally. From a Gibsonian perspective it might be argued that affordance theory is good at conceptually defining individual affordances. In other approaches, scholars define mere features or outcomes of multimodal expressions, thus forsaking the key relational action-promoting phenomenon in an ecology that Gibson pioneered (cf. Evans et al. 2017). Studying keep-your-distance floor-stickers through a Gibsonian lens, it is easy to doubt their validity. Such stickers, perceived from a stand-still, perhaps waiting in line, appearing badly lit on the floor, are the opposite from being close to the horizon; they are extremely low in visibility and high in anonymity. This may partly explain the COVID-19 warning designs’ alleged lack of efficacy.

It is in no way argued in this paper that the original affordance definition is superior to the more recent semiotic one. Nearly all information needs to be interpreted. If an instructional sign’s primary message cannot be successfully interpreted, its key message is most likely lost. Instead, this is a matter of distinctly different affordance categories – ecological and semiotic ones – in the best of circumstances complementing each other and reinforcing the key message. In other words, we suggest that the issue is not really whether ecological or semiotic affordances are more important in a specific communication context, but that “incongruent designs” become incongruent in part due to stimuli-triggered processes that are automatic (cognitive scientists mostly discuss these in terms of bottom-up processes), cancelling out or disrupting interpretation (top-down processes) (Swenberg and Eriksson 2018). In such cases the main message becomes unclear or confusing: the mannequin example in Figure 7 is a case in point.

The main point is that, if we as multimodality scholars combine the original affordance theorizing with the semiotic approach, in more stringent and coherent ways, we would not only promote theoretical developments, but we may also be more likely to approach the “holy grail” of research into visual instructions: their efficacies. Visual instructions’ efficacies, we suggest, may only be sufficiently identified if scholars of visual instructions and warning designs fully consider the tenacious and complex relationship between interpretative processing that is mind-based, on one hand, and reactions that, on the other hand, take place in our bodies, thereby acknowledging that humans are mind and body combined. On this note, it would be worthwhile to develop a comprehensive analytical model that offers a highly operational and coherent way of combining ecological and semiotic affordances in empirically-based research on multimodal compositions. This would prove useful for researchers and designers of warning designs alike.

The analysis we have presented has methodological implications. When taking ecological affordances into consideration, aspects such as space and human movement are key considerations. This means that documenting the surrounding context of, for example, a warning design, becomes important in order to understand more fully its affordances. The relevant data for research is not just the object itself, but also its immediate context. The issue of what a photograph really is evidence of in the context of empirical research on visual objects in real-life scenarios deserves further discussion (cf. Bremner and Roxburgh 2015).

Acknowledgements

We gratefully acknowledge support from Dalarna University’s 2020 funding for interdisciplinary research networks.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Högskolan Dalarna.

Notes on contributors

Per Erik Ekwall

Per Erik Ekwall received his PhD in Innovation and Design from Mälardalen University, Sweden and is currently faculty director of the Department of Media, Dalarna University, Sweden. His research focus is visual instructions, learners’ cognitive capacities and visual literacy. He has a background as TV producer/documentary filmmaker and received his MA in Communication from Stanford University, USA.

Annelie Ädel

Annelie Ädel received her PhD in English linguistics from Gothenburg University, Sweden, and is currently full professor at Dalarna University, Sweden. Her research interests include discourse analysis, corpus linguistics and English for Specific Purposes. She has been affiliated with the University of Michigan’s English Language Institute as director of Applied Corpus Linguistics and with Stockholm University as a research fellow.

Catharina Nyström Höög

Catharina Nyström Höög received her PhD in Scandinavian Languages from Uppsala University, Sweden, and is currently a senior lecturer at Stockholm University, Department for Swedish Language and Multilingualism. Her research interests include organizational discourse, genre and discourse analysis, plain language, and text linguistics. She has been affiliated with Dalarna University.

Notes

1 The material was collected at the beginning of the pandemic, during the first wave in 2020.

2 Other signs collected carried main messages such as “wash your hands,” “use hand sanitizer,” “do not enter the premises if you have cold symptoms,” or they provided an explanation for why selected tables in a restaurant could not be used.

3 In the systemic-functional framework (see further Section 3), congruence usually refers to interstratal relationship, which is not related to the congruence discussed in this paper. Cf Royce (2007), who discusses intersemiotic complementarity as one way to address congruence between modalities, while we address congruence between different types of perception.

4 Regarding the use of “think” (tänk) in Figures 3 and 7, we may note that it is potentially problematic to use a verb that is highly polysemous, even if it is used specifically as a prepositional verb “think about” (tänk på). One dictionary definition of “tänk på” is “consider”, as in “consider the consequences” or “mind your behaviour”, which indirectly links it to actions in the physical world.