Introduction: the semiotic repertoire: assemblages and evaluation of resources

KEYWORDS:

• repertoire
• translanguaging
• assemblage
• affect
• language mixing
• plurilingualism

‘Semiotic repertoire’: initial use

Recent years have seen a revitalisation in the use of Gumperz and Hymes’ notion of the verbal repertoire, meaning ‘the totality of linguistic resources (i.e. including both invariant forms and variables) available to members of particular communities' (1972, pp. 20–21). The resurgent use of the concept ‘repertoire' is largely due to shifts in conceptual understandings of language use and goes paired with the expansion from concepts such as ‘bilingualism' and ‘code-switching' to concepts such as ‘translanguaging'. Translanguaging theory is predicated on the notion that languages are not bounded systems, and that features of different named languages can flexibly and fluidly be used together and blended into each other (Garcia & Wei, 2014). In this context, many scholars have used the term ‘repertoire’ to talk about these features as a collection of resources that people deploy in their interactions. This special issue engages with the semiotic repertoire, which is the totality of semiotic resources that people use when they communicate (such as speech, image, text, gesture, sign, gaze, facial expression, posture, objects and so on) (Kusters et al., 2017b).

The concept of the ‘semiotic repertoire’ was introduced in an earlier special issue in this journal (Vol 14, Issue 3) as a specific response to research in translanguaging that has foregrounded translanguaging as fluidly blending (features of) languages and has overlooked or downplayed the role of other resources in this practice (such as gestures, objects, and images). To be sure, a number of scholars of translanguaging have acknowledged that communication is multimodal and involves diverse resources beyond words (Canagarajah, 2013; Garcia & Wei, 2014). For example, Canagarajah (2013, p. 6) defined ‘translingual practice’ as communication that ‘transcends individual languages [and] words and involves diverse semiotic resources and ecological affordances’. As researchers of multimodality have emphasised, ‘linguistic’ resources (such as words from ‘bounded’ languages) may not necessarily carry most weight in interactions. In a given interaction, an image, a gesture or sound may be more salient than e.g. spoken or written words. Therefore, Kusters et al. (2017b) saw the use of the term ‘linguistic repertoire’ (e.g. Busch, 2012; Gumperz, 1982) as symptomatic for (or leading to) a narrow focus on the use of linguistic codes in translingual practice.

When suggesting the use of the concept ‘semiotic repertoire’ instead, Kusters et al. (2017b) were not primarily concerned with a definition of the semiotic repertoire. Instead, the ‘semiotic repertoire’ was framed as a conceptual lens:

The notion ‘semiotic repertoire’ departs from the idea that languages are bounded systems (an understanding that is central to translanguaging theory); and that repertoires are merely linguistic (they are multimodal and embodied). (…) [T] he lens of semiotic repertoires enables a holistic focus (addressing ideologies, histories, potential and constraints) on action that is both multilingual and multimodal. (Kusters et al., 2017b, pp. 10–11)

Kusters et al. (2017b) argued that to come to a better understanding of translingual practice, we need to pay more attention to manual forms and dimensions of communication; which had been broadly studied in studies on multimodality but underresearched in studies of multilingualism. Key to the promotion of the term ‘semiotic repertoire’ by Kusters et al. (2017b) was to bring together signed and spoken language linguistics, multimodality studies, and gesture studies. Scholars studying signs and gestures have investigated how modalities and languages can be expressed alternately or simultaneously, such as people gesturing and speaking; or signers voicing/mouthing and signing. Signers’ communicative practices form a crucial case in point to study semiotic repertoires: signers habitually and aptly blend different modalities (e.g. features of spoken/written language such as mouthing and fingerspelling) with signing. Using the notion of the ‘semiotic repertoire’ pushes this understanding of translingual practice as multimodal to the forefront. In research practice, this means attending to the role of e.g. speech, signs, mouthings, gestures, images, smells, and objects in interactions; studying how these different resources make meaning in specific constellations. Different resources may play different roles in different interactions and exist in (hierarchic) constellations specific to those interactions or contexts. Below, the term ‘assemblage’ is used to talk about these constellations.

Summarised, Kusters et al. (2017b) presented the concept of ‘semiotic repertoire’ as a response to limitations of the concept ‘linguistic repertoire’ as used in translanguaging research, primarily treated it as a lens rather than as a theoretical innovation, and as a tool to bring together different fields of language sciences. The current special issue takes the uses of the concept further in the sense that authors engage in an exercise to define what the semiotic repertoire is and does and theorise how the concept should be understood and used in research. There are several connecting threads that run through the set of papers, which are introduced below.

The distributed semiotic repertoire: the ‘semiotic assemblage'

The semiotic repertoire as approached by Kusters et al. (2017b) is a mostly individual one. Take for examples:

The concept of repertoire offers a way in which to articulate the ways individuals draw on their diverse resources mapping them onto functions in a communicative act (Kusters et al., 2017b, p. 5, emphasis added)

The examination of signed, gestured and spoken language interaction in multimodal (simultaneous) configurations explored in this issue provides the opportunity to observe communication between individuals with semiotic repertoires that do not show significant overlap, including between individuals who do not share the same primary ‘languages’, ‘codes’ or ‘modalities’. (Kusters et al., 2017b, p. 11, emphasis added)

This focus on individuals ‘bringing’ their ‘own’ semiotic repertoires as evolved in people's life histories was inspired by approaches of scholars who located the (linguistic) repertoire in speakers and their life trajectories (e.g. Blommaert & Backus, 2013; Busch, 2012). These scholars were interested in how parts of individual repertoires could be matched, mismatched, taught, accessed or not accessed within the contexts and communities in which they communicated; often to the background of individual mobility and migration.

Some scholars, even when approaching the repertoire as nested in individuals, also attended to spatial repertoires (e.g. Blackledge & Creese, 2017; Kusters, 2017a; Pennycook, 2017). Spatial repertoires include elements that can become sedimented in a particular context, related to practices in them, such as interactions in markets (Pennycook & Otsuji, 2015). Spatial repertoires can be appropriated in people's personal (individual) repertoires and the other way around. In these approaches, the wider material and social context are sometimes treated as subordinated to the agentive individuals located in it. The exercise of classification or inventory of semiotic resources that individuals use, Canagarajah (this issue) argues, is a necessary first step when working with the new concept of the ‘semiotic repertoire’. However, a way to advance their study would be to situate these repertoires much more firmly in their contexts, to decentralise the individual. Approaches that go beyond this methodological individualism include New Materialism, which gives agency to objects and material resources (see Canagarajah, this issue) and posthumanist applied linguistics, questioning the centrality of humans in the study of language (Pennycook, 2018). A key concept in these approaches is that of distribution:

The idea of distributed language helps open up the discussion since it challenges the idea of languages as internalised systems or individual competence, and argues instead for an understanding of language as embodied, embedded and distributed across people, places and time. (Pennycook, 2017, p. 8)

This approach implies that meaning emerges from distributed practice and that semiotic repertoires do not solely reside in individuals or even in humans. In some analyses, an object or a surface or an image may be semiotically relevant while in others it may not be. The semiotic repertoire is not an internalised construct that is simply brought to an interaction. Semiotic resources pose affordances and constraints which are co-constructed, shaped within, or taken up within, interactions between people, objects, and material environments (Tagg and Lyons, this issue).

There is the need for a theory then, to index which of these resources become relevant; ‘we cannot merely add more semiotic items to our translinguistic inventories, but need instead to seek out a way of grasping the relationships among a range of forms of semiosis’. (Pennycook, 2017, pp. 3–4). To theorise these relationships, Pennycook suggests the notion of the semiotic assemblage, which

expands the semiotic inventory and relocates repertoires in the dynamic relations among objects, places and linguistic resources, an emergent property deriving from the interactions between people, artefacts and space. (Pennycook, 2017, pp. 11–12)

This notion of the (semiotic) assemblage was taken up by the majority of authors in the current special issue. For example, Canagarajah; Tagg & Lyons; and Stone & Koehring show how repertoire assemblage is shaped by constraints and affordances of the contexts in which these take place – respectively located in a meeting room, in online messaging apps and in a TV studio. The authors highlight how repertoires are not merely approached as sets of resources that are ‘owned’ or used by individuals, but are also located in space and get pooled together in processes of ‘repertoire assemblage’ (Tagg & Lyons, this issue).

‘Repertoire assemblage' as the unit of analysis

As Busch and Canagarajah point out in their papers in this issue; thinking about what methodologies are most suitable to theorise semiotic repertoires is crucial to the process of ‘grasping’ them. The ‘repertoire’ is ‘difficult to grasp empirically’; as Busch (this issue) writes: ‘the repertoire is not an object or a fact that can be perceived by itself’. A methodological question is then how to use the concepts of ‘semiotic repertoire’ and ‘semiotic resources’ within research. How do we identify which resources are meaningful for particular interactions; ie what is the unit of our analysis? Busch (this issue) organises the works of scholars who used the notion of the ‘repertoire’ along three different approaches, which she argues are paired with different sets of methodologies:

1. a focus on interaction (methodologically corresponding to interactional linguistics and conversation analysis);

2. a focus on discourse (methodologically corresponding to discourse analysis and language ideology research);

3. a focus on the subject (methodologically corresponding to phenomenological, language biographical approaches and positioning theory).

In its entirety, this special issue covers all three different approaches and their attendant methodologies, with some papers combining two or three approaches. The methodologies used by the authors of the seven research papers in this special issue vary from language portraits (Busch); moment analysis of video-recorded interactions (Canagarajah, Stone & Koehring, Goico); conversations in apps (Tagg & Lyons); interviews (Moriarty & Kusters); and auto-ethnography (Boldt & Valente). What most authors have in common though, is the approach of the semiotic repertoire as an assemblage.

When thinking about semiotic assemblages, Canagarajah (this issue) proposes the unit of analysis could be ‘activities’ and ‘frames’. A focus on activities (the approach taken by Canagarajah, Stone & Koehring, Goico, Tagg & Lyons and Boldt & Valente) means we ask the question: ‘How do meanings emerge in the activity that semiotic repertoires are part of?’ (Canagarajah, this issue) Frames, then, ‘help both interlocutors and researcher narrow down their focus to the semiotic resources that matter for the activity in question’. (Canagarajah, this issue). Frames may include ‘the communicative activity's task structure, participant frameworks, genre conventions, and language ideologies’ (Canaragajah, this issue). Importantly, since activities and frames vary, semiotic assemblages are contingent: as Bagga-Gupta and Carneiro point out in their commentary on this issue, we need to understand the concept of the semiotic repertoire ‘in terms of unstable assemblages’.

In the process of repertoire assemblage, repeatedly used resources, salient for an activity's frame, can become sedimented into a repertoire for this activity. Goico (this issue) discusses how within the context of interactions between deaf youth and their relatives in Peru, semiotic assemblages can become scripts, so that repeated similar interactions can happen in very economical ways. Bagga-Gupta and Carneiro (this issue), inspired by Agha (2004), suggest the framing of ‘enregisterment processes’ to look at how specific resources get ‘enregistered’ in repertoires.

So, if there are assemblages that are linked to particular activities and frames, how do semiotic resources get connected within these assemblages? Several authors in this issue take up the concept of ‘chaining’ to study these connections. Chaining consists of ‘the patterned ways in which potential communicative resources across named-modalities and named-languages are interlinked in mundane meaning-making processes of communication’ (Tapio, 2019, Bagga-Gupta and Carneiro, this issue). The scholarship related to the concept chaining has drawn attention to people's deployment of resources across oral, written, and signed linguistic resources together with other tools (material and intellectual) in and across physical and digital contexts (Bagga-Gupta, 2017). Chaining is a term that has existed since at least the late 1990s (Bagga-Gupta, 2000) and has been pushed from sign language scholarship before becoming part of general language studies scholarship, such as in this special issue. Stone & Koehring (this issue) show in their contribution how different resources can be simultaneously chained to each other; or sequentially: the use of certain resources can pre-empt what will happen next. Stone & Koehring thus show how different timings of the uses of resource leads to chaining them in different ways.

Evaluations of resources in semiotic assemblages

Within the process of repertoire assemblage as a distributed practice, different resources are evaluated by individuals: they are connected to moral values, to emotions, to affect, and to the body image. The authors in this special issue who work on theorising affect, morality and/or the body in the semiotic repertoire (Busch; Boldt & Valente; and Moriarty & Kusters) all take as starting point that repertoires cannot necessarily be seen/heard or perceived empirically in moment analysis (in contrast with the first four articles which engaged with video-recordings of observable events). Busch (this issue) asks what makes somebody's acquired individual resources available and accessible as spatial repertoires unfold. Availability and accessibility are broad terms that can refer to physical, emotional, and political considerations. To study people's evaluative stance vis-à-vis resources, Busch argues, we need to engage in ‘mapping out a space of potentialities (desires, aspirations) and constraints (anxiety, insecurity)’. These can be elicited in different ways, such as via language portraits (in which participants visualise their semiotic resources on a figure of a body silhouette), as Busch shows.

Evaluations of semiotic resources are parts of the ‘frames’ of activities in which semiotic repertoires occur. Crucial in this process of evaluation is attention to the senses. Sign language research in particular is instructive because of specific sensory ecologies that are involved in situations where sign languages are used. De Meulder et al. (2019, p. 901) pointed out that ‘the hidden presumption made by translanguaging scholars is that all their informants have unfettered sensorial access to linguistic input’. Practices that could be labelled as ‘translanguaging’, e.g. because they involve combinations of resources of spoken and signed languages, could be experienced by deaf people as oppressive due to them not having sensory access to resources. Sensory assymetry means certain resources are more salient than others (eg. gesturing and writing may be more salient than speaking); but also that certain resources are not accessed with the body. In their study of weather forecasts, Stone & Koehring emphasise that the interpreters (who are deaf themselves), have this specific sensory ecology in mind when constructing their message. In another example, Goico's (this issue) deaf participants have very limited access to named spoken or signed languages and are especially tuned in on facial expressions and body language.

Repertoire assemblage, as Tagg and Lyons (this issue) show, can be ‘explicit and subject to metalinguistic comment and critique [or] implicit and unremarked upon’. In that respect, Moriarty & Kusters’ paper (this issue) takes up the role of moralistic language ideologies regarding what resources are deemed appropriate for use within a given frame. For example, in deaf cosmopolitan encounters, the use of International Sign, American Sign Language, or certain mouthings and fingerspelling alphabets are deemed morally appropriate in some contexts but not in others. The moral stance towards not using sign languages that are associated with linguistic imperialism and the active use of features of host country's (sign) languages are ideologically connected with a transnational connection experienced by deaf signers based on their shared deafness. This links well with Busch’ observation (this issue) that: ‘doing communication depends not simply on having resources and competencies one can situationally draw on but significantly also on self-conceptions of one's being in the world’.

An issue related to evaluation of resources is that of affect, in the form of attachment to or rejections of certain resources. In their autoethnography, Boldt & Valente (this issue) focus on how a big shift in the semiotic assemblages in their workplace interactions caused disruption. Valente underwent a rapid loss of his residual hearing, and went voice-off in the work place. For both, Valente's going voice off and working with sign language interpreters cut off a particular stream of semiotic resources, which went went paired with experiences of loss, uncertainty, frustration and fear. Taking up the link with the assemblage; and in line with Busch’ approach (this issue), Boldt & Valente's show that the semiotic assemblage is infused with affect or intensities, and that affect is not property of individuals: ‘Intensities are not the property of a body, but rather are the product of a given assemblage of time, space, bodies, materials, histories, and enunciations which may or may not flow through human bodies’. (Boldt and Valente, this issue)

Boldt & Valente's and Moriarty & Kusters’ contributions studying evaluations of resources, and how people may not use the resources that may be most effective in a given situation, demonstrating Busch (this issue) point about there being a gap between acquired and accessible resources. People's evaluative stance towards semiotic resources impacts on how people use resources and how assemblages are constructed. This all relates back to the theme of hierarchies of resources in semiotic assemblages: in given activities, acquired resources may be shoved to the bottom for language ideological, political, practical aims and/or because they are not accessible for some or all participants.

Conclusion: defining the semiotic repertoire

The semiotic repertoire is not a toolkit of resources and/or located in individual people, but a distributed set of resources that are chained together contingently in activities; in a process called repertoire assemblage. Through repeated assemblages, repertoires can become sedimented in some contexts. The process of repertoire assemblage is imbued with practical and affective evaluations of resources, in relation to their availability and accessibility.

Disclosure statement

No potential conflict of interest was reported by the author(s).