Improving the Ideas Behind Multimodal Communication

Abstract

Our society is biased towards communicating with spoken and written language. This has a profound effect on how we all think about communication, and the role of language in it. The idea of “multimodal communication” has been useful for highlighting issues of communication access, but its value as a technical, professional concept is questionable. I argue that we can improve the ideas supporting multimodal communication and transform it into a technical concept by carefully reflecting on the presumptions of our profession, the nature of communication, and nature and functions of the modalities used for communication. As a starting point, we should relate core aspects of communication to different meaning-making modalities, and consider how modalities are combined to achieve communicative acts. In the longer term, this reconceptualisation will provide a basis for more targeted assessment, intervention, and advocacy for people with communication disorders.

Keywords::

• Multimodal Communication

Notes

1 Of course, languages like Auslan, tactile sign language, whistled registers of languages, etc., demonstrate that there are a variety of ways that the symbolic functions conventionally associated with lexico-syntactic channel of the vocal modality (i.e., words, phrases, and sentences) can be redistributed to alternative modalities and channels.

2 By contrast, some communicative acts (e.g., sarcasm and jokes) are designed to leverage contrasts between modalities (e.g., delivering a ‘serious’ facial expression alongside “non-serious” talk).

Additional information

Notes on contributors

Scott Barnes

Scott Barnes (@ScottBarnesMQ) is a speech pathologist and senior lecturer in the Department of Linguistics at Macquarie University. He is the program director of the Master of Speech and Language Pathology, and conducts research on conversation and communication disorders.