A linguist looks at AAC: Language representation systems for augmentative and alternative communication, compared with writing systems and natural language

Abstract

Writing systems are technological innovations that make it possible to record and reproduce the spoken utterances in a human language. They are the oldest, but not the only, kind of language representation system developed by human technology. The field of Augmentative and Alternative Communication (AAC) has created various technologies to facilitate communication for people who cannot communicate through language in the normal way. Users of AAC devices are persons with a physical or mental disability which compels them to produce speech using a technological device; and these persons depend on the Language Representation System (LRS) associated with their particular device in order to communicate. The purpose of this paper is twofold: first, to introduce the kinds of Language Representation Systems used in AAC devices to the audience of this journal; and second, to compare these LRSs with written language. In making this comparison, I show that they are forced into certain inevitable parallels by the structure of natural language which they must represent. They obey the same constraints, among these the impossibility of a truly semantic language representation system. In this paper, I illustrate the range of these LRSs used in AAC devices by illustrating and comparing three different systems, using a tripartite taxonomy of these systems. The three systems are compared with each other and with writing systems, focusing on how they map to the linguistic units of natural language and the compositional structure of natural language. I focus on a subset of the population employing AAC devices: those persons who are physically impaired but cognitively intact; and whose linguistic ability is comparable to any person who communicates through language in the normal way. Next, I compare one of these systems, with the writing system for Japanese, and show that they have converged on some similar responses to different engineering challenges. Finally, I propose that writing systems should be regarded as a subclass of a larger set of Language Representation Systems, of which both they and these LRSs for AAC technology are members.

Keywords:

• AAC
• Compostionality of natural language
• Language representation systems

I am grateful for discussion and comments from: Deb Witkowski, Richard Sproat, Helen Stickney, Dorolyn Smith, Eric Nyberg, Satoshi Nambu, Lori Levin, Chris Klein, Katya Hill, Bruce Baker and two anonymous reviewers.

I am grateful for discussion and comments from: Deb Witkowski, Richard Sproat, Helen Stickney, Dorolyn Smith, Eric Nyberg, Satoshi Nambu, Lori Levin, Chris Klein, Katya Hill, Bruce Baker and two anonymous reviewers.

I am grateful for discussion and comments from: Deb Witkowski, Richard Sproat, Helen Stickney, Dorolyn Smith, Eric Nyberg, Satoshi Nambu, Lori Levin, Chris Klein, Katya Hill, Bruce Baker and two anonymous reviewers.

Notes

¹ Sometimes inflectional morphology may be associated with unpredictable meanings and part-of-speech changes (e.g., a driven person, a driving rain). Larger combinations or constructions of linguistic units, such as idioms, can also have unpredictable or idiosyncratic meanings. Figure 1 is a schematic diagram of natural language design, and although it is somewhat simplified, it captures the structure of natural language sufficiently accurately for our purpose of comparison with writing systems and AAC systems.

² An anonymous reviewer points out the need to emphasise that the syllables as well as phonemes may be represented indirectly. Syllabaries are, in fact, rarely one-for-one encodings of the syllables of a language, as discussed in Sproat (2000), following Poser (1992). The crucial fact is that, direct or indirect, the mapping is to linguistic units.

³ Higginbotham (1992) gives an overview of several approaches to keystroke savings in text-based AAC technologies.

⁴ It is important to note that the potential for automatic motor production goes only one way: from the LRS to the spoken language (i.e., reading); not from the spoken language to the LRS (i.e., writing). This asymmetry comes about because the speaking modality is consistent, coherent and parallel across languages. Although reading has to be learned it can become automatic, carried out without conscious thought. Writing, or converting spoken utterances to an LRS, on the other hand, is not consistent in the nature of its modality. For a native writer with the free use of their hands and fingers, it could become automatic. For a stonemason carving letters on a stone monument, it would be an entirely different matter, not at all parallel to someone writing a letter by hand. Typing can become automatic too, but it takes a different kind of training and different motor skills than handwriting. Writing Japanese by hand can be carried out with little thought by a native writer of Japanese, but writing in Japanese on a computer can involve navigating to menus, selecting from menus, attending to and focusing on such choices and decision points. Thus, the asymmetry in the potential automaticity of converting between writing systems and spoken utterances: the mapping from a writing system to the spoken language is a mapping to a consistent modality; the reverse, the mapping from the spoken language to a writing system, is not a mapping to a consistent modality.

⁵ An anonymous reviewer asks about the possibility of sentence prediction for AAC technology, comparable to word-level prediction from spelling. To equip a system with predictive power on the sentence-level similar to predictive spelling on the word-level would require language technology for syntactic language generation far beyond the reach of current AAC systems. Such a system would have to have syntactic generative power in the language that requires a quite different and far more complex kind of technology, even to produce a possible sentence, let alone the desired one. Some technologies attempting related ideas are in use by large companies with access to ‘Big Data’, doing, for example, natural language processing or machine learning or speech-recognition; but these would not be useful, applicable or available for AAC systems.

Some AAC systems make it possible to create particular sentences to store in the system and then produce when desired. However, they do not end up being widely used by our population. Hill (2001) found that the AAC speakers she interviewed used these sentences at a frequency of 0.1%, compared to word-prediction via spelling 7.0%.

⁶ Although a concept by itself is not a linguistic unit, lexical semanticists have found some ‘conceptual primitives’ that are composed into the meanings of verbs in natural languages. These are composed in regular ways that (unlike the conceptual primitives of Blissymbolics^©) have linguistic, syntactic and morphological properties. They are different from the conceptual primitives of Blissymbolics^©. See Levin (1993) and Tenny (1994) for discussion.

⁷ There are some Minspeak-users who do not even refer to the symbols on the overlay in automatically moving a finger or toe over the keyboard. There is also anecdotal evidence for motor automaticity from users who have worn the labels off their keyboards, and rather than taking the time to send them in for repair, they continue to use them fluently without the labels. AAC-speakers using Minspeak^® appear similar in this way to persons who produce spoken language automatically, without reflection or conscious effort. (See video: DennisMED.mov, at http://www.youtube.com/user/brucebaker11)

⁸ This is the system used by Chris Klein, as of this writing the president-elect of USSAAC (United States Society for Augmentative and Alternative Communication).

⁹ Higginbotham (1992) looks at a Minspeak^®-based system (Word Strategy^®) with spelling/word prediction from the point of view of keystroke savings. The text of his discussion addresses an outdated system, but the chart in Figure 2 (p. 266) showing WS w/optimised spelling, reflects the cost effectiveness of combining Minspeak^® with spelling and word-prediction.

¹⁰ To add an interesting thought: one difference between natural (spoken) language and written language is that written language has to be taught, while natural or spoken language is acquired by children without lessons. There is a case of a young boy learning to speak German via a German Minspeak^® system, whose acquisition of the German language seemed to follow the normal developmental trajectory for children acquiring German syntax and morphology (Ortloff 2010). This suggests that children who cannot otherwise speak, who are given access to a language output method during the critical ages for language acquisition, will use it just as speaking children use the spoken languages they are learning. The paucity of research in this area allows us to do no more than speculate, however, about what is likely a complex process.

¹¹ Recall from section ‘Writing systems and the combinatorics of natural language’ that kanji in isolation are not linguistic units—they must map to specific words or morphemes with linguistic properties, including pronunciation. The pronunciation of the kanji is predictable only by the word that it appears in.

¹² Richard Sproat (personal communication, August 10, 2014), August 10, 2014 points out that the examples of ‘table salt’ and ‘anteater’ illustrate the varying etymological depth of modern Japanese kanji compounds, and they show the kind of arbitrary historical accident that can make a writing system the way it is. In cases where 食 is interpreted as a verb, the compound 食言 ‘table salt’ (EAT+SALT) displays the expected Chinese VO order; whereas the compound 蟻食 ‘anteater’ (ANT+EAT) displays the expected Japanese word OV word order. (Compare with Chinese 食蟻獸). So considered historically, there can be more than semantic opacity in these compounds.

¹³ Although Japanese might seem an outlier among modern writing systems, an anonymous reviewer points out that complex systems like this were common among ancient writing systems; notably, cuneiform Hittite and aramaeograms in Persian languages. (See Daniels & Bright, 1996.)

¹⁴ The growing interest in classes of symbol systems relating to natural language is reflected in Sproat (2014), which came out just as this paper was going to press. Sproat presents a review of some less-than-satisfactory attempts to use statistical techniques to distinguish linguistic and non-linguistic classes of symbol systems.