Common ground in AAC: how children who use AAC and teaching staff shape interaction in the multimodal classroom

Abstract

Children who use augmentative and alternative communication (AAC) are multimodal communicators. However, in classroom interactions involving children and staff, achieving mutual understanding and accomplishing task-oriented goals by attending to the child’s unaided AAC can be challenging. This study draws on excerpts of video recordings of interactions in a classroom for 6–9-year-old children who used AAC to explore how three child participants used the range of multimodal resources available to them – vocal, movement-based, and gestural, technological, temporal – to shape (and to some degree, co-control) classroom interactions. Our research was concerned with examining achievements and problems in establishing a sense of common ground and the realization of child agency. Through detailed multimodal analysis, this paper renders visible different types of practices rejecting a request for clarification, drawing new parties into a conversation, disrupting whole-class teacher talk-through which the children in the study voiced themselves in persuasive ways. It concludes by suggesting that multimodal accounts paint a more nuanced picture of children’s resourcefulness and conversational asymmetry that highlights children’s agency amidst material, semiotic, and institutional constraints.

Keywords:

• AAC
• agency
• common ground
• conversational asymmetry
• multimodal communication

A major stream of AAC research has focused on studying everyday conversations involving people and AAC. This research has tended to adopt either quantitative, distributional perspectives of interaction, focusing on quantifying the use and functions of language, or qualitative perspectives, providing insights on the ways that social actions are achieved in everyday conversations through conversational analysis and microanalytic methods (e.g., Bloch & Wilkinson, 2011; Clarke & Wilkinson, 2007; Higginbotham, 2009; Norén et al., 2013). Conversation analysis and microanalysis involve the close examination of recorded data and detailed transcription for the purposes of studying how people engage in everyday, spontaneous conversations (See Higginbotham & Engelke, 2013). By focusing on the communicative work that participants undertake in conversation, conversational analysis and microanalysis in AAC have informed collective understanding across a number of areas. These areas include understanding collaborative participant efforts for supporting conversational flow and shared meaning (Bloch & Wilkinson, 2011; Clarke & Wilkinson, 2007; Pilesjö, 2014; Solomon-Rice & Soto, 2011), problematizing the constraints of AAC devices in conversation (Higginbotham et al., 2016; Higginbotham & Caves, 2002; Ibrahim et al., 2018), generating design implications for technology (Ibrahim et al., 2018; Pullin et al., 2017; Valencia et al., 2020), and proposing implications and interventions for ways of supporting conversations that involve people and AAC (Clarke et al., 2017; Norén et al., 2013). This prior research has highlighted the collaborative effort by children who use AAC and their conversation partners during conversations.

Clarke and Wilkinson (2007) studied the role of speaking partners in organizing interactions that involved children with cerebral palsy, their peers and speech generating devices (SGDs). Using conversation analysis (Sacks et al., 1974), the authors studied in detail how interactions were created on a moment-by-moment basis. They attended to how speaking conversational partners would treat certain behaviors as relevant over the course of a sequence of turns. The study highlighted the shared responsibility of participants for incorporating SGD use into talk, illustrating the ways that orally speaking children established opportunities for children who used AAC to contribute to the conversation. By showing how both parties worked together to use the SGD as a shared resource, the authors suggested that the boundaries for ownership of turns and talk are less clear, highlighting issues of timing, communication mode, and establishment of shared meaning.

Reflecting the perspective that conversational contributions are a shared accomplishment, co-construction strategies have also featured prominently in research examining conversations involving adults and child AAC users (Solomon-Rice & Soto, 2011). For example, by focusing on the strategies that speaking adults used to support child AAC users with personal storytelling, Solomon-Rice and Soto (2011) identified a range of co-construction strategies used by adults to support children’s narratives. These included: questioning (e.g., tell me about who came to your party¹), adult prompting, (e.g., tell me more) and repetitions (e.g., child: “mama make my hair”, adult: mama make your hair?) The findings highlighted that co-construction strategies such as these questions are a key strategic tool for supporting the development of child-centered, personal narratives in conversations involving adults and child AAC users.

Focusing on the structural aspects of discourse, Savolainen et al. (2020) have reported that in conversations involving school-aged, aided communicators and familiar speaking adults, both parties demonstrated commitment to achieving shared understanding by clarifying implicit meanings through linguistic and temporal resources. For example, when one aided speaker used a pre-stored phrase (e.g., “all my relatives are very dear to me”), the orally speaking partner ascribed meaning to it to facilitate shared understanding. In this case, the orally speaking partner interpreted the child’s fondness for their relatives as the reason why they often speak about this topic, which was then reinforced by both parties in their following turns (Savolainen et al., 2020). Building on prior interactional research, Savolainen et al.’s example illustrates that both parties contributed in varied, strategic ways to establish shared understanding. This suggests that participant contributions may be less concerned with the quality and/or structure of contributions, and more concerned with collaborative practices for negotiating shared meaning, given the differences in available resources.

Through their focus on distributed and collaborative effort, the body of conversation and microanalytic research in the AAC field demonstrate how interactional analysis can offer rich insights on how social action is achieved by attending to multiple strategies and resources used in conversation.

Multimodal communication and establishing common ground

A benefit of qualitative microanalysis is the opportunity such work affords in revealing the ways in which repertoires of multimodal, unaided and aided AAC resources can be effectively utilized in interactions, for example in establishing common ground (Norén et al., 2013). Common ground is defined as the state where both parties have reached mutual understanding of their joint attention to a topic, idea, or object (Clark, 1996; Higginbotham & Caves, 2002). Establishing common ground is a shared interactional accomplishment (Sterponi & de Kirby, 2016), not only sharing mutual understanding, but also acknowledging and signaling understanding of the other person’s intention to act, given the available resources and social context (Goodwin & Heritage, 1990; Kress, 2010). For example, in an interactional study involving a girl with severe speech and physical impairment and her everyday conversation partners, Pilesjö (2014) showed that participants collaboratively created local meanings by attending to the girl’s coordination of gaze and arm/hand movements during naturally occurring interactions (Pilesjö, 2014). Similarly, Norén et al. (2013) found that during interactions involving a SGD, a child AAC user and an adult using oral speech, the adult would orientate toward the multiple and explicit ways that the child projected that they were about to contribute to the topic; focusing on the child’s gaze, vocalization, hand movements toward the screen, and on-screen folder navigation (Norén et al., 2013). In the case of Norén et al., establishing common ground was achieved in a child-led manner, with the adult orienting to the child’s explicit projections. In line with prior interactional research literature, the studies also reinforce that all parties take responsibility in gradually establishing joint understanding (Clark, 1996), often supported by the orally speaking partner’s verbal commentary of the child’s multimodal actions (Clarke & Wilkinson, 2013; Savolainen et al., 2020).

Task management, classroom roles, and agency

Considering that interactions are shaped by the contexts within which they occur (Higginbotham, 2009), at an institutional level, the norms of interaction within classroom settings and the teacher/student roles that individuals take when carrying out tasks can also impact on the conversation. Similarly, the conversational patterns that are enacted within the classroom reflect and are shaped by their institutional context, offering insights into the ways that those in student roles display agency over how and what they communicate. In the context of this paper, we define agency in terms of having a capacity to act within the structures and constraints that are present within the moment (Manyukhina & Wyse, 2021; Valencia et al., 2020). As such, we are interested in the ways that child AAC users can exercise agency over classroom interactions. Classroom interactions involving students and teachers are commonly characterized by the teacher guiding students through tasks that are intended to support student learning (Korkiakangas & Rae, 2014). As such, in classroom settings, the role of the teacher involves setting up and managing activities that allow for students to be actively involved in their own learning (Niemi, 2002).

A small body of work has suggested that in classroom interactions involving students who use SGDs and their teachers, student contributions to topics can be even more organized than typically seen in classrooms, as a way of securing space for student SGD-mediated contributions. For instance, the student’s SGD-mediated turns can be achieved by teachers explicitly pre-allocating opportunities for SGD-mediated turns, by visibly projecting to the device or through the design of their questions that summon particular students (Norén et al., 2013; Tegler et al., 2020). Although these studies demonstrate that teachers often aim to support students who use SGDs to have agency over contributing to topics, these practices also reinforce that child-led initiations of new tasks and topics for children using SGDs are highly rare.

Whilst these types of adult-structuring strategies can be helpful for creating space for children to contribute to interactions, adult structuring can also be problematic. For example, in related research that has considered how teachers managed conversational flow during class group conversations involving children with autism, who have communication difficulties, Heller and Kern (2021) research illustrated how teaching staff would manage conversational flow by offering interruptions and corrections during the student’s response which consequently inhibited the student’s confidence and engagement in the conversation (Heller & Kern, 2021, see also Antaki & Wilkinson, 2013). Similarly, Fasulo and Fiore (2007) observed that in conversations involving students with autism and their speech therapists, the therapist’s preoccupation with linguistic appropriateness disrupted and inhibited talk-in-interaction (Fasulo & Monzoni, 2009). These studies suggest that the opportunities for children to have agency over shaping interactions can be closely associated with the opportunities that the adult helps to create within the social setting. However, in interactions that involve children who use AAC and their teachers –where the teacher can be responsible for carefully organizing these opportunities– it is unclear how this orchestration impacts on the child’s agency over contributing to classroom interactions in unexpected ways. In particular, more work is needed that considers the ways that children with little or no speech use a broader array of modes to communicate, for example, bodily action, timing and their immediate environment, and how this is interpreted or treated by their adult conversation partners. This is an important step for understanding and foregrounding non-normative ways of communicating that have traditionally received less focus in research and clinical practice (DeThorne & Searsmith, 2021).

Given that children who use AAC are multimodal communicators (Clarke, 2016; Jagoe & Smith, 2016; Norén et al., 2013; Pilesjö, 2014; Soto & Olmstead, 1993), a small body of research has started to unpack how aspects of unaided communication such as gaze, gesture, body orientation and a broader remit of modes are used to co-create shared meaning (Norén et al., 2013; Pilesjö, 2014), suggesting that different modalities and resources hold different potentials for establishing common ground. The overarching aim for the current analysis was to examine how children and staff engaged in the classroom when SGDs were not present/available or not explicitly used in conversation. Acknowledging that the school setting produced particular norms in terms of adult and student roles, we were interested in investigating interactions in which adults inhabited a role of educator in one-to-one and group situations. Within this context our research questions concerned what problems emerged in the interaction for children and staff, in particular, in relation to achieving a sense of common ground and in the realization of child agency. We adopted a multimodal co-constructed view to explore these issues. The following research questions were asked: During classroom interactions, how do children who use AAC and their adult conversation partners work toward achieving common ground and task accomplishment, and What impact does task accomplishment have on child agency?

Method

Participants

Three 7-year-old girls Maya, Clara, and Grace, 7 (pseudonyms) and their teaching staff were recruited, with the help of the school team. These participants were chosen in line with a critical case sampling strategy (Patton, 1990) that would produce rich insights by maximizing children’s differences based on their communication and physical profiles. The sampling criteria were primary age students identified as having marked communication and physical impairments and using some form of SGD alongside an assortment of communication methods that included no-tech paper-based communication books, bodily movement, eye pointing and vocalization. The three participants were all in the same class. As gatekeepers to the school site, the headteacher was contacted and once they had given consent via email, the school then contacted parents/carers to invite participants to take part on the researcher’s behalf. Once written parental consent was obtained, verbal child consent was gained with participants and renewed at each visit. Table 1 describes participants’ communication and mobility profiles.

Table 1. Participant profiles (all names are pseudonyms)

Name	Age (in years)	Sex	GMFCS level	Communication and mobility profile
Maya	7	F	V	Mainly uses eye-pointing to communicate, including a symbol communication system on an eye gaze-controlled device, mounted to wheelchair. For mobility, uses partner assisted manual wheelchair. Has uncontrollable movements
Clara	7	F	III	Mainly uses approximations of single manual signing to communicate. Signing is unclear to unfamiliar people. Uses symbol communication system on a touch screen tablet with a key guard. Walks short distances unaided indoors but very unsteady and mostly with adult support. Uses a walking frame outdoors with helmet or partner assisted chair
Grace	9	F	V	Eye pointing, facial expression and tone of voice are clearest form of unaided expressive communication. Uses a symbol communication system on an eye gaze control device that is mounted to her wheelchair. For mobility, uses partner assisted manual wheelchair

Note. GMFCS: gross motor function classification scale

Setting

Fieldwork was carried out in a primary special school in the UK. This was treated as the primary context where children are supported to become competent communicators. The special school classroom context was also important for studying particular norms in terms of adult and student roles, and for investigating typical interactions that exist during tasks that have wide ranging learning objectives. For example, adult-supported self-care; and eating, recreational, exploratory, and craft-based activities.

Materials and measures

Video-based observation was used to record, study, and archive detailed cases (Derry et al., 2010). In the study, teaching staff were responsible for setting up the classroom resources and conversation environment, which inadvertently included decisions about the availability of different communication aids (e.g., paper-based systems and SGDs). This may have explained why SGDs were not always present in our data. Measures for studying interaction included identifying task events and attending to the structure of social interaction (detailed in data analysis section).

Research design

Qualitative participatory observational research was carried out to study interaction involving staff and students in classroom settings. Ethical approval was obtained from the University College London, Institute of Education ethics board.

Procedures

Data collection

In total, 10.5 h of video data of classroom interactions was captured over 14 weeks as part of an empirical study that investigated the salient features of multimodal communication involving adults and children who use AAC. The first author was present during the video recordings, which allowed for focusing on the contextual aspects of how children acted, which informed interpretations about the implications of children’s actions while re-watching videos at a later date. Observations were intertwined with the first author’s participation in interaction and dialogue with participants. To minimize the risk of the researcher’s presence interfering with the natural flow of everyday routines and conversation patterns, the researcher spent many hours within the classroom. The first author’s routine engagement with everyday classroom activity over the 14 weeks of data collection desensitized the staff and students to her presence and the presence of the recording equipment. The researcher’s involvement in the classroom provided advantages for the research in offering, as this insider knowledge provided context around events of interest and formed part of the analysis.

Data analysis

We undertook a microanalysis of adult-child interactions. First, in order to segment and analyze the video recordings, a whole-to-part inductive approach to video analysis was taken (Derry et al., 2010; Erickson, 2006), whereby videos were viewed multiple times and indexed to identify shorter segments of interactional events. Events² were defined as episodes within an interaction where any behavior is initiated by the child or adult, that develops or sustains the topic of conversation, or that repairs and/or facilitates conversational flow (Müller & Soto, 2002). The 215 events initially identified were further filtered to include only those in which the adult adopted an educator role and events that had a task-oriented goal. Task-oriented was defined as events involving an adult and child that are intended to lead to a learning outcome and/or are related to class routines. This resulted in a dataset of 138 events (approx. 2.25 h of video, where each event lasted approximately 2–8 min). The next step was to characterize each task-oriented event with a descriptive label (e.g., the teacher asks the student a question during registration, or the student seeks the teacher’s attention during story telling activity). A breakdown of these tasks is presented in the Supplementary Appendix.

Next, the task-oriented events were clustered according to different outcomes reflecting our interests in the achievement of common ground and the realization of child agency. This was operationalized by grouping events where adults and children were aligned in task accomplishment and where they were misaligned. For each event a description was made of the ways in which unaided AAC (e.g., body action, gaze, vocalization) and wider tools and strategies (e.g., classroom objects, timing) were deployed. To apply a systematic and rigorous analysis, videos were watched multiple times so that the different possible interpretations of the developing patterns, or themes, would be exhausted (see for example, Higginbotham & Engelke, 2013; Korkiakangas & Rae, 2014; Tegler et al., 2020). Group viewings involving the authors allowed for determining whether different researchers noticed similar phenomena or alternatives, testing out the different explanations of data as the team began to build on clustering patterns in the data. Where group members identified alternative explanations, these were resolved by documenting and discussing examples across the data that evidenced the pattern, guided by a shared document that listed all of the developing data examples. We also undertook an inter-rater reliability check to ensure that we categorized alignment in task and common ground as accurately as possible. To achieve this, the first author coded all of the data (which consisted of video excerpts for all 138 task events that were approximately 2–8 min) as aligned or not aligned for both task and common ground. Using the same coding scheme, the second author then coded approximately 20% of the data (O’Connor & Joffe, 2020) comprising 29 task events, which for task alignment produced a Cohen’s Kappa score of 0.65, and for common ground alignment produced a Cohen’s Kappa score of 0.79, both suggesting substantial agreement. The excerpts were selected as representative examples, as following a theoretical sampling approach (Patton, 1990), they were the clearest examples of the intersection of task alignment and common ground that we were looking at. The three main task-oriented outcomes we describe are: (a) alignment in task, misalignment in common ground; (b) misalignment in task, alignment in common ground; and (c) alignment in task, alignment in common ground.

The multimodal microanalysis approach was informed by principles of social semiotic multimodal analysis (Bezemer & Kress, 2015; Kress, 2010) and micro analysis within in-person interactional AAC research (Higginbotham, 2009). Social semiotics focuses on the ways in which people use the resources that are available to them to create meaning in accordance to their interests (Bezemer & Kress, 2015). It emphasizes the agency of the sign maker and is interested in meaning making through multiple modes, for example, through gesture, speech and others. As such, social semiotics opposes a view that verbal communication is primary and that non-verbal communication is a secondary category to communication. Instead, the social semiotic perspective proposes that all forms of communication and representation must be attended to as they each offer distinct potentials for meaning making (Bezemer & Kress, 2015). Micro-analysis of in-person interaction focuses on real time interactions of two or more individuals and is interested in how features of interaction such as spatial orientation and communication modalities are employed by people to achieve common ground. The fine-grained inductive approach follows a turn-by-turn and moment-by-moment examination of participants actions and reactions to each other (Higginbotham, 2009).

A multimodal transcription method was employed that attended to the multiplicity of ways in which participants communicated in ways of their choosing (Bezemer & Mavers, 2011). For example, rather than attending to any one particular mode or communicative function, transcriptions foregrounded different modes, depending on participant modal choices in the moment. Videos were transcribed multimodally using Microsoft Word and Adobe Photoshop, and then organized in Microsoft Word. This enabled flexibility to adapt the transcription layout and capture in detail the full repertoire of participants’ communicative behaviors, for example, by inserting line drawings of video stills that illustrated spatial arrangements and the orientation of participants.

Results

The analysis generated three main task-oriented outcomes for establishing common ground and child agency: alignment in task, misalignment in common ground; misalignment in task, alignment in common ground; and alignment in task, alignment in common ground. For each of these outcomes, we describe how establishing common ground was unresolved at the expense of achieving a task, and how child agency was dependent on the teacher’s sensitivity to the meaning potential of unaided modes.

Alignment in task, misalignment in common ground

Unresolved common ground

Excerpt 1 is taken from an interaction that occurs during a class-based cooking session involving a child participant who uses AAC (Maya) and a familiar adult teaching staff member (Sally). Here, Sally and Maya are making an omelet and Sally is whisking the egg mixture. At the start of the excerpt, Sally asks Maya if she would like to add salt into the mixture. Maya’s reaction, or a lack of a recognizable action (apparent inaction), is interpreted as an affirmation of Sally’s question. Drawing on the wider conversation that surrounds this excerpt and knowing that Maya later comments that the omelet is salty (later in the conversation, not presented in the excerpt), we propose a possible alternative interpretation of Maya’s actions in response to Sally.

Excerpt 1. Teacher asks child participant a question during cook activity.

Time (min) →		Line no. ↓	07:16		07:19		07:20		07:24		07:25		07:26		07:27
Sally (teacher)	Body action	1	Whisks egg mix	7	Pauses whisking and leans to Maya	13	Leans on Maya’s tray	19		25		31		37	Resumes whisking, stands up and moves to collect salt
	Gaze	2	bowl	8	Maya	14	Maya	20	Maya	26	Maya	32	Maya	38	Bowl then ahead
	Vocal	3		9	You want some salt in there?	15	Yes?	21	No?	27		33	you do.	39
Maya (child)	Body action	4	Hands move to face	10	Hands lower to tray	16	Moves hands near face	22		28	Lowers arms to tray	34	Hands near face	40
	Gaze	5	bowl	11	bowl	17	SGD screen	23	closes eyes then gazes to mix and SGD	29	bowl	35	SGD screen	41
	Vocal	6		12		18		24		30		36		42

Sally asks Maya: You want some salt in there? It is observed that Sally is successful in advancing her own project (of seeking Maya’s answer to the teacher question) as she structures the interaction around her prior question to Maya about adding salt. However, Maya carries out several actions that raise the possibility that she may want to respond with her SGD or take the interaction in a new direction. However, Sally does not treat these actions as potentially communicative as she seeks further clarification to her original question (Sally: yes? no?). Maya moves her gaze between the bowl and SGD screen (Line 17), closes her eyes (Line 23), gazes to the bowl (Line 29) then back to the SGD screen (Line 35), each projecting the possibility that she may be about to initiate SGD use (see Clarke & Wilkinson, 2007; Savolainen et al., 2020). However, at Lines 31–33, Sally treats these actions as a response to her question and a trigger to proceed with the task (Sally: you do). Throughout the interaction, as Sally appears solely oriented on seeking a response to her original question, any of Maya’s consequent actions that may signal other intentions are treated as a response to Sally’s question.

In our data, the institutional roles of teacher and student, and separately the conversational roles of oral speaker and augmented speaker, explicitly signal asymmetry in distributions of power in several ways, for example, through the ways that the orally speaking adult structure their spoken turns and how they treated certain turns by the child as meaningful in particular ways (answering a yes/no question), the teacher sought control in advancing her own project and agenda within the task.

Issues with child agency

Acknowledging that inequality in available resources is a given factor in conversations involving children who use AAC and oral speakers (Tucker & Kretschmer, 1999), it was observed that orally speaking teachers would seek to resolve problems of understanding through verbal confirmation requests. Furthermore, while teachers had trouble ascribing meaning to children’s subtle multimodal signs created through bodily action (as illustrated by our previous example), they used their own bodily actions to support their verbal commentary in trying to resolve ambiguity. In one example of this, referring again to Excerpt 1, Sally recruits several strategies to encourage Maya to respond to her question. For instance, at Lines 13–15, Sally orally asks: Yes? while also leaning her elbow on Maya’s wheelchair tray and gazing toward her. Through her bodily actions, Sally explicitly signals that she is pressing Maya to respond to her prior question. In contrast, Maya’s subtle gaze and arms movements, which possibly signal a request to opt out or decline, are not treated as communicatively relevant by Sally (see Lines 10–35). Maya’s actions suggest that she is attending to Sally’s question, yet one possible explanation would be that Maya does not have additional resources available to her to resolve the unfolding ambiguity, and the signs that she is using hold less salience for Sally in attracting her attention.

In this instance, Sally uses confirmation requests to come to a shared understanding. Sally has organized the interaction by asking a closed question that allows her to attribute any of Maya’s actions following a question as an answer and as a yes or no response. Sally is having trouble interpreting Maya’s actions yet owing to the structure of the conversation, Sally verbally articulates what she believes Maya is communicating through body movement and gaze. As Maya’s actions are treated as an answer to Sally’s question, her possible sign to request something other than yes is lost. These findings suggest that certain resources were treated as more definitive than others in conversation, with orally speaking adult partners utilizing their own verbal modes to articulate interpretations of what was being expressed through children’s bodily movement and gaze.

Misalignment in task, alignment in common ground

Common ground established, yet child pursues a different task

In contrast to Excerpt 1, in other cases, children and adults were able to signal their mutual understanding of each other’s interest yet held different task goals. For one of the participants, Clara, this meant she was successful in steering the adult’s action and diverting their attention from a whole class activity.

Excerpt 2 is taken from a longer sequence where Clara competes with the teacher for the class’s attention as she attempts to redirect the group toward what she is doing. Clara used several tools and strategies to redirect the attention of the teaching assistant toward her own interests. By drawing on multiple resources with escalating shades of intensity (that increasingly pose more risk in damaging or breaking teaching equipment), Clara re-directed the whole class’s attention and commanded a response from Ada, the nearby teaching assistant, who was a familiar adult and could typically interpret Clara’s embodied communication.

Excerpt 2. Child participant uses varied tools to gain teaching assistant’s attention.

Time (min) →		Line no. ↓	12:25		12:28		12:32		12:38
Ada (TA)	Body action	1	Sitting, leaning back in her chair	7	Slightly orientates toward Clara shuffles	13	Spins chair slightly to face Clara	19	Pushes to stand up from chair, toward Clara
	Gaze	2	ahead toward child on other side of circle	8	Clara	14	Clara	20	Clara
	Vocal	3		9		15		21
Clara (child)	Body action	4	Hands held behind her back, kneeling forward	10	Whole body in a semicircle to face whiteboard	16	Stands up facing whiteboard	22	Reaches to whiteboard pens, begins throwing them to the floor
	Gaze	5	turns to Ada	11	whiteboard	17	whiteboard	23	whiteboard
	Vocal	6		12		18		24

Over the course of the longer sequence, Clara throws whiteboard marker pens onto the floor; taps on the interactive whiteboard to change the screen display; pulls the whiteboard downwards on its moving mechanism so that it is within her reach; and attempts to sit on the ledge of the whiteboard. Each of these actions demands a reaction from Ada. It is not until Clara’s final action of attempting to sit on the ledge of the whiteboard, that Ada physically stops Clara from doing so by taking her arm and guiding her to sit on a nearby chair.

Excerpt 2 reflects a smaller part of this interaction, illustrating how Clara’s gaze and actions are synchronized with Ada’s reactions toward her. In the excerpt, Clara has moved next to a large whiteboard display that is mounted on the wall. Clara turns to her left to gaze toward Ada who is sitting on the edge of the semicircle, then turns to face the whiteboard. As she does this, Ada turns to face Clara (Line 7), then gradually stands up and moves toward Clara, who is about to throw the pens onto the floor (Line 19).

Throughout the sequence, Clara’s actions are timely and purposeful. She waits until Ada is attending to her before carrying out her next action. Her actions are coordinated with Ada’s actions, and she gradually steps up the intensity of what she is doing based on Ada’s reaction to her prior turn. Over the course of the sequence, unlike in the case of Excerpt 1, the child’s actions are more salient, based on the adult’s reactions. During the interaction, Clara slowly and smoothly moves from sitting on the floor to spinning around to face the whiteboard, then methodically dropping marker pens onto the floor one-by-one. These escalating actions are coordinated with Ada’s acknowledgement of her. Even though they each have different task goals, their actions are aligned as they express mutual understanding of what the other is doing, with Clara working to challenge the power dynamic. Here, we observe that Clara communicates through full bodily action and through her non-normative use of the markers, which is in concert with Ada’s reactions to minimize the disruption to the whole class activity. As such, Clara and Ada have established mutual understanding of each other’s intention (even though their intentions are not aligned), and consequently hold different task goals.

Alignment in task, alignment in common ground

Sensitivity to child-led action

Lastly, in some cases, children were able to regulate power dynamics by establishing common task interests, mutual understanding on what both participants were expressing, and could pick up the signs of what the other was intending. Communicating through bodily action enabled children to signal their capability to establish and regulate common topics of interest by drawing on richer and more cogent ways of communicating. Often, by combining multiple modes, children could express themselves for a broader range of functions, could regulate shades of intensity, and were able to advance their interests to a greater extent. For example, we observed that in certain conditions, for example, unstructured craft-based tasks, children undertook semiotic work that enabled them to redirect the adult’s attention toward their own task and support them in setting up optimal conditions for conversations that would follow. In such cases, children structured their environments to support communication on their own terms.

Excerpt 3 illustrates one of the ways that one child participant, Grace, directed the teacher, Steph, a familiar teacher, to position herself so that Grace was able to converse in a manner that was appropriate for her. The interaction occurs during a collage making craft activity within the classroom. The teacher is working with two students, and at the start of the excerpt begins explaining the collage making task to the students (teacher: What we’re going to do now-) but then pauses whilst organizing some picture cards she is holding. Grace reaches upwards and calls out with a sudden burst of volume and vocal intensity, gaining Steph’s attention. Unlike in the case of Maya (Excerpt 1), Grace’s signs are highly salient. Grace combines many modes to redirect Steph’s attention from sorting the materials. She pushes her body upwards and backwards in her chair, raises both of her arms outwards and calls out whilst gazing toward the teacher (Lines 10–12). Even though Grace’s verbal speech is unclear, beyond spoken words, she uses vocal properties –sudden volume and a short burst (Grace: Ahh!)– which is paired with her sudden change in body movement to attract the teacher’s attention. It is evident that she has gained Steph’s attention (at Line 13), as Steph moves her chair backwards and rotates herself slightly toward Grace.

Excerpt 3. Child participant directs teacher’s attention during craft task.

time →		Line no. ↓	34:48		34:50		34:52
Steph (teacher)	Bodily action	1	Sorting picture cards, facing table ahead	7	Sorting picture cards, facing table ahead	13	Moves chair backwards and spins chair around to face Grace slightly
	Gaze	2	toward materials on table	8	Toward picture cards in her hands	14	Picture cards in her hands
	Vocal	3	What we’re going to do now -	9		15
Grace (child)	Bodily action	4	Facing table, begins raising arms	10	Raises arms and body, stretching upwards and sideways, turning head from materials toward teacher. Visible tension in her arms and legs	16	Begins lowering her arms to her sides, tension in body reduces
	Gaze	5	toward materials that teacher is reaching toward	11	toward materials that teacher is holding	17	toward materials that teacher is holding
	Vocal	6		12	Ahh!	18

The intensity of Grace’s sign making combined with the teacher’s ability to interpret these signs is successful in gaining the teacher’s attention and directing her to sit in a position that arguably enables Grace to interact more comfortably with the teacher. Through this prompt and response pattern, Grace commands the teacher to be fully present in communicating the activity with her, rather than moving between the two students. Grace can also organize her communication environment so that it complements the arrangement of resources around her. For example, once Steph is facing Grace, they have shared access to an array of picture cards, magazines and scissors which are used in the collage making task. Having established common ground about Grace’s wish for the teacher to work with her, Grace is freely able to use gaze actions to direct the teacher toward her interests.

Discussion

The overall aim of the current study was to examine how children and school staff interact in task-oriented events when SGDs were not present or not focal. We addressed this by examining what problems came about during child-adult interactions, and in particular, in relation to achieving a sense of common ground and in the realization of child agency. The analysis identified not only that problems occurred but how they came about. This suggests that common ground—that is comprised of speakers establishing mutual understanding about the topic/idea/object, and alignment in signaling these shared understandings with each other (Clark, 1996)—as well as agency, are critical concepts to study for future research. The implications of these findings for AAC practitioners and researchers, and suggested directions for future research are discussed by highlighting the importance of recognizing a broader remit of communicative modes.

By treating children’s bodily actions (vocalizations, gazes, gestures) as meaningful signs,—as semiotic expressions designed to contribute to and direct the course of action– the analysis has shown instances where these signs are not noticed, misinterpreted, or not responded to and/or acknowledged by the speaking interlocutor, who continued to pursue the course of action; and instances where they were, leading to a change in the course of action. We might say the findings offer examples of both successful and unsuccessful attempts by students to intervene in a course of action that a teacher was displaying a commitment to. The study builds on AAC literature that recognizes the collaborative effort of children who use AAC and their conversation partners in SGD-mediated conversations (Clarke & Wilkinson, 2007; Savolainen et al., 2020; Solomon-Rice & Soto, 2011) and the central role of multimodal communication in SGD-mediated talk (Clarke, 2016; Norén et al., 2013; Pilesjö, 2014). Specifically, by considering a broader range of distributed factors during classroom tasks, (e.g., material objects and shared meaning making practices), the study critically considers how social and contextual arrangements are a key part of naturally occurring, co-constructed communication. The impact of this is that it allows for more sensitivity to children’s bodily actions alongside the range of social constraints within a classroom (e.g., teacher-led goals and available resources), drawing on these subtler modes to inform ways of supporting children to communicate.

One methodological benefit of studying interaction from an interpretive, multimodal and co-constructed perspective, was that it helped to critically consider issues of authorship for meaning making. This is important for contributing new understandings for studying interaction as a collaborative effort, in keeping with prior research that advances a constructivist perspective (Bezemer & Kress, 2015; Clarke & Wilkinson, 2007; Goodwin, 2000; Savolainen et al., 2020). As tasks are part of the norms of a classroom environment, tasks as a whole were examined. By describing the course of the interaction from a task, common ground and agency perspective, the study showed that often, adult and child actions were dependent on what had happened prior (Goodwin, 2004), and social constraints (e.g., institutional roles and the availability of adult-controlled material resources) impacted on how aligned individuals were in engaging in the same task and establishing common ground.

Children’s resourcefulness, recognizing and valuing modes

In employing a micro-analytic methodological approach, we were able to focus on the communicative and material resources that participants used to support meaning making within specific classroom tasks. In line with a broader commitment to recognizing the non-normative ways that children who use AAC can communicate, the paper advocates a need for describing and working with augmented speakers in ways that acknowledges semiotic modes that have traditionally been given less attention in interaction (DeThorne & Searsmith, 2021). For example, this can include ways of communicating that will typically be noticed secondary to expressive language content, such as features of vocalization, volume, breathing qualities.

Considering ways of studying children’s multimodal communication, these findings demonstrate the importance of attending to modes from the task perspective. In the examples, the three attempts of children who use AAC to initiate or respond during an interaction varied in terms of task objective and difficulty, (e.g., rejecting a yes-no question may have been more difficult than attracting attention). As illustrated through the study’s methodological approach, investigating modes at a micro level allowed for examining how far the child participants were able to exercise agency over their conversations and daily lives in relation to class-based tasks with varying difficulty. This was demonstrated by focusing on the persuasive ways that children expressed themselves during particular tasks through full bodily action, as an important component of interactional activity. In the case of one child participant, Grace, bodily action coupled with the adult conversation partner’s sensitivity to her actions, enabled Grace to arrange her environment in ways that made it possible for them both to access the tabletop craft task more readily.

Perspectives on conversational asymmetry

A recurring theme in the AAC interactional literature concerns characterizing the asymmetries of conversational interaction. In the current study, we refer to asymmetry as imbalances in participant speech acts and modalities that children who use AAC and their conversation partners use in conversation, rather than referring to asymmetry between speech input and non-speech output. The current paper contributes to that body of work by (1) studying how tasks are co-created and managed (i.e., who initiates and how aligned individuals are in jointly carrying out the task), (2) attending to whether individuals can reach mutual understandings about what each other is communicating, and (3) examining the ways that individuals respond to each other’s multimodal communication to reach alignment in task and common ground.

This current study also raises insights into how asymmetry in communicative resources and power imbalances play out during classroom interactions involving teachers and students who use AAC. The analysis suggests asymmetry operated through imbalances in agency and the use of unaided AAC, inconsistencies in teacher sensitivity to unaided modes of AAC, and imbalances in opportunities for advancing student-led projects. Therefore, we need to be mindful of how these layered asymmetries impact on how we treat multimodal communication, and in particular, how we attend to subtler modes of communication.

Implications for practice

The findings have several implications for school staff and therapists who are responsible for supporting the communication and learning needs of children who use AAC. By being equally sensitive to a wide range of modes, the findings showed that it was possible to focus on interactional features that might traditionally have been treated as secondary to talk (Jewitt et al., 2016). Also, the multimodal analysis approach prompted consideration of signs that were not typically treated as communicative. For example, noting Maya’s subtle pause in hand movements when her teacher asked a question, or the muscular tension in Grace’s arms when calling out for the teacher’s attention. By attending to a broader range of modes as communicative resources in their own right, the study identifies a need for strategies that can help speaking conversation partners in the classroom to recognize and acknowledge children’s subtler and personalized communicative actions.

One of the inescapable challenges of adopting this perspective involved making decisions about whether a child’s bodily movements were communicatively purposeful. For example, in the case of Grace, it was observed that owing to her marked communication and physical impairment, it was difficult for her to sit unsupported and coordinate her neck, torso, arm and leg muscles. When analyzing video data involving Grace, one interpretation might have been to treat all of Grace’s body movements as not intentionally communicative. The findings reinforce an important consideration for practitioners and for interactional researchers, that classifications systems and language choices significantly impact on how we recognize and provide a commentary to semiotic work that is undertaken by participants (Goodwin, 2000). For instance, by continuing to label and categorize modes as verbal and non-verbal action in AAC research, this distinction reinforces the idea that the many varied forms of bodily action are grouped into a reduced category and may all be secondary to speech. Further, contextual resources and timing, which also constitute marked means of communication, have received less attention as modes. Instead, this paper advocates for more nuanced readings of children’s bodily action that acknowledges the more graded levels of control that children can have over their movement, as well as attending to how these movements are treated by conversation partners.

Limitations and future directions

One of the challenges faced in this study was capturing a wide range of examples that were representative of teacher-student interactions. We have presented a small set of examples that offer a detailed, descriptive examination of teacher-student interactions. By selecting representative examples from across the data, we have attempted to highlight some of the problems that can occur in classroom interactions that involve children who use AAC and familiar teaching staff. However, we acknowledge that more work is needed with larger data and sample sizes in order to be able to draw generalizable conclusions about how communication manifests in school settings more broadly. Also, owing to the challenges of video recording spontaneous interactions involving children and adults in school settings, the dataset focused on interactions in the classroom. This impacted on the range of task-based events that were able to be studied. Also, the ways in which we defined tasks were teaching and learning oriented and we recognize that the notion of task would likely be defined differently for conversations in different settings, with different foci. Future work should focus on interactions from a wider range of contexts (e.g., within the home), to identify whether the findings are representative of interactions involving children who use AAC and their communication partners more broadly, as well as considering how these findings relate to child peer interactions.

Conclusion

This video observation study sought to investigate how children and school interact when SGDs were not available or not focal. This was achieved by taking a detailed, multimodal view to study how children who use AAC and their adult conversation partners used a range of communicative resources for achieving common ground during task-oriented interactions. AAC interactional research is at a crucial point for re-thinking its observational practices so that it can attend to the multiple and unbounded ways that meaning-making is achieved in conversations involving oral adult speakers and children who use AAC. This study suggests that co-constructed, multimodal accounts offer informative insights, highlighting children’s agency amidst material, semiotic and institutional constraints.

Disclosure statement

No potential conflict of interest was reported by the author(s). This article is partly based on a Ph.D. dissertation completed in 2020 by the first author.

Additional information

Funding

This work has received funding via an Economic and Social Research Council (ESRC) Doctoral Studentship and ESRC Postdoctoral Fellowship, under grant numbers ES/J500021/1 and ES/P000592/1, awarded to the first author.

Notes

1 Notation convention hereafter in line with von Tetzchner & Basil, 2011).

2 Event is deliberately used to segment interactions into multimodal units. We acknowledge that in their original reference, Müller and Soto (2002) used discourse unit; however, in the current article, we avoid using discourse to avoid any linguistic associations connected with our multimodal perspective.