The impact of impaired semantic knowledge on spontaneous iconic gesture production

Abstract

Background: Previous research has found that people with aphasia produce more spontaneous iconic gesture than control participants, especially during word-finding difficulties. There is some evidence that impaired semantic knowledge impacts on the diversity of gestural handshapes, as well as the frequency of gesture production. However, no previous research has explored how impaired semantic knowledge impacts on the frequency and type of iconic gestures produced during fluent speech compared with those produced during word-finding difficulties.

Aims: To explore the impact of impaired semantic knowledge on the frequency and type of iconic gestures produced during fluent speech and those produced during word-finding difficulties.

Methods & Procedures: A group of 29 participants with aphasia and 29 control participants were video recorded describing a cartoon they had just watched. All iconic gestures were tagged and coded as either “manner”, “path only”, “shape outline” or “other”. These gestures were then separated into either those occurring during fluent speech or those occurring during a word-finding difficulty. The relationships between semantic knowledge and gesture frequency and form were then investigated in the two different conditions.

Outcomes & Results: As expected, the participants with aphasia produced a higher frequency of iconic gestures than the control participants, but when the iconic gestures produced during word-finding difficulties were removed from the analysis, the frequency of iconic gesture was not significantly different between the groups. While there was not a significant relationship between the frequency of iconic gestures produced during fluent speech and semantic knowledge, there was a significant positive correlation between semantic knowledge and the proportion of word-finding difficulties that contained gesture. There was also a significant positive correlation between the speakers’ semantic knowledge and the proportion of gestures that were produced during fluent speech that were classified as “manner”. Finally while not significant, there was a positive trend between semantic knowledge of objects and the production of “shape outline” gestures during word-finding difficulties for objects.

Conclusions: The results indicate that impaired semantic knowledge in aphasia impacts on both the iconic gestures produced during fluent speech and those produced during word-finding difficulties but in different ways. These results shed new light on the relationship between impaired language and iconic co-speech gesture production and also suggest that analysis of iconic gesture may be a useful addition to clinical assessment.

Keywords:

• Gesture
• Aphasia
• Semantic knowledge

When communicating, speakers use a range of movements that depict the conceptual content of their message. These movements occur alongside speech and are often referred to as iconic gestures (McNeill, 1992). The nature of the relationship between iconic gesture and language has been the focus of a large body of research and discussion (e.g., McNeill, 2000). Findings of research which has investigated spontaneous iconic gesture production by people with aphasia suggests that aphasia can impact on the frequency of iconic gesture (Carlomagno & Cristilli, 2006; Cicone, Wapner, Foldi, Zurif, & Gardner, 1979; Hadar, Wenkert-Olenik, Krauss, & Soroker, 1998; Hogrefe, Ziegler, Weidinger, & Goldenberg, 2012; Lanyon & Rose, 2009). This has led many researchers to believe that iconic gesture and language therefore are part of the same system or are two highly integrated systems (for discussion of different models, see De Ruiter, this issue; McNeill, 2000). If gesture and language are intrinsically linked, then it is likely that they both rely on intact semantic knowledge. This suggests that if semantic knowledge is impaired, as can occur in aphasia, both the frequency and form of iconic gestures would be affected.

People with aphasia produce gestures during two very different conditions: alongside fluent speech and alongside word-finding difficulties (Cocks, Dipper, Middleton, & Morgan, 2011). Hadar and Butterworth (1997) proposed that the gestures produced during word-finding difficulties had a different origin to those produced during fluent speech. They proposed that gestures produced during fluent speech originate early in speech processing, before formulation has occurred, and thus have a conceptual origin, whereas, gestures produced during word-finding difficulties originate at a later stage when lexical retrieval fails. Word-finding gestures thus have a lexical origin. While the gestures produced during word-finding difficulties have a lexical origin, Hadar and Butterworth (1997) proposed that when a word-finding difficulty occurs, there is a new search for the target word or an alternative word. This new search results in a new set of conceptual processes being activated. They propose that conceptual processing results in visuo-spatial images being activated and when a word-finding difficulty occurs, a new set of visuo-spatial images are activated reflecting the new search. There is thus a “rerun” through conceptualisation. Gestures that occur during word-finding difficulties are thus a “by-product” of this imagistic activation. This therefore suggests that impaired semantic knowledge would have differing impacts on the frequency and form of gestures produced during both fluent speech and word-finding difficulties.

In the earlier versions of their model, Hadar and Butterworth (1997) proposed that the gestures produced during word-finding difficulties facilitated word-finding difficulties in three different ways: they helped to focus conceptualisation; they held important features during semantic reselection; and they directly activated word forms in the phonological lexicon. They thus proposed that there was a direct route from the visual image to the phonological form which bypassed semantics. The direct route however, was removed from later versions of the model (Hadar et al., 1998).

Investigations of the frequency of iconic gesture production by people with aphasia have found that there are differences between types of aphasia, but this can be dependent on how frequency is calculated. In a study by Cicone et al. (1979), and more recently Carlomagno and Cristilli (2006), participants were categorised as having either Wernicke’s aphasia or Broca’s aphasia. Cicone et al. (1979) found that while participants with Wernicke’s aphasia produced a higher total frequency of gestures than participants with Broca’s aphasia and control participants, the participants with Broca’s aphasia produced a higher proportion of iconic gestures. Carlomagno and Cristilli (2006) challenged this finding, and reported that the results relating to frequency of iconic gestures differed depending on how frequency was calculated. They found that when frequency was calculated per unit of time, the participants with a diagnosis of Wernicke’s aphasia produced more iconic gestures than participants with Broca’s aphasia and control participants. However, when frequency was calculated per word, the participants with Broca’s aphasia produced a higher frequency of iconic gestures than participants with Wernicke’s aphasia and control participants.

Similarly, Lanyon and Rose (2009) also explored gesture frequency across different types of aphasia but unlike Cicone et al. (1979) and Carlomagno and Cristilli (2006) they compared the frequency of specific types of gestures that were produced during fluent phases of speech with those that were produced during word-finding difficulties. They found that people with aphasia produced a high frequency of gestures during word-retrieval difficulties. There was not a significant difference between the number of word-retrievals that were accompanied with gestures that were resolved, and those that were accompanied with gestures that were unresolved.

While the frequency studies have made an important contribution to the field of research, most did not assess semantic knowledge specifically, or investigate the impact of impaired semantic knowledge on gesture frequency. In contrast to the frequency studies already discussed, Hadar et al. (1998) did investigate the impact of impaired semantic knowledge on the frequency of gesture production. They categorised the participants in their study as presenting primarily with semantic, phonological, or conceptual impairments. The conceptual participants had relatively impaired semantic knowledge of objects, as indicated by poor scores on the Pyramids and Palm Trees assessment (PPT; Howard & Patterson, 1992). They found that participants with either a phonological impairment or a semantic impairment produced a higher frequency of “wide and complex” gestures (which they refer to as “ideational gestures”) than control participants, whereas those with conceptual difficulties produced a similar frequency of ideational gestures to the control participants. They also found that those participants with phonological impairment or a semantic impairment produced a proportionally similar amount of iconic gesture to the control participants, whereas those with conceptual difficulties produced less. However, this study explored gesture frequency in just twelve participants, of which only four had a conceptual impairment. Unlike Lanyon and Rose (2009), they also only measured overall iconic gesture frequency, and did not separate frequency of iconic gesture produced during fluent speech from frequency of iconic gesture produced during word-finding difficulties.

In addition to frequency, there is also a possibility that impaired semantic knowledge may impact on iconic gesture form. However, the majority of the studies that have investigated the impact of aphasia on iconic gesture form have used a single case study design and most of the participants had relatively intact semantic knowledge.

Kemmerer, Chandrasekaran, and Tranel (2007) described the form of the iconic gestures produced by Marcel, a man with anomia, when describing the Sylvester and Tweety “Canary Row” cartoon. His discourse production was described as “non-fluent”. An analysis of Marcel’s gestures indicated that he was able to produce appropriate and elaborate iconic gestures in the absence of verbal language and when verbal language was impaired. However, as Marcel had relatively intact semantics and had difficulty “activating the appropriate lexical-phonological structures from lexical-semantic input” (Kemmerer et al., 2007, p. 14), the impact of impaired semantic knowledge on gesture production was not explored.

Similarly, Cocks et al. (2011), Dipper, Cocks, Rowe, and Morgan (2011), and Pritchard, Cocks, and Dipper (in press) also investigated the form of the iconic gestures produced by participants with aphasia when describing the same Sylvester and Tweety cartoon as that used in Kemmerer et al. (2007). These recent studies also used a single case study design and explored iconic gesture production by participants who also had relatively intact semantic knowledge but impaired phonological encoding. The findings indicated that when speech was relatively unimpaired, the individuals with aphasia and the control participants used a high proportion of iconic gestures that depicted the path and manner of actions. However, when participants experienced word-finding difficulties, they produced a higher proportion of iconic gestures that outlined the shape of objects (referred to as “shape outline” gestures; Cocks et al., 2011; Pritchard et al., in press). The participants in Cocks et al. (2011) and Pritchard et al. (in press) also had relatively intact semantic knowledge and so again the question of how impaired semantic knowledge impacts on gesture production could not be explored.

One study that did explore the impact of impaired semantic knowledge on gesture form, using a group study design, was that by Hogrefe et al. (2012). They explored iconic gesture production by 24 participants with severe aphasia during a cartoon description and found a correlation between semantic knowledge of objects and the formal diversity of hand gestures produced. While not specifically investigated by Hogrefe et al. (2012), this finding indicates that impaired semantic knowledge of objects may impact on what semantic information iconic gestures depict. However, like Hadar et al. (1998), Hogrefe et al. (2012) did not explore the differences between gestures produced during fluent speech and those produced during word-finding difficulties.

While Hogrefe et al. (2012) and Hadar et al. (1998) have made significant contributions to our understanding of the impact of impaired semantic knowledge on gesture production, they only assessed semantic knowledge of objects. The degree of semantic knowledge impairment can be dependent on word class. A number of cases of people with aphasia who have difficulties with nouns but do not have difficulties with verbs, and vice versa, have been described in the literature. Marshall (2003) hypothesised that this difference is due to semantic or conceptual differences or different distributions of semantic features. She suggested that one possibility for this difference is that during semantic or conceptual processing, verbs may be defined by action or thematic features, whereas nouns by perceptual or sensory features. The difference in semantic or conceptual properties of nouns and verbs led to the creation of the Kissing and Dancing Test (KDT; Bak & Hodges, 2003), which assesses the semantic knowledge of verbs. This was designed to be used alongside the PPT test, which assesses the semantic knowledge of nouns. By using both assessments, clinicians and researchers are able to compare noun and verb semantic knowledge (Bak & Hodges, 2003). Therefore in order to understand the relationship between semantic knowledge and iconic gesture production it is important to consider semantic knowledge of objects and actions separately.

As well as not exploring the impact of impaired semantic knowledge of actions, Hadar et al. (1998) and Hogrefe et al. (2012) also did not explore whether impaired semantic knowledge impacted differently on the frequency and form of gestures produced during fluent speech versus those produced during word-finding difficulties. As already discussed, Hadar and Butterworth (1997) have suggested that gestures produced during both fluent speech and word-finding difficulties both rely on intact semantic knowledge. However, in Hadar and Butterworth’s (1997) model of gesture production, gestures produced under each of these conditions have different origins. Gestures produced during word-finding difficulties had a lexical origin, whereas those produced during fluent speech had a conceptual origin. There is also some evidence that the types of gestures produced under these conditions differ (Cocks et al., 2011; Pritchard et al., in press). Impaired semantic knowledge may therefore impact on gestures produced in each of these conditions in different ways. This has not previously been explored.

In summary, there have only been a handful of studies which have investigated the impact of semantic knowledge on iconic gesture production. However, up to this point, none have investigated how impaired semantic knowledge affects the form and frequency of iconic gestures produced during fluent speech versus those produced during word-finding difficulties, nor have they explored the impact of semantic knowledge of objects versus actions. These gaps in the research motivated the present investigation.

THE CURRENT STUDY

The research presented here investigated the relationships between semantic knowledge and the frequency and form of iconic gestures produced by 29 individuals with aphasia and 29 control participants when describing a Sylvester and Tweety cartoon. This was the same cartoon used in the studies by Kemmerer et al. (2007), Cocks et al. (2011), Dipper et al. (2011), Pritchard et al. (in press) and many other studies of unimpaired individuals (e.g., Kita & Özyürek, 2003). The frequency and form of iconic gestures produced were analysed during times of relatively fluent speech, and times during which the participants were experiencing word-finding difficulties. Relationships between semantic knowledge of objects and actions, and gesture frequency and form was analysed in the two conditions.

METHOD

Participants

Participants with aphasia

Twenty-nine participants with aphasia (12 female, 17 male) were recruited from community support groups to take part in this study. The inclusion criteria were relatively broad. Participants were included if they had “self-reported” mild-moderate aphasia and spoke English as a first language. Participants were excluded if they had coexisting neurological diagnoses such as dementia, or were unable to consent to the study due to significant comprehension difficulties. Comprehension difficulties were determined by their score on the Western Aphasia Battery-Revised (WAB-R; Kertesz, 2006).

Only one participant was left-handed. Participants were all more than one year post-stroke (range = 16 months to 32 years). Their average age was 60.9 years old (SD = 14.85). Seven participants had completed tertiary level education, 21 had reached and completed secondary school level education, and one had completed only junior school level education.

Healthy participants

The participants with aphasia were compared to 29 neurologically healthy control participants (18 female, 11 male). Control participants did not have a history of psychiatric disorder, neurological illness or insult, nor any other serious medical condition. All participants were right-handed and all spoke English as their first language. The average age of control participants was 59.69 years old (SD = 13.63). Thirteen participants had completed tertiary level education and the remaining participants had reached and completed secondary school level education.

Assessment data

Assessment data are summarised below and in Table 1, also below.

TABLE 1 Participants with aphasia information in order of severity as indicated by aphasia quotient score

					Spontaneous speech score			Object and action naming battery		Pyramids and palm trees	Kissing and dancing test
Age	Months post-stroke	Information about stroke	Aphasia type	Aphasia quotient	Fluency (Total = 10)	Information content (Total = 10)	Total (20)	Object score (Total = 81)	Action score (Total = 50)	(Total = 52)	(Total = 52)
73	126	Left Hemisphere (ischaemic)	Broca’s	40.1	4	5	9	25	6	52	52
75	23	Left Hemisphere	Conduction	46.1	7	5	12	2	0	52	43
82	17	Left Hemisphere (ischaemic)	Conduction	54.0	5	5	10	53	17	52	51
65	17	Left Hemisphere (ischaemic)	Wernicke’s	55.7	6	9	15	54	19	31	37
73	21	Left posterior parietal lobe, basal ganglia and insular cortex infarct	Conduction	58.0	7	5	12	7	16	46	49
54	91	Subarachnoid (haemorrhage)	Broca’s	58.4	4	8	12	51	19	52	50
62	96	Left Hemisphere (ischaemic)	Broca’s	62.5	4	8	12	41	15	49	52
64	141	Left Hemisphere (ischaemic)	Wernicke’s	62.8	6	9	15	59	10	43	41
80	240	Left Hemisphere (ischaemic)	Conduction	69.1	5	8	13	50	23	47	36
46	57	No information available	Broca’s	69.6	4	9	13	78	14	49	46
39	120	Left Hemisphere	Anomic	71.2	5	9	14	71	27	45	51
50	54	Left Middle Cerebral Artery (ischaemic)	Wernicke’s	71.5	9	7	16	75	26	51	52
78	84	Left Hemisphere (ischaemic)	Anomic	72.3	5	8	13	65	27	41	39
61	29	Left Hemisphere (ischaemic)	Conduction	77.8	9	8	17	69	38	41	43
65	132	Left Hemisphere (ischaemic)	Anomic	80.0	9	9	18	57	35	52	52
83	142	Left Hemisphere (ischaemic)	Conduction	80.0	9	9	18	70	28	36	38
42	36	Left Hemisphere (haemorrhage)	Anomic	81.4	6	9	15	76	41	51	51
36	18	Right Hemisphere	Anomic	82.3	9	9	18	67	36	43	44
48	30	Left Vertebral Artery Dissection	Anomic	83.2	9	6	15	75	43	49	50
93	89	Left Hemisphere (ischaemic)	Anomic	84.5	9	10	19	60	15	39	43
47	33	Right Hemisphere	Anomic	84.5	9	9	18	71	36	50	46
57	384	Left Hemisphere (ischaemic)	Anomic	85.2	9	9	18	61	43	49	51
58	132	Right Hemisphere (haemorrhage)	Anomic	85.2	9	9	18	62	41	44	44
48	372	Left Hemisphere (haemorrhage)	Anomic	86.8	9	10	19	66	30	49	52
46	254	Left Hemisphere	Anomic	86.8	9	10	19	70	37	50	46
67	57	Left Hemisphere (ischaemic)	Anomic	88.4	9	10	19	78	33	45	45
73	16	Left cerebellar infarct	Anomic	88.5	9	9	18	72	29	49	47
49	24	Subarachnoid (haemorrhage)	Anomic	89.5	9	9	18	79	36	51	52
52	22	Left Hemisphere (haemorrhage)	Anomic	89.7	6	9	15	69	44	45	43

Motor skills and apraxia

All participants completed the Action Research Arm Test (Lyle, 1981), which tested strength and range of movement in the right and left upper limb. Five participants with aphasia scored 0/57 for the right upper limb and two scored 0/57 for the left upper limb, indicating complete paralysis. One participant with aphasia scored 12/57 for the right upper limb and one scored 3/57 for the right upper limb, indicating limited use. All participants who had limited use or complete paralysis of one of their upper limbs had full use of the other upper limb. All control participants obtained perfect scores for both the left and right upper limbs on this test.

All participants with aphasia completed two apraxia assessments: the Birmingham University Praxis Screen (BUPS; Bickerton et al., 2012), and the test for motor apraxia (Poek, 1986). None of the participants with aphasia obtained scores that would suggest that they had a diagnosis of limb apraxia.

Language

A battery of formal language assessments was completed, consisting of WAB-R and An Object and Action Naming Battery (OANB; Druks & Masterson, 2000).

Participants’ aphasia quotient scores on the WAB-R ranged from 40.1 to 89.7 (M = 73.46, SD = 14.15). The majority of the participants were classified according to their WAB-R scores as having anomic aphasia (16). The next most common classification was conduction aphasia (6), then Broca’s aphasia (4), and the least most common was a classification of Wernicke’s aphasia (3). See Table 1 for details.

On the OANB, participants’ naming scores for objects ranged from 2.47 to 97.5% and for actions 0 to 88%. On average, participants were better at naming objects (M = 73.77%, SD = 24.03) than actions (M = 54.07%, SD = 24.06), t(28) = 5.49, p < .05 (see Table 1 for details).

Semantic knowledge

In order to determine whether the participants with aphasia had impaired semantic knowledge in relation to objects and actions, the three picture versions of the PPT and KDT were carried out. In both of these tests, the participant is required to select a picture from a choice of two that is associated with the target picture.

The picture version of the PPT is scored out of 52, with a score of 50 and above considered to be within the normal range (Bak & Hodges, 2003). Scores ranged from 31 to 52 (M = 46.73, SD = 5.20). See Table 1 for the exact scores.

The picture version of the KDT is also scored out of 52, with a score of 48 and above considered to be within the normal range (Bak & Hodges, 2003). Scores ranged from 36 to 52 (M = 46.43, SD = 4.97). See Table 1 for the exact scores.

A composite semantic knowledge score was also calculated by adding the two scores together.

Procedure

Participants were not informed that the project focused on gesture production until after participation. Participants were instead invited to take part in the “describing events project”, examining the impact of stroke on “telling stories”.

All participants watched the Sylvester and Tweety “Canary Row” cartoon. It was divided into 8 episodes of approximately 1 minute. The cartoon description task was chosen because of the large body of research which has used this procedure when investigating gesture production by control participants and participants with aphasia (Cocks et al., 2011; Dipper et al., 2011; Kemmerer et al., 2007; Pritchard et al., in press). This would therefore allow for comparisons to be made between the gesture of the participants in this study, and other people with aphasia.

Before the participants watched the cartoon, checks took place to ensure that it was audible and visible to participants.

After watching each episode, participants were asked to describe the clip they had just watched. They were instructed that they should do this as though they were describing it to somebody who had not seen the cartoon before.

Equipment

Participants’ cartoon descriptions were recorded on a digital video camera placed approximately 1 metre in front of the participant. Cartoon clips were presented to participants on a laptop with a 15-inch screen.

Coding procedure

Gesture production

The videos of participants were segmented and coded using the gesture and sign language analysis program ELAN (Wittenburg, Brugman, Russel, Klassmann, & Sloetjes, 2006). All iconic gestures were tagged. Iconic gestures included any gestures that depicted semantic aspects of the concurrent speech (McNeill, 1992), the target word (in the case of gestures that occurred during word-finding difficulties) or appeared to depict a semantic aspect of the concurrent speech or target word but it was not clear what it was (these gestures were classified as “other”). Gestures that were considered not iconic were not tagged or included in the analysis. Iconic gestures were then classified using the categories similar to those used in Cocks et al. (2011) and Pritchard et al. (in press). See Table 2 for categories.

TABLE 2 Iconic gesture classifications used

Type	Definition
Path only	Depicts the direction of movement, for example, hand moves up to depict a cat climbing up a drainpipe. As these gestures only indicate the movement of an object in space from one location to another, they were considered “semantically light”.
Manner	Depicts the way in which a movement or action takes place, for example, index finger and thumb form a pincer grip to depict how an item has been picked up. Gestures that contained both path and manner were included in this category. These gestures contain a large amount of semantic detail about the action that is carried out, and were therefore considered “semantically heavy”.
Shape outline	Moulds or traces the outline of an object, for example, hands mould a circular shape to represent a bowling ball.
Other	Gestures are clearly iconic, but semantic features or relationship to co-speech is unclear.

Total iconic gesture frequency was calculated using the methods outlined in Carlomagno and Cristilli (2006). This included number of iconic gestures per unit of time (total number of iconic gestures divided by total time of discourse), and number of words per gesture (total number of words divided by number of gestures). For frequency calculations that involved time, all participants were included. For frequency calculations that involved number of words, those participants who did not produce any iconic gestures were removed from the analysis, as a zero value would have suggested they had a higher frequency, and the inclusion of the total number of words would have suggested they had the same frequency as those participants who only produced one iconic gesture.

All narratives were transcribed verbatim. Periods of word-searching behaviours were tagged. These were defined as the following based on Murray and Clark’s (2006) indicators of word retrieval difficulty:

• Longer than typical pauses of <500 ms. Pauses that were clearly due to reasons other than word-finding difficulties, for example, before an individual said “I can’t remember what happened next”, were excluded.

• Circumlocution of a target word.

• Producing onomatopoeia in the place of a target verb.

• Semantic errors.

• Phonological paraphasias.

• Neologisms.

• Metalinguistic comments such as “I can’t remember the word”.

• Repetitions included a word or phrase immediately repeated. If the repetition was for emphasis (e.g., “so the cat came back with a big, big cage”), this was not counted.

For each period of word-finding difficulty (WFD) it was recorded whether an iconic gesture occurred during this phase, whether the WFD was for an object or an action or other, and whether the WFD was resolved. The WFDs were classified into object, action, and other, based on the positioning of a WFD in an utterance, and the hypothesised target. For example, a long pause, followed by “the bird”, would be classified as a WFD for an object. Similarly, a participant producing “oh you know … the little thing” in the same position would also be classified as a WFD for an object. A WFD was classified as resolved if the item produced by the participant appropriately described the cartoon, and the participant appeared satisfied with it. Instances of unresolved WFD included semantic errors which were not corrected by the participant; occasions where the participant appeared to “give up” and move on; or occasions where the participant produced an item they were not satisfied with, for example, “the bird looked through the … not a telescope, but you know”. If an iconic gesture occurred during the WFD, the gesture was classified using the gesture categories described above.

Gestures were then grouped into two categories: “non-WFD gestures” and “WFD gestures”, depending on whether they occurred during a period of WFD or not. Analysis of these groups was then carried out separately.

Analysis

In order to investigate the relationship between semantic knowledge, iconic gesture form, and frequency, a series of correlations was carried out using semantic knowledge scores on either the PPT, the KDT, or the composite score combining PPT and KDT scores

As WFD for nouns was more likely to be related to semantic knowledge of objects, for correlations that considered WFD for nouns just the PPT score was used. Similarly, for most of the correlations that involved WFD for verbs, just the KDT score (which indicated semantic knowledge of actions) was used. There was one exception to this. Gestures that depict manner usually depict aspects of both the action and the object, for example, when gesturing placing a box on a table the shape of the object is depicted by flat hands and the action is depicted by movement. It was therefore possible that “manner” gestures may be associated with semantic knowledge of both actions and objects. An additional correlation was therefore carried out between the composite semantic score and the proportion of WFD gestures produced during WFD for verbs that were classified as “manner”. For all correlations involving non-WFD gestures, and when all WFD gestures were considered together, the composite score was used. This was because it was not possible to predict whether the target description included an object, an action or a combination of the two.

Inter-rater agreement

A second judge coded all iconic gestures and all WFDs during a 2-minute sample from each participant, taken 1 minute from the start of the narrative. For the three control participants who produced narratives of less than 3 minutes, gesture was coded from 1 minute from the start until the narrative was completed. The percentage of agreements was 85% for coding iconic gesture and 85.2% for identifying periods of word-finding difficulty.

RESULTS

The participants with aphasia spent significantly longer telling their narratives (M = 537.4 seconds) than the control participants (M = 299.2 seconds): t(56) = 3.84, p < .05. However, there was no significant difference between the groups for the number of words that their narratives contained (M for participants with aphasia: 666.4 words; M for healthy control participants: 738.3): t(56) = .91, p < .05.

Frequency

The participants with aphasia produced a significantly higher frequency of total iconic gesture than the control participants when both methods of calculating frequency were used: words per iconic gesture (with participants who did not produce any iconic gesture removed): t(53) = 2.87, p < .05; iconic gestures per minute (with participants who did not produce any iconic gestures included): t(56) = 3.39, p < .05.

However, when only the non-WFD gestures were considered, this difference was no longer significant: words per iconic gesture (with participants who did not produce any iconic gesture removed, which included an additional participant who only produced gestures during WFD): t(52) = 1.73, p < .05; iconic gestures per minute (with participants who did not produce any iconic gestures included): t(56) = .06, p < .05.

In order to determine whether there was a relationship between the semantic knowledge of the participants with aphasia and the frequency of their non-WFD gestures, a correlation was carried out between the frequency of non-WFD gestures (calculated using both methods) and the composite semantic knowledge score. The composite semantic knowledge score was used as non-WFD gestures related to both objects and actions. There was not a significant relationship between the frequency of non-WFD gestures and the composite semantic knowledge score (gestures per minute: r = .052, p < .05; words per gesture: r = −.217, p < .05).

Non-WFD iconic gesture form

The non-WFD gestures of both the control participants and participants with aphasia were most frequently classified as either “path” or “manner”. As “other” gestures were rare, these were removed from the analysis. A repeated measures analysis of variance (ANOVA) was carried out with the proportion of iconic co-speech gestures that were classified as “path only”, “manner” and “shape outline” as the dependent variable, and group classification as the independent variable. Mauchly’s test indicated that the assumption of sphericity had been violated, (χ²(2) = 12.4, p < .05), therefore degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity. The results showed that there was a main effect of gesture type, F(2,104) = 81.38, p < .05. There was not a significant interaction between gesture type and group, F(2,85.54) = 2.98, p < .05. There was no main effect of group, F(1,52) = .023, p < .05. These results therefore indicated that the participants with aphasia used similar types of non-WFD gestures to the control participants.

Further analysis of the “manner” gestures of the participants with aphasia indicated that there was a significant relationship between composite semantic knowledge scores and the proportion of non-WFD iconic gestures that were classified as “manner”, r = .372, p < .05 (see Figure 1 below). While the correlation between composite semantic knowledge scores and the proportion of non-WFD iconic gestures that were classified as “path only” did not reach significance, there was a negative trend, r = −.310, p = .054 (see Figure 2 below). These results therefore suggested that participants with intact semantics were more likely to produce “semantically rich” gestures when not experiencing a WFD.

Figure 1. The relationship between composite semantic knowledge scores and proportion of non-word-finding gestures that were classified as “manner”.

Figure 2. The relationship between composite semantic knowledge scores and proportion of non-word-finding gestures that were classified as “path only”.

No other correlations between the composite semantic knowledge scores of the participants with aphasia and the proportion of their gesture types for non-WFD gestures were significant (shape outline: r = .06, p < .05; other: r = −.065, p < .05).

Semantic knowledge and word-finding difficulties

While the participants with aphasia frequently experienced word-finding difficulties (M = 50.65, SD = 28.75), the control participants rarely did (M = .28, SD = .96). There was not a significant relationship between the number of WFDs and the composite semantic knowledge scores of the participants with aphasia (r = .091, p < .05), or the percentage of WFDs resolved and the composite semantic knowledge scores (r = −.287, p < .05). There was, however, a significant correlation between composite semantic knowledge scores and the proportion of WFDs that contained iconic gestures for participants with aphasia, r = .398, p < .05 (see Figure 3 below), thus indicating that participants with intact semantics were more likely to produce iconic gestures when experiencing a WFD.

Figure 3. The relationship between composite semantic knowledge scores and proportion of word-finding difficulties that contained gesture.

Word-finding difficulties for objects or characters

For the participants with aphasia, WFDs for objects and characters were common (M = 31.69, SD = 18.24). These were less common for the control participants (M = .28, SD = .96). On average participants with aphasia produced iconic gestures during WFDs for objects or characters on 40.52% of occasions (SD = 22.53). Of the four control participants who had word-finding difficulties, they produced iconic gestures during these WFDs on average 85% of occasions. When only the participants with aphasia were considered, there was not a significant relationship between the PPT score and the proportion of WFDs for objects or characters that contained iconic gestures (r = .261, p < .05).

Iconic gestures that occurred during WFDs for objects or characters were most frequently classified as “other”, “manner” or “shape outline” (see Figure 4 above). Whilst for the group of participants with aphasia, the correlation between PPT score and the proportion of iconic gestures that occurred during WFDs for objects or characters that were shape outline did not reach significance (r = .289, p = .06), there was a positive trend. There was not a significant relationship with the proportion of any of the other gesture types that were produced during WFDs for objects and the PPT score of the participants with aphasia (path only: r = .146, p < .05; manner: r = −.249, p < .05; other: r = −.139, p < .05).

Figure 4. Types of gestures used when experiencing word-finding difficulties for characters or objects.

Word-finding difficulties for actions

None of the healthy control participants had word-finding difficulties for actions. WFDs for actions were less common for the participants with aphasia than for objects (M = 14.93, SD = 12.18). However, it was more common for these WFDs to be accompanied by iconic gesture (M = 60.29%). There was a positive correlation between the KDT scores of the participants with aphasia and the proportion of their WFDs for actions that contained iconic gestures, r = .464, p < .01.

Iconic gestures that accompanied WFDs for actions were most frequently classified as either “path only” or “other” followed by “manner” (see Figure 5 above). There were no significant correlations between the proportion of iconic gesture types and KDT scores (path only: r = −.035, p < .05; manner: r = .243, p < .05; shape outline: r = .083, p < .05; other: r = −.141, p < .05). There was also not a significant correlation between the composite semantic score and manner, r = .193, p < .05.

Figure 5. Types of gestures used when experiencing word-finding difficulties for actions.

Word-finding difficulties for words other than actions or objects/characters

None of the healthy control participants had word-finding difficulties for words that were not objects, characters or actions. The participants with aphasia rarely had WFDs for words that were not objects, characters or actions (M = 4.65, SD = 5.42). On average, 44.84% of these were accompanied with iconic gesture. As there were so few WFDs for words other than actions or objects or characters, no further analysis was carried out on these.

Success at resolving word-finding difficulties

There was no difference between the number of resolved WFDs that contained gesture (350) and the number of unresolved WFDs that contained gesture (351).

Summary of findings

Participants with aphasia produced a higher frequency of iconic gestures than control participants. However, when the gestures that were produced during WFDs were removed, the difference was no longer significant. While there was not a significant relationship between frequency of iconic gestures produced during fluent speech and semantic knowledge, there was between semantic knowledge and the proportion of WFDs that contained gesture. These results indicated that those with intact semantic knowledge were more likely to produce gestures when experiencing a WFD. However, when WFDs for verbs and nouns were considered separately, this relationship was only significant for verbs.

There were also significant relationships between semantic knowledge and gesture type in both non-WFD and WFD conditions. While overall, participants with aphasia and control participants produced a similar proportion of different iconic gesture types during non-WFDs, participants with intact semantics were more likely to produce semantically rich “manner” gestures. This finding signifies a role for lexical-semantic knowledge in gesture production. Semantic knowledge also impacted on the types of gestures produced during WFDs for nouns but not for verbs. There was a positive trend between semantic knowledge of objects and the proportion of “shape outline” iconic gestures produced when experiencing a WFD for nouns, again revealing a link between lexical semantics and gesture.

Finally, we found no evidence that spontaneous co-speech gesture production during WFD aided retrieval, as there were a similar number of resolved and unresolved WFDs that contained gesture.

DISCUSSION

The findings of the current study indicated that while the participants with aphasia produced a higher frequency of iconic gestures than the control participants, this was largely due to a high frequency of gestures produced during word-finding difficulties by the participants with aphasia. The findings were similar to Lanyon and Rose (2009), who found that word-retrieval difficulties were associated with a higher frequency of all types of gesture. The current study adds to this picture by emphasising the link between word-finding and specifically iconic gesture. In addition, the current study found that impaired semantic knowledge affected both iconic gestures produced during word-finding difficulties and iconic gestures produced during relatively fluent speech, but in different ways.

Overall, individuals with aphasia produced similar types of iconic gestures during fluent speech as the control participants. Similar to the studies by Cocks et al. (2011) and Pritchard et al. (in press), the most common types of gestures produced during fluent speech by both control participants and those with aphasia were path and manner gestures. Interestingly however, speakers’ semantic knowledge was related to the proportion of the two most frequent gesture types (“path only” and “manner”) produced during fluent speech. There was a significant correlation between semantic knowledge and the proportion of non-WFD gestures that were classified as manner and a negative trend between semantic knowledge and the proportion of these gestures that were classified as “path only” gestures. In spoken languages the expression of manner is particularly complex due to the diverse types of concepts that need to be encoded by the linguistic system (Talmy, 2000). Similarly, manner gestures are semantically rich with gestures taking advantage of the modality to describe conceptual space and movement with physical space and movements of the hands. “Path only” gestures, however, are semantically “light” in that they only depict the direction of movement.

The narratives of people with aphasia are often characterised as having a high proportion of semantically light verbs (e.g., Breedin, Saffran, & Schwartz, 1998). While there is limited evidence available which suggests that there is a relationship between semantic knowledge and the proportion of verbs used by participants with aphasia that are semantically light, there is evidence for this relationship in other populations where semantic knowledge is impaired. For example, Méligne et al. (2011) found that people with semantic dementia who have impaired semantic knowledge also produce a higher proportion of less specific or semantically light verbs. Therefore, the findings of the current study suggest that just as impaired semantic knowledge can impact on the semantic richness of verbs (Méligne et al., 2011), it also impacts on the semantic richness of iconic gestures that are produced during fluent speech. Thus both the iconic gestures produced during fluent speech and language are modulated by the semantic abilities of the person with aphasia. These findings therefore support the theory that iconic gesture relies on intact semantic knowledge (Hadar & Butterworth, 1997), and provide further evidence that language and gesture therefore are part of the same system or are two highly integrated systems (McNeill, 2000).

Impaired semantic knowledge also impacted on gestures produced during word-finding difficulties, but in a different way to those produced alongside fluent speech. There was a positive relationship between speakers’ semantic knowledge and the proportion of word-finding difficulties that were accompanied by iconic gesture. This therefore suggests that the ability to produce iconic gesture when language fails is also dependent on relatively intact semantic knowledge. This was explored in more detail by looking at the differences between word-finding difficulties for nouns and verbs separately.

The type of iconic gesture used during word-finding difficulties was also affected by semantic knowledge. Word-finding behaviours for objects and characters were common, and a very high proportion of these contained iconic gestures. Like the two case studies described in Cocks et al. (2011) and Pritchard et al. (in press), many of the iconic gestures produced when participants were having word-finding difficulties for nouns were “shape outline” gestures. Although it did not reach significance, there was a positive trend between semantic knowledge related to objects and the proportion of iconic WFD gestures for objects that were classified as “shape outline”. If this finding is combined with the finding that participants with relatively intact semantic abilities produced a higher frequency of gestures during word-finding difficulties, then this suggests that the observation of a high frequency of shape outline gestures during a narrative may indicate relatively intact semantic knowledge. This is both a clinically and theoretically important finding. In particular, it provides further support for Hadar and Butterworth’s (1997) model of gesture production, which suggests that semantic knowledge is important for iconic gesture production during word-finding difficulties. However, the relationship is more complex when the results regarding gesture production during word-finding difficulties for verbs are considered.

Unlike the two participants in Cocks et al. (2011) and Pritchard et al. (in press), some of the participants in the current study did exhibit behaviours associated with word-finding difficulties for actions. These were rarer than the word-searching behaviours for objects; however, a higher proportion of these were accompanied by iconic gesture. There was a positive relationship between semantic knowledge for actions and the proportion of word-finding difficulties that contained iconic gestures, similarly highlighting the relationship between semantic knowledge and the ability to use gesture when language fails. Interestingly however, unlike word-finding difficulties for nouns, there were no significant correlations or trends between semantic knowledge for verbs and the proportion of iconic gesture types produced during word-finding difficulties. This may be because of the conceptual processing differences between nouns and verbs (Marshall, 2003). Alternatively, it could be because there are other variables that may influence the types of gesture produced during word-finding difficulties for verbs other than semantic knowledge, for example imageability of the word or the cognitive flexibility of the person with aphasia. This was not explored in the current study. Further investigation is needed to explore how other variables may impact on iconic gesture production type during word-finding difficulties.

The production of iconic gesture during word-finding difficulties did not help with resolving the WFDs, as there were a similar number of resolved and unresolved WFDs that contained gesture. This finding is similar to that of Lanyon and Rose (2009), who also found that there was not a difference between the number of resolved and unresolved WFDs that contained gesture.

There were limitations in the current study that deserve further consideration. One of the behaviours that was used to indicate that a WFD had occurred was a pause. It is possible that some of these pauses were caused by factors other than WFDs, such as attempting to recall the next episode in the story. It was not possible to separate these out from the WFD pauses reliably. The classification of the WFDs was determined by the word that followed the word-finding behaviour. This of course was a somewhat subjective judgement and it cannot be undeniably stated what the target words were for all the word-finding difficulties. This may have impacted on the results of the study.

Furthermore, none of the participants with aphasia in this study presented with limb apraxia. This was a surprising finding given that limb apraxia is common in people with aphasia (Kertesz & Hooper, 1982) and “limb apraxia” was not an exclusion criterion for this study. However, this may reflect the fact that the participants had mild-moderate aphasia and did not have significant comprehension difficulties. Apraxia is more commonly associated with more severe aphasia and comprehension difficulties (Kertesz & Hooper, 1982). Given that participants with mild-moderate aphasia were specifically selected for this study, it is worth stressing that these participants do not necessarily reflect a “typical” group of people with aphasia. Whilst not reducing the significance of the findings, it is important to note that this potentially restricts their wider generalisability.

While this is one of the largest studies of spontaneous gesture production by people with aphasia to date, it still only featured 29 participants. Future research should include a larger number of participants.

In summary, the results of this study support the model by Hadar and Butterworth (1997) which suggests that both word-finding gestures and gestures produced during fluent speech rely on semantic knowledge. These results suggest that when semantics is intact there is feedback to the gestural system during word-finding difficulties, but when semantics is impaired there is no feedback to gesture and so no gesture is produced. This raises the possibility that feedback from language to gesture must therefore be mediated by semantics. This possibility therefore enhances the Hadar and Butterworth (1997) model by increasing the specificity of the processing involved in the language-gesture link.

In addition to these theoretical implications, these results also suggest that analysis of spontaneously produced iconic gesture could be of use clinically. The findings presented here indicate that gestural indicators of intact semantic knowledge include a high proportion of semantically rich gestures during fluent speech; a high frequency of iconic gesture during word-finding difficulties; and an increased prevalence of “shape outline” gestures during word-finding difficulties for nouns. Because these gestural indicators are noted from a sample of narrative discourse, they provide clinicians with an efficient means of detecting intact semantic knowledge.

This study was supported by the Dunhill Medical Trust (Grant Reference: R171/0710).

Notes

* Author is currently affiliated with the School of Psychology and Speech Pathology, Curtin University, Perth, Australia.