1,815
Views
1
CrossRef citations to date
0
Altmetric
Research Article

Do gestures have a hand in verb retrieval? Investigation of iconic and non-iconic gestures in aphasia

ORCID Icon, ORCID Icon & ORCID Icon
Pages 781-804 | Received 27 Oct 2020, Accepted 19 Mar 2021, Published online: 05 Aug 2021

ABSTRACT

Background

According to the lexical retrieval hypothesis (LRH), the primary function of gestures is to facilitate word retrieval. Thus, individuals with aphasia (IWA) might benefit from gestures in case of word retrieval impairments. However, the facilitative effect of gestures on word finding is still unclear since most facilitation/treatment studies combined gesture with linguistic cues. The LRH claims that the semantic content of a gesture is decisive for a facilitation effect on word retrieval. However, it remains unclear if and how IWA can perform adequate iconic gestures, especially in the presence of hemiparesis or limb apraxia. Thus, in the present study, we systematically studied the effect of iconic and non-iconic gesture production on oral verb naming in IWA and analysed the iconicity of gestures produced by IWA and healthy controls.

Aims

In the present study, we aimed (1) to investigate the potential facilitation effect of iconic and non-iconic gestures on verb retrieval in IWA and (2) to examine the IWA’s ability to produce adequate iconic gestures.

Methods & Procedures

Six IWA with impairments in naming verbs arising from a phonological, a lexical-semantic, or a combined deficit participated in a multiple single-case study targeting oral verb production. In the first experiment, IWA were asked to name pictures and to simultaneously perform either a relevant iconic gesture, a non-iconic gesture, or no gesture at all. In the second experiment, the video-recorded gestures produced during verb naming in the iconic gesture condition were rated for iconicity and compared to the iconicity of iconic gestures produced by 12 language-unimpaired controls.

Results

The accuracy in oral naming of verbs (Experiment 1) did not differ significantly between all conditions. Most IWA (n = 5) were able to produce iconic gestures, irrespective of hemiparesis or limb apraxia, since the iconicity ratings did not differ significantly from language-unimpaired controls (Experiment 2).

Conclusions

In contrast to the general assumption of the LRH, no gesture facilitation effect was observed in the verb naming performance of IWA. Moreover, the majority of the IWA could perform iconic gestures in a similar way as language-unimpaired participants despite the presence of a hemiparesis or limb apraxia. Hence, the absence of a facilitation effect in the iconic gesture condition seems not to originate from an inability to produce adequate iconic gestures. Further research with more IWA is required in order to replicate our results.

1. Introduction

Gestures frequently occur during spoken communication. Concerning the function of gesture, several theories are discussed in the literature. Addressee-directed approaches suggest that gestures have a communicative function (De Ruiter, Citation2000; Kendon, Citation1994), whereas speaker-internal approaches assume that gestures support conceptual processing (information packaging hypothesis; Kita, Citation2000) or lexical retrieval (Hadar & Butterworth, Citation1997; Krauss et al., Citation2000). According to the lexical retrieval hypothesis (LRH) the primary function of gestures is to facilitate word retrieval (Krauss et al., Citation2000). Hence, for individuals with aphasia (IWA) suffering from word-finding problems gesture could possibly be a means to alleviate word retrieval. Word retrieval impairments are a common symptom in aphasia which can affect different word classes and lead to hesitations, circumlocutions and paraphasias in spoken communication. They may arise at different levels of the speech production process, that is, they can be caused by impairments at the lexical-semantic or the phonological level (e.g., Howard & Gatehouse, Citation2006).

In several facilitation/treatment studies, gesture production/action observation was used as a cue to facilitate word retrieval (e.g., Bonifazi et al., Citation2013; Boo & Rose, Citation2011; Crosson et al., Citation2007; Crosson et al., Citation2005; Ferguson et al., Citation2012; Marangolo et al., Citation2010; Marangolo et al., Citation2012; Raymer et al., Citation2007; Raymer et al., Citation2012; Raymer et al., Citation2006; Richards et al., Citation2002; Rodriguez et al., Citation2006; Rose & Douglas, Citation2001, Citation2008; Rose et al., Citation2002; Rose & Sussmilch, Citation2008; Routhier et al., Citation2015). However, the facilitative effect of gestures on word finding is still unclear. In almost all studies using gesture production to facilitate word retrieval (except Rose and Douglas (Citation2001) who examined the effects of gesture production alone for facilitating word retrieval), gesture was combined with verbal cues, for example, phonological/semantic cues or word repetition (cf. Marshall, Citation2006; Rose et al., Citation2013). Even in the gesture alone conditions, the production of gestures was either paired with the production of the target word (e.g., Boo & Rose, Citation2011) or with the production of phonological features, for example, number of syllables of the target word (e.g., Rose et al., Citation2002). Moreover, none of these studies reported higher naming accuracy when the production of gestures was paired with other verbal output (e.g., Boo & Rose, Citation2011; Raymer et al., Citation2007; Raymer et al., Citation2012; Rodriguez et al., Citation2006; Rose et al., Citation2002; Rose & Douglas, Citation2008; Rose & Sussmilch, Citation2008; cf. also Rose et al., Citation2013). Concerning the facilitative effects of gesture/action observation on verb retrieval, the results were ambiguous. While some authors report improved verb retrieval in IWA (Bonifazi et al., Citation2013; Marangolo et al., Citation2010; Marangolo et al., Citation2012), others did not observe facilitative effects of action observation on verb production (Routhier et al., Citation2015). However, all studies have in common that primarily individuals with phonological impairments benefited from gestural intervention techniques, whereas in individuals with lexical-semantic impairments, the results were inconsistent (e.g., Bonifazi et al., Citation2013; Boo & Rose, Citation2011; Marangolo et al., Citation2010; Marangolo et al., Citation2012; Raymer et al., Citation2007; Raymer et al., Citation2012; Raymer et al., Citation2006; Rodriguez et al., Citation2006; Rose & Douglas, Citation2001, Citation2008; Rose et al., Citation2002; Rose & Sussmilch, Citation2008).

Two different gesture types are discussed as possible candidates for the facilitation of word retrieval: iconic gestures and non-iconic gestures. Iconic gestures are meaningful gestures that depict concrete objects or simulate actions (e.g., McNeill, Citation1992), for example, the verb knocking is represented as moving the hand as if knocking at a door. In contrast, non-iconic gestures are meaningless gestures, that is, they do not express semantic meaning. Examples for non-iconic gestures are circular or tapping movements of the hand (e.g., McNeill, Citation1992; Ravizza, Citation2003). According to Krauss et al. (Citation2000), the semantic content of a gesture, that is, their iconicity plays a crucial role in facilitating word retrieval. The authors claim that the specific semantic features (spatial/dynamic) of a concept realised in an iconic gesture may help to facilitate word retrieval. Furthermore, embodied cognition theories reveal that sensorimotor experiences with the words’ referents play a crucial role in word processing (e.g., Aziz-Zadeh et al., Citation2006; Barsalou, Citation1999; Gallese & Lakoff, Citation2005; Hauk et al., Citation2004). For word meanings that are grounded in sensorimotor features and hence rely on sensorimotor brain areas (i.e., many verbs), gesture production/action observation should facilitate word retrieval by activating sensorimotor networks. Evidence was obtained in a study by Marangolo et al. (Citation2012) who observed improved verb naming for human actions (e.g., kicking) but not for non-human action (e.g., barking) following action observation.

However, the observation that non-iconic gestures also occur during word retrieval difficulties (e.g., Beattie & Coughlan, Citation1999; Frick-Horbury & Guttentag, Citation1998; Pine et al., Citation2007; Theocharopoulou et al., Citation2015) led to the assumption that these gestures might also have a positive impact on word retrieval. An activation and interaction of brain areas involved in the processing of both language and movement is assumed in order to explain their facilitative effect (e.g., Lucero et al., Citation2014; Ravizza, Citation2003).

With respect to IWA, it remains unclear if and how adequate iconic gestures can be performed, especially in the presence of hemiparesis or limb apraxia. According to Krauss et al. (Citation2000), the semantic content of a gesture, that is, the ability to realise a concept’s semantic features in a gesture is decisive for an improvement in word retrieval. Hence, an impaired ability to produce adequate iconic gestures could have an impact on their facilitation effects. Results from previous studies focusing on gesture production for communication purposes in IWA mostly indicate no influence of hemiparesis on the formal diversity of gestures (Hogrefe et al., Citation2012), the number of gestures per word (Kong et al., Citation2015), the use of various gestural representation techniques in pantomime production (Van Nispen et al., Citation2016), and on the comprehensibility of gestures (Hogrefe et al., Citation2012, Citation2017; Hogrefe et al., Citation2013). Solely Van Nispen et al. (Citation2018) found an influence of hemiparesis on the comprehensibility of pantomimes in IWA. Concerning limb apraxia study results were equivocal. Van Nispen et al. (Citation2016) and Van Nispen et al. (Citation2018) investigated pantomime production on command in IWA and found an effect of limb apraxia on the use of different gestural representation techniques and the comprehensibility of pantomimes. Furthermore, some studies reported an effect of limb apraxia on spontaneous gesture production in communication contexts, that is, the skillfulness and success of gestural communication (e.g., Borod et al., Citation1989; Feyereisen et al., Citation1988), the formal diversity (Hogrefe et al., Citation2012), or on the comprehensibility of gestures (Hogrefe et al., Citation2012, Citation2017; Hogrefe et al., Citation2013). However, other studies did not observe an influence of limb apraxia on spontaneous gestures production (e.g., Lausberg et al., Citation2000; Rose & Douglas, Citation2003).

The aims of the present study were (1) to investigate possible facilitative effects of iconic and non-iconic gestures on verb retrieval in IWA and (2) to examine the ability of IWA to produce adequate iconic gestures even in the presence of hemiparesis and/or limb apraxia. Based on previous results of facilitation/treatment studies, we expected to observe a facilitative effect of iconic gesture production on verb retrieval in IWA. More specifically, we assumed that participants with verb retrieval deficits that predominantly originate from an impairment at the phonological word form level would benefit from iconic gestures. This is in line with several facilitation/treatment studies reporting gestural facilitation effects mainly in individuals with phonologically based word retrieval impairments (e.g., Bonifazi et al., Citation2013; Marangolo et al., Citation2010; Marangolo et al., Citation2012; Rodriguez et al., Citation2006; Rose et al., Citation2002; Rose & Douglas, Citation2001; Rose & Sussmilch, Citation2008; but see Raymer et al., Citation2007; Raymer et al., Citation2012; Raymer et al., Citation2006; Rose & Douglas, Citation2008 for evidence of improved naming in some individuals with deficits in word retrieval originating from a lexical-semantic impairment). According to these studies, iconic gestures might facilitate word retrieval (i.e., verb retrieval) either via the semantic system by activating the sensory-motor features of multimodal semantic representations (e.g., Bonifazi et al., Citation2013; Marangolo et al., Citation2010) or by cross-modal priming at the phonological level of the word production process (e.g., Rose & Douglas, Citation2001; Rose et al., Citation2002). As little is known concerning the influence of non-iconic gesture on verb retrieval, we exploratively analysed whether non-iconic gestures are facilitative. Furthermore, the analysis of the gestural iconicity was also exploratory.

In Experiment 1, we investigated the impact of producing iconic and non-iconic gestures on verb retrieval in IWA utilising a multiple single-case across participants design. In Experiment 2, we determined the iconicity of the iconic gestures produced by the IWA in Experiment 1 by comparing the performance of each IWA with the performance of language-unimpaired participants. The study was approved by the local ethics committee of the University of Potsdam. We will first introduce the participants for each experiment followed by the description of the word and picture material. Subsequently, we provide an overview of the tasks and procedures used in Experiments 1 and 2.

2. Experiment 1: Facilitation study

2.1 Participants

Six IWA (one female, five male) aged between 59 and 78 years (M = 68.67 years, SD = 7.76) who had suffered a unilateral stroke in their language-dominant hemisphere participated in the study. They were recruited from a database of individuals with speech and language disorders maintained by the Patholinguistics/Neurocognition of Language group at the University of Potsdam and gave written informed consent to participation in the study. All individuals had normal or corrected-to-normal vision. Time post-stroke ranged between 1 and 20 years. All participants were native speakers of German, and all but one IWA were (pre-morbidly) right-handed, which was assessed by a German version of the Edinburgh Handedness Inventory (Oldfield, Citation1971). Participant P4, who suffered from a right cerebral hemorrhage, was included in the study since he showed a similar linguistic profile compared to the IWA with a left hemispheric brain injury (cf. Adelt et al., Citation2020). In four IWA, stroke resulted in hemiparesis. Namely, the four individuals were not able to use their dominant hand for daily activities such as grabbing objects, eating, or writing. Moreover, two participants demonstrated limb apraxia as determined by a pantomime-to-command task (Goldenberg et al., Citation2003; Goldenberg et al., Citation2007).

Verb production and possible underlying deficits were assessed with a verb naming and verb-picture verification task developed in the context of the study. In the oral naming task, IWA were asked to name the actions depicted in the black-and-white line drawings. In the receptive verb-picture verification task, IWA had to decide whether an orally presented verb corresponded to the action depicted in the black-and-white line drawing or not. The presented verbs were either corresponding to the picture (i.e., correct, e.g., the picture of eating was presented with the word eating) or not corresponding but semantically related to the target verb (i.e., incorrect, e.g., the picture of eating was presented with the word drinking). Thus, the black-and-white drawings were presented twice, once with the orally presented target verb and once with an orally presented semantic distractor. Responses for an item were only scored correct if the participant both accepted the correct verb and rejected the semantic distractor (cf. Breese & Hillis, Citation2004; Raymer et al., Citation2006). Tasks in both modalities included the same 34 verbs controlled for word frequency (www.dlexdb.de, Heister et al., Citation2011), transitivity (i.e., transitive, intransitive, pseudo-transitive verbs), and name relation to a noun (verb stem identical to a noun depicted in the picture).

The participants’ clinical and demographic data as well as the results of the assessments are provided in . In addition to naming deficits, participant P4 suffered from a mild apraxia of speech with errors predominantly at the segmental level (i.e., phonemic/phonetic errors) which was assessed using a German word and non-word repetition test for the assessment and diagnosis of apraxia of speech (Hierarchische Wortlisten, HWL; Liepold et al., Citation2003). Participant P5 showed impaired semantic knowledge indicated by the results of subtests 1–3 of the Bogenhausener Semantik-Untersuchung (BOSU; Glindemann et al., Citation2002), a German odd-one-out task to test semantic processing abilities.

Table 1. Demographic data and assessment results of individuals with aphasia (P1–P6)

2.2 Material

A total of 75 verbs were selected and divided into three sets of 25 items each. The items in each set were balanced for linguistic/psycholinguistic variables known to affect verb retrieval: word length (number of syllables, phonemes, phonemes in the stem), transitivity (three subgroups: transitive, intransitive, pseudo-transitive verbs), instrumentality (two subgroups: actions requiring/not requiring a man-made instrument), name relation to the instrument (not fully homophonous in German: verb = noun + n, e.g., Säge (noun: a saw) – sägen (verb: to saw)), word frequency (logarithmic normalised lemma frequency, determined using the German dlexDB database (Heister et al., Citation2011), age of acquisition, concept familiarity, imageability (all three gathered empirically in three different rating studies with 20 elderly language-unimpaired people each), and naming agreement. provides an overview of the matched linguistic/psycholinguistic variables in the three sets of verbs (means/standard deviations).

Table 2. Means and standard deviations (SD) of matched linguistic/psycholinguistic variables in the three sets of verbs

Black-and-white line drawings for all verbs with a mean naming agreement of 95,39% (obtained from 69 language-unimpaired young adults) were drawn for the study. Sample pictures are provided in . The order of the items in the sets was pseudo-randomised.

Figure 1. Sample pictures for verbs used in Experiment 1

Figure 1. Sample pictures for verbs used in Experiment 1

2.3 Procedure

Pre-facilitation assessment: Naming without gesture. IWA initially named all 75 pictures without producing a gesture, in order to determine error rates prior to facilitation. Target pictures were presented for 10 seconds on a laptop screen and participants were required to name in one word the depicted actions. The participants were prevented from gesturing by wearing oven mitts which were fixed on a Styrofoam board using hook-and-loop tapes (cf. Pine et al., Citation2007). The task started with five practice items involving corrective feedback, whereas no feedback was provided during the experimental trials.

Facilitation. The facilitation phase consisted of naming all 75 pictures in three different conditions (25 items in each condition) (see ). For each naming attempt, pictures were presented for 10 seconds. In a first attempt, IWA were required to name a target picture without gesture production. Similar to the pre-facilitation procedure, participants were prevented from gesturing with the fixed oven mitts. In case of an incorrect response, the target picture was displayed again and participants were asked to either produce a specific gesture (iconic/non-iconic) or no gesture at all before oral naming (see facilitation conditions in ). In the iconic gesture condition, participants were encouraged to produce a gesture depicting the action in the picture before naming. In the non-iconic gesture condition, participants were required to produce a tapping gesture with their fingers on the table prior to naming. Participants used one or both hands for gesture production. In the no gesture condition, the participants’ hands remained in the oven mitts while naming the action of the target picture. In total, each participant attended three sessions administered with an interval of approximately 7 days each with another facilitation condition (iconic gesture, non-iconic gesture, no gesture).

Figure 2. Procedure of Experiment 1: Facilitation Study

Figure 2. Procedure of Experiment 1: Facilitation Study

In order to reduce order effects, sequence of conditions and order of item sets was systematically varied within and across participants. Similar to the pre-facilitation assessment, the experiment started with five practice trials, where participants were familiarised with the procedure. The study design is depicted schematically in .

Naming responses and gestures were videotaped for later analysis. The responses were coded by the first author, a speech and language therapist, who also conducted the pre-facilitation assessment and the facilitation sessions. Responses were scored as correct only if the participants produced the target verb (either as a single word or as part of a sentence), whereby infinite as well as finite verb forms (which were mainly first- or third-person singular) were taken into account. Moreover, responses including phonetic distortions were accepted as correct. Other responses (e.g., synonyms, circumlocutions, semantic/morphological/phonemic/formal paraphasias, onomatopoeia, neologisms, or no responses) were treated as errors. Synonyms were assessed as incorrect since they can be used to compensate for word retrieval difficulties. However, IWA produced very few synonyms in all three naming conditions. The data from all six IWA for the second naming attempt in each condition were plotted as bar charts (see ) and analysed using descriptive statistics. Furthermore, we used McNemar’s test (two-tailed) to investigate whether the baseline was stable (pre-facilitation assessment/first naming attempt during facilitation) and Fisher’s exact test (two-tailed) to analyse whether any of the conditions led to significantly enhanced naming performance in the participants.

Figure 3. Number of items retrieved (white)/not retrieved (grey) with iconic gesture, non-iconic gesture, and no gesture in the second naming attempt for each IWA (P1-P6).

Notes: The figure displays the results of the second naming attempts in all three conditions. Only items that could not be named in the first naming attempt were entered in the second naming attempt, e.g., P1: iconic gesture condition: n = 15 items, non-iconic gesture condition: n = 13 items, no gesture condition: n = 19 items; Lex.-sem. = lexical-semantic naming impairment, Phon. = phonological naming impairment, Lex.-sem./phon. = lexical-semantic/phonological naming impairment

Figure 3. Number of items retrieved (white)/not retrieved (grey) with iconic gesture, non-iconic gesture, and no gesture in the second naming attempt for each IWA (P1-P6).Notes: The figure displays the results of the second naming attempts in all three conditions. Only items that could not be named in the first naming attempt were entered in the second naming attempt, e.g., P1: iconic gesture condition: n = 15 items, non-iconic gesture condition: n = 13 items, no gesture condition: n = 19 items; Lex.-sem. = lexical-semantic naming impairment, Phon. = phonological naming impairment, Lex.-sem./phon. = lexical-semantic/phonological naming impairment

2.4 Results

All but one IWA showed stable naming performance without gesture production in the pre-facilitation assessment and the first naming attempt without gesture production in the facilitation sessions in all three conditions and throughout all verb sets (all but P1: p > .05, two-tailed, McNemar’s exact). For P1, the performance differed only in the non-iconic gesture condition (p = .008, two-tailed, McNemar’s exact). This can be taken as an indication of a rather stable baseline.

Analysis of naming performance in the first naming attempt revealed differential severity of verb retrieval deficits for the IWA. A mild verb naming impairment was assumed when the participants were able to name more than 50% of the verbs in the first naming attempt, which was true for P2, P3, P4, and P6 (all conditions). A moderate verb naming impairment was indicated by performances between 25% and 50% correctly retrieved verb names (P1, iconic and non-iconic gesture condition), and a severe verb naming impairment was assumed when less than 25% of the verb were retrieved correctly in the first naming attempt (P5, all conditions; P1, no gesture condition).

Concerning facilitation (second naming attempt for those items that could not be retrieved during the first naming attempt), visual inspection of the graphed data (second naming attempt) revealed only minor benefits from all three facilitation conditions with merely minimal differences between the conditions for all six IWA (see ). For none of the participants, significant facilitation effects of iconic gesture, non-iconic gesture or no gesture were observed in any of the three conditions (all comparisons: p > .05, two-tailed, Fisher’s exact). Hence, the underlying functional deficit of the participants (i.e., phonological, lexical-semantic, or combined deficit) was not considered in detail.

In all three naming conditions, a total of 25 pictures was presented in the first naming attempt, but only items that could not be named in the first naming attempt, were entered in the second naming attempt. For this reason, the number of items in the second naming attempt differed for each IWA and in each condition. For example, participant P1 was unable to name 15 pictures in the iconic gesture condition, 13 pictures in the non-iconic gesture condition and 19 pictures in the no gesture condition in the first naming attempt. These items were entered in the second naming attempt. With iconic gestures, P1 was able to name four pictures in the second naming attempt, but 11 pictures were not named correctly. In the non-iconic gesture condition two verbs were retrieved correctly, 11 verbs were not retrieved and in the no gesture condition only one picture was named correctly. The data revealed that the number of correctly retrieved verbs in all three conditions (with iconic gesture, non-iconic gesture, or no gesture) in the second naming attempt was quite low in all six participants.

2.5. Interim summary

Taken together, the results of Experiment 1 indicate that the production of a gesture (iconic or non-iconic) in our controlled setting did not result in better naming performance in IWA. This finding contradicts the findings of several facilitation/treatment studies that demonstrated beneficial effects for gestural intervention techniques in IWA. These will be discussed in Section 4. According to the LRH (Krauss et al., Citation2000) the semantic features (spatial/dynamic) of a concept realised in a gesture may help to facilitate word retrieval. This raised the question whether the IWA were able to produce adequate iconic gestures, since some individuals suffered from hemiparesis (P1, P3, P4, P5) and/or limb apraxia (P1, P5). Therefore, we conducted a second experiment in which the iconic gestures produced by the IWA were rated for their iconicity.

3. Experiment 2: Rating study

The results of Experiment 1 revealed that IWA did not benefit from gestures production to facilitate verb retrieval in a controlled setting. Thus, the aim of Experiment 2 was to investigate whether the IWA were able to produce adequate iconic gestures despite the presence of hemiparesis and/or limb apraxia, and thus, whether missing facilitation effects may be attributed to an inability to realise semantic features (spatial/dynamic) of a concept in an iconic gesture.

3.1 Participants

In order to compare the iconic gestures of the IWA (i.e., the gestures produced in the iconic gesture condition during the facilitation sessions) to iconic gestures produced by healthy controls, 12 language-unimpaired adults (six female, six male) ranging between 60 and 76 years of age (M = 68.67 years, SD = 6.14) participated as performers in the rating study. They were recruited via direct contact with potential study participants and gave written informed consent to participate in the study. All performers were native speakers of German and had normal or corrected-to-normal vision. According to a German version of the Edinburgh Handedness Inventory (Oldfield, Citation1971) all participants were right-handed. None of the subjects reported any history of neurological or language disorders and none of them had limitations of hand or arm movements. Half of the participants (n = 6; three female, three male; Mage = 68.67 years, range = 60–75 years) produced the iconic gestures with both hands, while the other half of the participants (n = 6; three female, three male; Mage = 68.67 years, range = 61–76 years) produced the iconic gestures with their left (non-dominant) hand.

The iconicity ratings were obtained from 30 healthy female individuals aged between 18 and 45 years (Mage = 21.83 years, SD = 5.11). Twenty-eight participants were monolingual speakers of German, while two participants were bilingual speakers of German-Polish or German-Croatian. They were students of Patholinguistics at the University of Potsdam and received course credit for their participation. The participants were divided into six groups of five raters, who rated a sample of videotaped gestures each (for details, see later).

3.2 Material

Gesture production by language-unimpaired participants. The same 75 black-and-white line drawings that were presented in the facilitation study were also used for the gesture production task. They were divided into three blocks of 25 pictures and presented to the language-unimpaired participants in a pseudo-randomised order. The items in the blocks corresponded to the item sets presented to the IWA in the facilitation study.

Rating. For the rating study, the stimuli consisted of videos of the iconic gestures, that the IWA produced in the facilitation study (n = 65) and the iconic gestures of the 12 language-unimpaired participants, specifically produced for the rating study (n = 612). The IWAs’ gestures were those produced in the iconic gesture condition during facilitation. Note, that the IWA produced iconic gestures only for those items that they were not able to retrieve. The gestures produced in the non-iconic gesture condition were not presented for rating since all IWA produced the same tapping gesture. In order to limit the time for rating, the six groups of five raters were presented with 167 gestures each, resulting in approximately 1 hour per rater. The gestures of the IWA were presented in all six groups. Furthermore, the gestures of three language-unimpaired participants (both hands) and three language-unimpaired participants (left hand only) from one item set each were assigned pseudo-randomly to the six groups. Gestures of IWA and language-unimpaired adults were presented in a pseudo-randomised order with no consecutive gestures for the same items produced by different participants and no consecutive gestures for different items produced by the same participants. Furthermore, no more than four consecutive gestures produced by one of the language-unimpaired participants or one of the IWA were presented.

3.3 Procedure

Gesture production by language-unimpaired participants. Two groups of language-unimpaired participants were requested to produce iconic gestures for each target picture and to name the picture. The naming responses were not analysed. They were included in order to make the experiment similar to the demands made on the IWA in Experiment 1. The pictures were presented for 10 seconds on a laptop screen. Whereas one group of language-unimpaired participants used both hands, the other group was instructed to perform the gestures only with their left (non-dominant) hand as the right hand was fixed in an oven mitt. The gesture production task started with five practice items and all gestures and naming responses were videotaped. For the iconicity rating, we selected only the gestures for those items that the IWA produced in the iconic gesture condition in the facilitation study.

Rating. In the gesture rating study, language-unimpaired participants were asked to judge the iconicity of the gestures on a 5-point scale from 1 (non-iconic) to 5 (very iconic). The rating task started with videotaped examples of a very iconic and a non-iconic gesture for the verb drinking. The very iconic gesture looked like holding an imaginary glass, moving it upwards, and tilting it near the mouth. As example for a non-iconic gesture, we chose the tapping gesture that the IWA were asked to produce in the facilitation study in the non-iconic gesture condition. The examples were produced by a language-unimpaired person who was not otherwise involved in the experiments. For each gesture, the participants were first shown the black-and-white drawing of the action together with the written verb for 5 seconds. After watching the video-recording of the corresponding gesture, the participants were given 10 seconds to mark their answer on a scoring sheet. Short 1-minute breaks were provided every 56 items. The raters were blind to group membership of the participants. The instructions for the iconicity rating were based largely on those used by Vinson et al. (Citation2008) in a study investigating iconicity as a linguistic factor for a set of lexical signs from British Sign Language.

For the analysis of ratings, all but one answers were considered. One answer was excluded because the rater marked two numbers of the scale. For each gesture, the mean iconicity ratings were calculated for each IWA and both groups of language-unimpaired participants (non-dominant hand gesture/both hand gesture). We compared the iconicity ratings for each IWA with the mean iconicity ratings for both groups of language-unimpaired participants using Singlims_ES, a modified independent samples t-test. The modification of the independent samples t-test takes into account the IWA’s score, the mean and standard deviation of the control sample as well as the size of the control group (Crawford & Garthwaite, Citation2002, Citation2007; Crawford et al., Citation2010). We report (1) whether the iconicity ratings differed significantly between the IWA and the language-unimpaired participants (non-dominant hand gesture/both hand gesture) and (2) the estimated percentage of normal population (i.e., language-unimpaired participants [non-dominant hand gesture/both hand gesture]) falling below the individual’s (i.e., IWA’s) score (for a similar analysis of results, see, e.g., Akhavan et al., Citation2018).

3.4 Results

As described earlier, the gestures were rated on a 5-point scale from 1 (non-iconic) to 5 (very iconic), and for all gestures, the mean iconicity ratings were calculated for the IWA and both groups of language-unimpaired participants (non-dominant hand gesture/both hand gesture). The number of iconic gestures differed in each IWA, since the participants only produced gestures for the pictures they could not name and that were entered to the second naming attempt in the iconic gesture condition in Experiment 1. For example, P1 produced 15 iconic gestures since she was not able to name 15 pictures in the first naming attempt in Experiment 1.

The gestures of all but one IWA (P5) were rated similar to the gestures of the control participants despite the presence of hemiparesis and/or limb apraxia (P1, P3, P4) (all comparisons: p > .05, two-tailed, Singlims_ES). We further examined the estimated percentage of normal population (i.e., language-unimpaired participants [non-dominant hand gesture/both hand gesture]) falling below the individual’s (i.e., IWA’s) score. As depicted in , for all but one IWA (P5) we found a substantial percentage of language-unimpaired participants (non-dominant hand gesture/both hand gesture) would have a lower iconicity rating than the IWA. Thus, we conclude that the IWA, except P5, don’t have an impairment in adequate iconic gesture production. For a visualisation of all six IWAs’ iconicity ratings compared to both control groups (both hand gesture/non-dominant hand gesture) each, see Figure A in the supplementary material.

Table 3. Overview of single-case statistics for iconicity ratings in IWA and language-unimpaired controls

P5 differed from the other IWA in that he produced predominantly pointing gestures with sparse iconic components and only few iconic gestures (e.g., milking: up-and-down movement with the stretched thumb and index finger of the left hand; stirring: stretched index finger with a slanting downward orientation and production of a circular movement). Whereas mean iconicity ratings for participant P5 did not differ significantly from the control group who produced the iconic gestures only with their left (non-dominant) hand (p > .05, two-tailed, Singlims_ES), they were rated significantly less iconic than those of the control group who could use both hands for gesture production (t = −2.958, p = .032, two-tailed, z-cc = −3.195 (CI = −5.259 to −1.112)). Furthermore, the estimated percentage of language-unimpaired participants (non-dominant hand gesture/both hand gesture) falling below P5’s score was 3.28% (non-dominant hand gesture) and 1.58% (both hand gesture) respectively. Note that he suffered from a hemiparesis, a limb apraxia, and an impaired semantic knowledge (according to BOSU, Glindemann et al., Citation2002).

3.5 Interim summary

In sum, iconicity was not affected by hemiparesis and/or limb apraxia. These results are in line with previous studies that did not find an influence of hemiparesis (e.g., Hogrefe et al., Citation2012, Citation2017; Hogrefe et al., Citation2013; Kong et al., Citation2015; Van Nispen et al., Citation2016) or limb apraxia (e.g., Lausberg et al., Citation2000; Rose & Douglas, Citation2003) on gesture production for communication purposes in IWA. However, impaired semantic knowledge seemed to have an influence on iconic gesture production. This assumption will be discussed in detail later. Since all but one IWA were able to produce adequate iconic gestures, the absence of a facilitation effect of gesture production on verb retrieval cannot be attributed to an inability of the IWA to produce iconic gestures.

4. Discussion

The overall aim of the current study was to examine the effects of iconic and non-iconic gesture production on verb retrieval in IWA within a multiple single-case facilitation study (Experiment 1). Furthermore, we investigated the iconicity of the iconic gestures produced by IWA within a rating study (Experiment 2) to determine whether and how the existence of a hemiparesis and/or limb apraxia influences the adequate production of iconic gestures. To summarise, in our controlled setting (i.e., the facilitation study) none of the IWA benefited from the production of a gesture in order to facilitate oral verb naming. Thus, the elicitation of a gesture did not result in better naming performance. This is not in line with results of previous facilitation/treatment studies utilising iconic and/or non-iconic gestures as cues to facilitate word retrieval in IWA (e.g., Bonifazi et al., Citation2013; Boo & Rose, Citation2011; Crosson et al., Citation2007; Crosson et al., Citation2005; Ferguson et al., Citation2012; Marangolo et al., Citation2010; Marangolo et al., Citation2012; Raymer et al., Citation2007; Raymer et al., Citation2012; Raymer et al., Citation2006; Richards et al., Citation2002; Rodriguez et al., Citation2006; Rose & Douglas, Citation2001, Citation2008; Rose et al., Citation2002; Rose & Sussmilch, Citation2008; Routhier et al., Citation2015). However, since most studies that used gesture production to facilitate word retrieval (except Rose & Douglas, Citation2001) combined gesture with linguistic treatment techniques, e.g., phonologic/semantic cues or word repetition, the results are not fully comparable to the current study. The actual role of gesture in word retrieval remains unclear in those studies since improvements might result from verbal cues, e.g., phonological/semantic cues or word repetition (cf. also Marshall, Citation2006; Rose et al., Citation2013).

However, in their facilitation study, Rose and Douglas (Citation2001) report significant facilitation effects of iconic gesture production on noun retrieval in three participants with phonologically based word retrieval impairments. Similar to our facilitation study, an improvement in word retrieval was measured immediately after the production of an iconic gesture. Yet, not the same word classes were examined. While Rose and Douglas (Citation2001) were interested in an improvement of noun retrieval difficulties, our facilitation study focused on verb retrieval impairments. Verbs are more complex than nouns with regard to their linguistic properties (e.g., their more complex conceptual-semantic representations and their role in sentence production) (e.g., Black & Chiat, Citation2003). Hence, nouns and verbs differ in terms of their processing demands at the semantic, morphological, and syntactic levels. Furthermore, nouns are organised in multilevel hierarchies (taxonomic structures), whereas verbs are linked in manner relations (troponymy) (e.g., Fellbaum, Citation1990; Huttenlocher & Lui, Citation1979). Additionally, verbs are of lower imageability than nouns (Bird et al., Citation2000). Therefore, it has been suggested that verb processing is more demanding in terms of executive resources. Evidence has been reported in studies focusing on verb processing in people with neurodegenerative disorders, particularly in people with frontotemporal dementia (e.g., Rhee et al., Citation2001; Silveri et al., Citation2003; cf. also Vigliocco et al., Citation2011). The retrieval of verbs with iconic/non-iconic gestures might hence be more difficult to achieve compared to nouns. This is in line with the results of several treatment studies employing various single-word methods to improve verb retrieval that are usually used to treat noun retrieval impairments (for a review, see Webster & Whitworth, Citation2012).

Moreover, there was a difference between the presentation time of the pictures-to-be-named, which was 10 seconds in our study and 20 seconds in Rose & Douglas’ study. The time limit comprised gesture and word production. Several studies found an influence of processing speed on IWAs’ performance with respect to accuracy in different linguistic tasks (e.g., McCall et al., Citation1997; Neto & Santos, Citation2012). Bearing this in mind, one could assume that a longer presentation time of the pictures possibly influenced the facilitative effect of iconic gesture production.

Furthermore, in treatment studies, a specific training is applied several times to a single item whereas it was only applied once in the current facilitation study. Concerning a gestural facilitation effect, Marshall (Citation2006) emphasises the importance of a close relationship between the gesture and the word whose retrieval should be supported. Thus, repeated exposure to gestures in treatment could possibly support the connection between a gesture and a word form. This should be borne in mind by speech and language therapists when applying gesture as a treatment technique and should be further investigated in future studies.

Moreover, some researchers critically discuss the assumptions of the LRH and argue that iconic gestures are often associated with phrases instead of single words (e.g., De Ruiter, Citation2006; De Ruiter & De Beer, Citation2013). As an example, they give the clause “rolling down the hill”, which is often accompanied by a gesture that comprises the meaning of the whole clause (e.g., a circular and downward movement of the hand). The authors question that these gestures can facilitate the retrieval of a single word since they are linked to whole clauses. Moreover, concerning the treatment of verb retrieval impairments, some researchers argue that verbs should be treated in sentence contexts since they play a central role in syntactic processing (e.g., Conroy et al., Citation2009; Edwards & Tucker, Citation2006). Future studies investigating the possible facilitative effects of gesture production on verb retrieval should include these aspects in their methodological considerations.

Furthermore, the severity of verb retrieval impairments could have influenced the absence of significant gestural facilitation effects. Four IWA (P2, P3, P4, P6) were still able to name a number of verbs during the first naming attempt, so the potential gestural facilitation effects were investigated only for a small number of items. One participant (P5) was severely impaired in verb retrieval. Raymer et al. (Citation2006) and Rodriguez et al. (Citation2006) found that participants with severe word retrieval deficits did not benefit from gestural facilitation.

Moreover, it must be borne in mind that iconic/non-iconic gestures can have a multifunctional role in IWA. Concerning the production of meaningful gestures, several studies attribute to iconic/deictic gesture types a compensatory role in order to convey information (e.g., Akbıyık et al., Citation2018; Akhavan et al., Citation2018; De Beer et al., Citation2017; Hogrefe et al., Citation2012; Hogrefe et al., Citation2013; Kemmerer et al., Citation2007; Özer et al., Citation2019; Pritchard et al., Citation2015; Van Nispen et al., Citation2017). Moreover, IWA produce non-iconic gestures (e.g., hands palm-up to indicate uncertainty) as social cues (e.g., Akhavan et al., Citation2018). Concerning gestural facilitation effects in sentential or narrative contexts, the results reported in the literature remain unclear. As in the case of single-word retrieval, some studies report that IWA benefited from gesture production (e.g., Akhavan et al., Citation2018; Kistner et al., Citation2019; Lanyon & Rose, Citation2009; Özer et al., Citation2019), while other studies could not find gestural facilitation effects (e.g., Cocks et al., Citation2011; Cocks et al., Citation2013; Pritchard et al., Citation2013).

Two more methodological considerations of the current study require further discussion. First, the IWA were prevented from gesturing by wearing oven mitts fixed on a Styrofoam board. The purpose of this approach was to allow for a direct comparison of verb naming performance with and without gesture. Several researchers criticise this procedure since prevention from gesture production is a very unnatural condition, which could possibly be cognitively demanding and thus could affect word production (e.g., De Ruiter & De Beer, Citation2013; Lucero et al., Citation2014). Moreover, when recording Electromyographic (EMG) signals, Morsella and Krauss (Citation2005) found muscular activity of the dominant forearm even without visible gesture production when language-unimpaired participants were asked to retrieve words from definition. Thus, we cannot exclude that our procedure just led to a restriction of overt gesture production, while invisible muscular activity possibly had an influence on verb retrieval in the IWA. Second, different demands were made concerning iconic and non-iconic gesture production, which led to a reduced comparability between the two conditions. While the IWA were required to produce iconic gestures spontaneously in response to a picture, the non-iconic gestures were predetermined and fixed tapping movements on the table. The request to spontaneously produce an iconic gesture and to retrieve the appropriate verb was a dual task, which may have led to an increased cognitive load which interfered with a successful verb retrieval. Studies focusing on gesture production in narration tasks in IWA observed that verb finding difficulties were frequently accompanied by path gestures, that are semantically light gestures solely describing the direction but not the manner of a movement (Cocks et al., Citation2013; Pritchard et al., Citation2013). Thus, one can assume that the production of semantically detailed iconic gestures becomes difficult when verb retrieval problems occur. Furthermore, as mentioned earlier, Marshall (Citation2006) argues that a close relationship between an iconic gesture and the corresponding word might be necessary in order to assist its retrieval. Concerning non-iconic gestures, individual rather than fixed non-iconic gestures could possibly help to alleviate verb retrieval problems, which should be further investigated in future studies.

Furthermore, the current study sought to address, to what extent IWA can perform adequate iconic gestures, especially in the presence of hemiparesis and/or limb apraxia. Since the realisation of a concept’s semantic features in a gesture is relevant to improve word retrieval, an impaired ability to produce adequate iconic gestures could impact their facilitation effects. In order to determine the iconicity of the iconic gestures produced by the IWA, their gestures were compared to gestures produced by language-unimpaired controls (non-dominant hand gesture/both hand gesture) in terms of their iconicity.

Overall, the mean iconicity ratings for all but one IWA (P5) did not differ significantly from those for both control groups. Hemiparesis and/or limb apraxia did not affect iconic gesture production in participants P1, P3, and P4. These results are in line with previous studies that did not find an influence of hemiparesis (e.g., Hogrefe et al., Citation2012, Citation2017; Hogrefe et al., Citation2013; Kong et al., Citation2015; Van Nispen et al., Citation2016) or limb apraxia (e.g., Lausberg et al., Citation2000; Rose & Douglas, Citation2003) on gesture production for communication purposes in IWA. Only in P5 the mean iconicity ratings differed significantly from the control group (both hand gesture). In addition to hemiparesis and limb apraxia, participant P5 suffered from impaired semantic knowledge (according to BOSU, Glindemann et al., Citation2002) which probably affected iconic gesture production. An influence of impaired semantic knowledge on spontaneous gesture and pantomime production was found in several studies (e.g., Cocks et al., Citation2013; Hogrefe et al., Citation2012, Citation2017; Hogrefe et al., Citation2013; Van Nispen et al., Citation2016). Since we did not have information concerning the semantic processing abilities (e.g., BOSU results) of the other participants in our study and since we cannot exclude, that P5’s gesture production was influenced by hemiparesis and/or limb apraxia, the probable relation of his semantic knowledge impairment (according to BOSU, Glindemann et al., Citation2002) and iconic gesture production must be interpreted with caution. However, since P5 was the only participant whose iconic gestures differed significantly from those produced by the unrestricted language-unimpaired controls (both hand gesture) and since the presence of a hemiparesis and/or limb apraxia did not influence the iconicity of the gestures produced by the other IWA in our study, this interpretation was taken into account.

Future studies should probably consider other more specific methods in order to determine gesture iconicity (e.g., the description of the relationship between the form of a gesture and its meaning [see sign language research: Bellugi & Klima, Citation1976; Griffith et al., Citation1981] or the determination (presence/absence) of the realised semantic features in a gesture [see pantomime-to-command task: Goldenberg et al., Citation2003; Goldenberg et al., Citation2007, or Beattie & Shovelton, Citation2001 for some other semantic features]).

One shortcoming of our study is that the responses were not scored by a second coder, neither in the facilitation study nor in the rating study. Thus, we could not control for blindness of the coders to the conditions (iconic gesture, non-iconic gesture, no gesture) nor could we measure interrater reliability. This should be considered in future studies.

To conclude, although the rated iconicity for all but one of the IWA did not differ significantly from both control groups, iconic gesture production did not lead to facilitation effects on verb retrieval. Thus, the absence of a facilitation effect cannot be attributed to an inability of the IWA to produce iconic gestures. The participants, except P5, were able to depict iconically the items that they could not verbalise, that is, they realised several semantic features of the items in their gestures. According to the LRH, the semantic features realised in iconic gestures should facilitate word retrieval. This assumption was not supported in our study. Furthermore, also non-iconic gesture production did not lead to an alleviation of word retrieval in IWA. Due to the small number of participants, the results of our study must be interpreted with caution and cannot be easily generalised to a larger population. Therefore, future research with more participants should elaborate on a possible relationship between the facilitation of verb production and the iconicity of the produced gestures.

Disclosure of interest

The authors report no conflict of interest.

Supplemental material

Supplemental Material

Download MS Word (334.7 KB)

Supplemental Material

Download MS Word (196.1 KB)

APH-PA_20-155-File008.docx

Download MS Word (412.7 KB)

Acknowledgments

The first author was funded by a PhD scholarship of the University of Potsdam. We would like to thank all participants who agreed to participate in the study. We are grateful to A. Wenglarczyk for drawing the pictures. Furthermore, we thank C. De Beer, R. De Bleser, E. Fleischhauer, J. Heide, and A. Lorenz for their advice and support during the preparation of the study. Moreover, we thank S. Hanne for her help with the statistical analysis of the data. Furthermore, we are grateful for the comments of two anonymous reviewers.

Supplementary data

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was supported by the Universität Potsdam [PhD scholarship].

References