Lost in translation? Issues of content validity in interpreter-mediated aphasia assessments

Abstract

In many parts of the world, speech-language pathologists (SLPs) are frequently called upon to assess aphasia in bilingual speakers, or in speakers of languages of which they have little or no knowledge. One of the strategies that SLPs employ in these situations is to involve an interpreter in the assessment process. Three authentic interpreter-mediated aphasia assessments were analysed for the present study, which aimed to determine the degree to which the content validity of the individual tests was compromised in the process of their administration through an interpreter. Findings reveal that content validity was frequently weakened either at the point of administration of the test or at the point at which responses were reported back by the interpreter to the SLP. Based on these findings, it is argued that the conduct of interpreter-mediated aphasia assessments needs to be fundamentally re-thought to take account of the limitations inherent in the interpreting process. To this end, this study presents a number of practical recommendations for the involvement of interpreters in aphasia assessments, with a view to making optimal use of existing assessment materials and enhancing the quality of diagnostic information to emerge from such clinical sessions.

Keywords:

• Aphasia
• bilingualism
• interpreter
• assessment

Introduction

Linguistic diversity and the assessment of aphasia

In a linguistically diverse world, clinicians are often faced with a situation in which they need to assess a person with aphasia who speaks a language (or languages) that the clinician neither speaks nor understands. Indeed, it is a situation that would necessarily arise in any part of the world where the languages spoken in the community at large are not all represented within the speech-language pathologist (SLP) population in that region. Research has demonstrated that this is a particularly common occurrence in countries with high levels of immigration, including the US (Faroqi-Shah, Frymark, Mullen, & Wang, 2010; Lorenzen & Murray, 2008), Canada (Roberts, 1998), Australia (Clark, 1998; Roger, Code, & Sheard, 2000; Whitworth & Sjardin, 1993), and the UK (Croft, Marshall, Pring, & Hardwick, 2010). Although tools for assessing aphasia are becoming available in an increasing number of languages, the question of precisely how clinicians can use these tools to assess aphasia in a language that they do not speak is seldom addressed in the research literature.

In published case studies of bilingual aphasia, language function has often been assessed by a bilingual clinician (e.g., Roberts & Deslauriers, 1999), by separate clinicians, each of whom are fluent in at least one of the aphasic speaker's languages (e.g., Dronkers, Yamasaki, Ross, & White, 1995; Goral, Levy, & Kastl, 2010), or by a combination of bilingual and monolingual SLPs (e.g., Ansaldo, Saidi, & Ruiz, 2010). In some reported studies, it is not specified whether or not the examiners/raters were clinically trained (e.g., Stadie, Springer, de Bleser, & Bürk, 1995) and in other cases the language and professional background of those administering the aphasia tests is not specified (e.g., Junqué, Vendrell, & Vendrell, 1995; Sasanuma & Park, 1995).

When one looks at published studies of bilingual aphasia, or studies of aphasia in languages other than English and frequently-studied European languages, one finds that the “standard” for assessment is one which is conducted by a fluent speaker of the language in question, using a test that has been specifically developed or adapted for use in the language in question. Baker (1993) and Whitworth and Sjardin (1993) argue that the assessment of aphasia in speakers of community languages by bilingual SLPs, if they were available, would be ideal, and preferable to the use of interpreters. Fabbro (2001) maintains that the focus of the assessment and the clinical manifestations of the individual's aphasia can determine who should carry out the assessment, with cases of pathological language mixing or switching being assessed by a person with knowledge of only one of the patient's languages, and translation abilities being assessed by one who is fluent in both languages.

In countries such as Canada, Belgium, Switzerland, Wales, and Singapore, where two or more languages have an official status and are each represented by a significant number of speakers, SLPs with proficiency in each (or sometimes all) of these languages are more likely to be available. In countries that receive immigrants from a wide variety of different source countries, however, “community languages” are likely to be under-represented (and sometimes not represented at all) in the local SLP profession. In Australia, for example, ∼200 languages and dialects are spoken, and, while English is the national language, 21% of Australian residents spoke languages other than (or in addition to) English at home according to 2006 census data (Australian Government Department of Foreign Affairs and Trade, 2008). This means that the assessment of aphasia in speakers of these languages by trained clinicians is often simply not an option.

In the literature on aphasia assessment, the importance of assessing all of the languages spoken by an individual with aphasia is frequently stressed (Fabbro, 2001; Lorenzen & Murray, 2008; Paradis, 1995; 2004). This principle would naturally apply to individuals who are bilingual or multilingual, but also to individuals who are unilingual in a language not spoken by the SLP who is charged with the responsibility of carrying out the assessment and planning a rehabilitation program. Using a test that has been developed or especially adapted for use in a particular language would seem to be the most effective way of carrying out such an assessment, and a variety of tests are available in an increasing number of languages. The Bilingual Aphasia Test (BAT) (Paradis & Libben, 1987) has particular appeal, as it allows the assessment of two or more languages using parallel instruments, and a particular strength is that the test stimuli have been designed to take account of structural differences between languages (rather than being mere translation equivalents) as well as being adjusted for cultural appropriateness.

Making use of these valuable resources is, however, often problematic. Where a clinician with proficiency in the client's language is not available, conducting the assessment will inevitably require the assistance of a third person, who may be a professional interpreter, an interpreter without professional training or certification, or simply a speaker of the language in question. Using the BAT or another test developed for the language in question will mean that the SLP must “hand over” the assessment to the assistant, which they may be reluctant to do, particularly if this person is a friend or family member with affective ties to the person with aphasia, or a member of the local “language community” unfamiliar with the need for confidentiality and discretion in health consultations. On the other hand, if a professional interpreter has been engaged, this person would naturally expect to “interpret” for the SLP and client; being asked to administer a series of aphasia tests that have already been translated may fall well outside their professional role and potentially place them in breach of their ethical code of conduct. This may explain why SLPs often choose to assess aphasia with English language tests that are interpreted during the course of the session itself.

Interpreters in aphasia assessments

A questionnaire survey of SLPs practising in metropolitan Sydney, Australia (Roger et al., 2000) revealed the almost universal (97%) involvement of professional interpreters in the assessment of aphasia in languages not spoken by the assessing clinicians. It is only in the past two decades that the issue of “interpreting for speech-language pathology”, or (from the other perspective) “assessing language through an interpreter” has received attention in the research literature. One possible reason for this is that professional interpreters have been (and still are) non-existent or unavailable for this type of work in many, perhaps most, countries of the world. In any case, the qualifications and level of expertise (linguistic, professional, and ethical) of the person helping to conduct the assessment should, we would argue, be important considerations.

In Australia, healthcare interpreters are accredited by the National Association for Accreditation of Translators and Interpreters (NAATI). The professional association, Speech Pathology Australia (n.d.) takes the position that professional, accredited interpreters and translators should be engaged whenever possible when working with clients from culturally and linguistically diverse backgrounds. In the UK, the Royal College of Speech and Language Therapists (2009) recommends the use of interpreters where this is necessary to ensure accurate and reliable assessment and to facilitate the implementation of treatment. According to Hwa-Froelich and Westby (2003), there are no uniform national or state certification guidelines for such interpreters in the US; the American Speech-Language-Hearing Association (2004) suggests that SLPs need to ensure that the interpreters with whom they work are competent in relevant areas.

There is now a growing recognition that the role of the interpreter when working with a SLP to assist in the assessment or treatment of aphasia is fundamentally different from the role of the interpreter in other settings (Clark, 1998; Frey, Roberts-Smith, & Bessell-Browne, 1990; Gentile, Ozolins, & Vasilakakos, 1996; Isaac, 2002; Isaac & Hand, 1996; Roger et al., 2000). The “normative” role of an interpreter is to facilitate communication between the parties present by rendering accurately and completely into the other language everything that is said during the encounter. Many of the difficulties that some SLPs report when working with interpreters are not surprising considering the additional expectations that the SLP has of the interpreter. Broadly, these expectations involve the provision of a detailed description of aspects of a patient's language, in addition to a rendition (where this is possible) of what is said into the language that the therapist can understand. There is also frequently an expectation that the interpreter will alert the SLP when ‘test’ questions will not work in the other language (Isaac, 2002; Isaac & Hand, 1997).

The view that an interpreter can be an entirely impartial and objective “invisible participant” has been shown to be unrealistic (Davidson, 2000; Wadensjö, 1998), and in the speech-language pathology setting, interpreters have been found to facilitate the interactions in ways that fall outside traditional “transmissionist” notions of the interpreter's role (Merlini & Favaron, 2005; Roger & Code, in press). Nonetheless, codes of ethics that interpreters (particularly professionally accredited interpreters) must follow may specifically proscribe certain behaviours. These include expressing opinions about other parties present (or commenting on what they say), as well as adding or omitting anything when interpreting from one language into another (NAATI, 2000). Requests from SLPs to describe (for instance) phonological or grammatical irregularities in the utterances of an aphasic speaker can thus stretch the role of interpreters and make it difficult for them to stay within their professional ethical boundaries.

Rationale and aims

SLPs are frequently warned about using ‘translated’ tests that have not been carefully adapted for use with a language other than that for which they were designed (e.g., Paradis, 2004), as doing so is likely to produce unreliable results. However, there is evidence that administering an English-language test with the assistance of an interpreter is a very common practice (Clark, 1998; Isaac, 2002; Roger et al., 2000). The aims of the present study were, therefore, (1) to examine whether there are threats to the content validity of tests administered in this way, and (2) if content validity is indeed compromised, to document how and at precisely what points such problems occur.

Armed with this information, clinicians will be in a better position to decide whether or not this approach to assessment is likely to yield meaningful results. If they feel that they have no other options, they will at least be alerted to potential pitfalls and ways of working more effectively with interpreters to minimize these where possible.

Methods

Data collection procedures

Three authentic aphasia assessment sessions, each involving a different language (Cantonese, Vietnamese, and Tagalog), speech-language pathologist, interpreter, and aphasic speaker, were video-recorded for this study. Formal approval of the ethical aspects of the study was sought and obtained from the relevant university and hospital committees. All recordings took place at the same hospital in Sydney, Australia, where the SLPs had indicated an interest in participating in the study. On each occasion, consent was obtained from all three parties for recording to take place. In the case of the people undergoing assessment, informed consent was obtained from an appropriate guardian if their language disorder precluded the possibility of obtaining consent from them directly. Although a post-session de-briefing directly following the sessions (involving the SLP and interpreter) would have been valuable, this was not possible as the interpreters in all cases were required for other appointments at other sites.

Participants

A brief profile of the participants is provided in Table I. All of the interpreters were accredited by the National Accreditation Authority for Translators and Interpreters (NAATI) to a level which enabled them to work in the healthcare sector. The SLPs at this particular hospital reported informally that assessing aphasia with the assistance of an interpreter was not an uncommon event for them. SLPs and interpreters used each other's given names throughout the sessions, and their pseudonyms reflect this. In two cases, the SLP addressed her client by title + surname, while in the other case, the client's given name was used; this pattern is again reflected in the pseudonyms.

Table I. Participant profiles.

Speech-language pathologists
Pseudonym	Qualifications	Workplace setting	Languages spoken
“Susan”	Bachelor of Applied Science (Speech Pathology)	Rehabilitation hospital	English
“Sarah”	Bachelor of Applied Science (Speech Pathology)	Rehabilitation hospital	English
“Simone”	Bachelor of Applied Science (Speech Pathology)	Rehabilitation hospital	English
People with aphasia
Pseudonym	Languages spoken	CT scan findings	Time post-onset at this assessment
“Mrs Pan”	Cantonese	Left temporo-parietal intracerebral haemorrhage	10 weeks
“Mrs Pham”	Vietnamese (first) Cantonese (first)	Left temporal haemorrhage and subarachnoid haemorrhage	18 months
“Pedro”	Tagalog (first) Illocano (first) English (second)	Right posterior territory intracerebral haemorrhage	4 weeks
Interpreters
Pseudonym	Accredited languages for healthcare interpreting
“Iska”	Tagalog ↔ English
“Ian”	Vietnamese ↔ English
“Ivy”	Cantonese ↔ English

Testing instrument

The main testing instrument used in all assessments was the Western Aphasia Battery (WAB, Kertesz, 1982), reflecting normal preference of the SLPs at this particular institution. This choice is in line with a multi-national survey of aphasia management in a number of English-speaking countries (Katz et al., 2000), which found that the WAB was the most commonly used aphasia test among 36 Australian respondents, with 72% of those surveyed reporting its use. Across the whole survey, the WAB was second only to the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983) in frequency of use. Other evidence for the popularity of the WAB has emerged from a recent survey (Bate et al., 2010). There does not appear to be any published research on the use of this particular test in interpreter-mediated assessments, however.

The WAB consists of two broad sections: (1) Oral Language sub-tests, and (2) Visual Language and Other sub-tests. The oral language sub-tests consist of spontaneous speech, auditory-verbal comprehension, repetition, and naming. The second section (visual language and other sub-tests) incorporates reading, writing, apraxia, and constructional, visuospatial, and calculation tasks.

The tests were administered to gain a general picture of the aphasic speaker's communicative abilities, and not to obtain a WAB aphasia quotient or aphasia type. In some cases the assessments were incomplete, and certain items were omitted.

Analysis

The methodology adopted for this study of content validity involved three bilingual SLPs viewing the videotapes of assessments conducted by unilingual English-speaking SLPs through interpreters. These bilingual SLPs were native speakers of Tagalog, Cantonese, and Vietnamese, respectively, and had experience using both their native languages and English in their own clinical practices. The bilingual SLPs (who were familiar with the rationale behind each of the sub-tests) were thus able to provide insights into the characteristics of the “interpreted test items” as well as the nature of the aphasic speaker's responses and the interpreted responses provided during the session. The primary intention here was not to arrive at a precise and comprehensive description of the individual's aphasia symptoms, but rather to look critically at aspects of the content validity of the assessments as they had been carried out.

In each case, the bilingual SLP and the first author viewed the video together. The bilingual SLP was asked to stop the tape and comment wherever s/he felt that something had been lost, misunderstood, or misrepresented unintentionally in the course of the encounter. Some segments of the tape were re-played multiple times for the purposes of transcription, or to confirm a particular observation. This process highlighted (for the researchers) the enormous challenges faced by the interpreters in the sessions, who were of course working in real time and did not have recourse to recordings of the speaker's performances on each of the tests.

Transcriptions and translations of the segments of interest were provided by the bilingual SLPs, and there was often considerable discussion between the researcher and the bilingual SLP about the way in which disordered utterances could best be represented in English. This process yielded an English translation which highlighted the deficit as far as possible, but which was often not complete without an accompanying explanation to clarify the nature of the deficit and to compensate for differences between English and the language in question. What these translations and accompanying explanations do allow is a comparison between the clinical impressions gained by a unilingual SLP working through an interpreter, and a bilingual SLP who (a) understands what a SLP is looking for, and (b) can access both the English and the target language portions of the encounter directly.

As Menn et al. (1995, p. 37) point out, transcribed speech is “scientific data”, while the translations of this transcribed speech are “interpretations”. Most bilingual SLPs are not trained in translation and interpreting, and this fact was recognized on the information and consent form to stress that the bilingual SLP was not being held up as a model interpreter. To regard a person as an expert interpreter simply by virtue of the fact that s/he is proficient in two or more languages would be to ignore the body of research and expertise which underpins the interpreting profession.

For ease of presentation, utterances in languages other than English in the transcribed extracts from the encounters appear as English back-translations here. Tagalog transcriptions are included in the body of the text; Tagalog is written with the Roman alphabet, and the inclusion of the “original” language data allows some discussion of phonological irregularities which were evident in this particular encounter.

Results

Overall, the findings suggest that certain components of the interpreter-mediated assessments reported here are vulnerable—by their very nature—to significant threats to content validity. Analysis demonstrates that this can occur (a) at the point of administration of the test item, or (b) at the point where the aphasic person's response is reported back to the SLP. The ways in which this can occur, together with illustrative evidence from the data, are presented below. The discussion of the extracts that follows is in no way intended as criticism of the performance of either the SLPs or the interpreters. Rather, the findings illustrate specific ways in which the process of assessing aphasia in this way is inherently vulnerable to disruption.

Threats to content validity at the point of test administration

Probability and frequency

As with all formal aphasia test batteries, items on certain sub-tests of the WAB are controlled for the frequency with which they occur in the language and the probability of their occurrence. In Extract 1, the interpreter has been asked to sight-translate the repetition sub-test from the text booklet, and has reached the sentence “The pastry cook was elated”.

Extract 1

36 Susan  OK .. just do this one .. don't worry about doing [inaudible]

37      just this last one

38 Iska   ang taga– ((hesitates))

39 Susan ⇒ just say “happy” … happy's fine .. instead of “elated” just say

40       “happy”

41 Iska   ang tagapagluto ay masaya (the cook is happy)

42 Pedro  ang tag-pagluto masaya

“The pastry cook was elated” is clearly a “low probability” English sentence which involves (according to the test manual) a degree of “oral agility”. The translated version here does not preserve this (indeed, a semantic translation cannot preserve all of these elements simultaneously). It is also modified here in two ways which appear to make it simpler: “pastry cook” becomes “cook” and the high frequency Tagalog word masaya (“happy”) is used in place of the relatively low frequency English word “elated”. The interpreter made a point of alerting the SLP to the fact that tagapagluto translates as “cook” rather than “pastry cook”. Although the end result represents a substantial modification of the test, the SLP here is fully aware of this (and in fact initiated the first modification herself) and would thus have been able to take it into account in her assessment. On a positive note, it was also apparent that the administration of earlier items on this particular sub-test (involving single words and numbers) with the assistance of the interpreter was relatively trouble-free.

Staging and length

In some of the sub-tests which make up the WAB, the complexity of the stimuli is controlled so that the test begins with the simplest and gradually progresses to more complex items. Complexity is controlled in several ways, including sentence length, the number of clauses, the use of certain syntactic patterns, and the number of steps, stages, or elements that the test item incorporates.

Equivalence in length and staging (and therefore the degree of difficulty) can be lost in the process of sight translation (of written test material) or on-the-spot interpreting (of oral tests spoken by the SLP). Interpreters are naturally accustomed to adjusting these variables to achieve maximum comprehensibility, but the task of interpreting for aphasia assessment requires a rigid adherence to set length and staging structure. Indeed, if comprehension is not achieved, the test has yielded useful information. Unless interpreters are explicitly made aware of this, they are unlikely to arrive quickly and spontaneously at an understanding of the linguistic elements that are crucial in any given test. Extract 2 from the Sequential Commands sub-test administered during the Tagalog assessment illustrates this point:

Extract 2

9 Susan    OK .. with the book point to the comb

10 Iska   hawakan ninyo ho yong notebook / tapos ituro ninyo / na

11       hawak .. yong suklay

        (hold the notebook / then you point / while holding .. the comb)

Here it is evident that the Tagalog command is significantly longer and more explicitly “staged” when compared with the English command. An element built into the English test item here is the preposition combination “with/to” (Kertesz, 1982). An unpublished Tagalog version of the WAB uses a form that is closer syntactically, but not particularly natural:

Sa pamamagitan ng libro, ituro ang suklay

(With the book, point to the comb)

This is a closer rendering into Tagalog of the preposition ‘with’ (as it is used in this particular test item). However, equivalence is still not achieved, as it does not preserve the relative subtlety of the “with/to” distinction, which is related to the fact that these are short, unstressed, high-frequency prepositions that can be reversed to alter meaning in English sentences such as those seen here.

The example above again illustrates the limits of what can be achieved by a so-called “word-for-word” interpretation. One element of Tagalog grammar is the use of “focus markers”, which precede each of the two nouns (libro and suklay) in the sentence above (i.e., ng libro and ang suklay). Ang indicates that the noun is the “focus” of a sentence, whereas ng signifies that a particular noun is not “in focus”. English has no equivalent morphemes, and a language test designed originally for English will not be able to test such elements unless it has been specifically adapted to do so.

Issues relating to staging and length of test items were also apparent in other tests, as illustrated by the following extract from the Yes/No Questions test, administered on this occasion with the assistance of a Vietnamese interpreter. The initial items (relating to the aphasic speaker herself) were interpreted and answered without complication. However, when the original English test questions became more linguistically complex, this complexity was not necessarily reflected in the way that the questions were put in Vietnamese.

Extract 3

1 Simone   are the lights on in this room?

2 Ian     (in this room, there are lights, right?)

3 Mrs Pham  (yes)… ((unintelligible sounds))

4 Ian     (the lights are on, aren't they?)

5 Mrs Pham  (yes)

6 Ian      yes

7 Simone   well done, Ian, you didn't look at them!

Here the interpreter breaks the two-element question down into two single-element questions. Although he does not give a clue by looking at the lights himself (something that the SLP Simone praises him for) he does verbally draw Mrs Pham's attention to the lights before putting the next “part” of the question. Not being a Vietnamese speaker, Simone would not be aware of this. When the second part of the question is put, it is framed so that Mrs Pham simply has to confirm a statement. Her answer is translated back to the SLP as “yes”, which matches the original question and is thus scored as a correct response.

This phenomenon was also apparent in another of the yes/no questions administered in the same assessment (Extract 4):

Extract 4

14 Simone    … will paper burn in fire?

15 Ian        (for example, can we use this paper to make a fire?)

16 Mrs Pham  (sorry?)

17 Ian     (the paper, if you light it, will it burn?)

18 Mrs Pham  (fake things ((unintelligible sounds)) I

19        don't have … I don't know)

20 Ian      I don't know

Once again, the nature of the test item has changed in the process of interpretation. While the original is an abstract general question about the relationship between two elements (paper and fire), the interpreted version (line 15) makes it concrete by referring to a specific action involving a particular piece of paper. When the question is repeated in line 17, the staging is altered, presumably in an attempt by the interpreter to facilitate communication (which is of course part of an interpreter's usual role).

Mrs Pham's reference to “fake things” (line 18) is interesting. The bilingual SLP viewing the videotape speculated that Mrs Pham might be referring to the practice common in Vietnam of burning fake money as a way of making it useful to ancestral spirits. It seems possible that words such as “paper”, “burn”, and “fire” triggered other associations for her which she attempted to put into words. Either the interpreter missed the reference to “fake things” among the unintelligible sounds, or he disregarded it as irrelevant to the SLP's question.

In terms of the frequency with which such instances occurred, it was apparent that none of the yes/no questions administered in this assessment closely matched the English originals in terms of the structure, and the nature of the change was slightly different in each instance. This is not a criticism of the interpreter, however, as yes/no questions in Vietnamese are formed differently from their English counterparts (Vietnamese Online Grammar, 2008), and there would obviously be no point in administering very awkward-sounding constructions in Vietnamese in order to mirror items on a English language test.

Syntax

As discussed above, aphasia test batteries such as the WAB contain items that have been carefully designed from a syntactic standpoint. Whether or not the syntactic elements of the original item are preserved in a semantic interpretation (as opposed to an adaptation) will depend upon the similarities and differences between the source and target languages at the level of syntax. The Sentence Completion component of the WAB provides an illustration of the way in which this factor can compromise the content validity of an aphasia test.

The Sentence Completion sub-test is dependent on language-specific (i.e., English-specific in this case) variables. First, as indicated in the description immediately above, the items which make up the sub-test tend to be produced automatically, as they represent very familiar “over-learned” associations. Translating them into another language will not necessarily preserve this feature. Second, the test items are declarative statements in which the word that occupies the sentence-final position is omitted, and must be supplied by the aphasic speaker. The instruction “finish the sentence with a word” reflects the expectation that the sentence would normally “trigger” the production of the final word.

When this particular test was administered as part of the Cantonese assessment, the following exchanges were observed:

Extract 5

14 Sarah OK … the grass is …

15 Ivy (what's the grass like?)

16 Mrs Pan (grass … grow … they … grow)

17 Ivy the grass .. people grow the grass

18 Sarah mm .. I need you to finish my sentence though .. you need to

19  finish my sentence with a word .. the grass is …

20 Ivy (you've got to follow the meaning when you say it … say what

21  is the grass like?)

22 Mrs Pan (the grass … growing something … is ah … at the tree)

23 Ivy is .. ah .. growing outside

24 Sarah OK, the grass is growing outside .. OK … sugar is …

25 Ivy (what is the sugar like?)

26 Mrs Pan (the sugar … drink sugar … um the sugar drink um … is the

27 drink … drink)

28 Ivy sugar used for . drinking something

29 Sarah used for drinking? …

In line with her normative role, it is evident that the interpreter here orients herself (probably automatically) to rendering semantic content, rather than trying to preserve the syntax of the SLP's utterance (which would of course be completely unnecessary in most interpreter-mediated interactions). Thus, while the SLP's instructions (lines 18–19) focus on syntactic aspects of the task (“you need to finish my sentence with a word”), the interpreter's instructions (line 20) tend to focus on semantics (“you've got to follow the meaning when you say it”). Similarly, an incomplete declarative sentence requiring completion is replaced by a question in the Cantonese interpretation (lines 15 and 25). Both are intended to elicit similar responses, but because the SLP is almost certainly unaware of the way in which the task has been altered, she would not be in a position to take this into account in attaching meaning to the response. In other words, the item as it is administered in Cantonese does not test what the SLP would be expecting it to test.

Consistency in repetition

In formal testing of this sort, test items may be repeated by the tester in some circumstances. When this occurs, the item is generally repeated verbatim rather than being re-formulated to aid comprehension. The facilitation of communication would normally be prioritized by interpreters, however, and verbatim repetition following apparent non-comprehension would be seen as communicatively insensitive. The following extract from the Tagalog encounter illustrates what can happen if the issue of “consistency” is not anticipated and addressed:

Extract 6

1 Susan Pedro .. point to the pen with the book

2 Iska ituro ninyo yong ballpen na hawak yong notebook

(point to the pen while holding the notebook)

At this point Susan reminds Pedro to wait until Iska has finished interpreting before he carries out the command. She then repeats the item:

3 Susan point to the pen with the book

4 Iska hawakan ninyo yong notebook, ituro ninyo yong ballpen

(hold the notebook, point to the pen)

Not surprisingly, the syntax of the repeated command is identical in English (being read from a test booklet by a SLP accustomed to the rules of standardized aphasia test administration), but different in Tagalog (being interpreted on-the-spot from spoken language by an interpreter accustomed to paraphrasing utterances that are not understood the first time). In other words, the interpreter has no written “script” to follow and does what is only natural in the face of an anticipated lack of comprehension: she “stretches out” the command so that it is more clearly staged.

A similar situation arises later in the same assessment session, but on this occasion it comes to the SLP's notice and she interrupts the assessment to check with the interpreter:

Extract 7

19 Susan… OK .. I'm going to ask you some questions now .. what do

20 you write with?

21  Iska may ilan ho akong itatanong sa inyo .. anong ginagamit ninyo

22   pag nagsusulat kayo? (I'm going to ask you some questions now .. what

    do you use when you write?)

23  Pedro my name

24 Susan OK .. I'll say it again .. what do you write with?

25 Iska sasabihin ko ho uli ang tanong .. pag ho nagsusulat, ano ho ang

26  ginagamit ninyo para kayo makapag sulat? (I will repeat the question .. when you write, what do you use so that you are able to write?)

27 Pedro ballpen (pen)

28 Susan mm.. did you change the phrasing then?

29 Iska y::es

30 Susan how did you word it?

31 Iska I .. I just told him “whenever you write, what do you use to be

32  able to write?”

33 Susan OK – ((writes)) …

Here, as in Extract 6, an identically repeated question from the SLP is repeated in a re-phrased form (lines 25–26) by the interpreter. This time, the question is put so that a hypothetical situation is set up (“when you write”) prior to the actual question being put (“what do you use so that you are able to write?”). A similar clause structure was in fact used by the Vietnamese interpreter in Extract 4. However, in this case the exchange that follows between Susan and Iska does clarify what has taken place (lines 28–32). Knowing the way that the question has been put in Tagalog means that Susan is in a better position to interpret Pedro's response.

Threats to content validity at the point of reporting

In addition to the problems presented above that arose at the point of test administration, analysis of the interactions recorded for this study revealed that content validity was frequently compromised when the interpreter reported back the responses to the SLP conducting the testing. Three basic ways in which this can occur are apparent from an analysis of the data. The first occurs when interpreters expand or edit the utterances of the person with aphasia based on their ability to guess the intended meaning and (in effect) “fill in the gaps” when reporting back to the tester. The second happens when one or more clarifying questions are put by the interpreter to the speaker, and the sequence of what is said in the other language is not relayed to the tester in the summary interpretation that follows. Finally, interpreters are faced with choices about the way in which they render the utterances of the aphasic speaker back to the tester; if the tester regards the interpreter's rendering as a literal interpretation without taking account of the relevant cross-linguistic differences, content validity will again be compromised. In all of these cases, it is important to bear in mind the normal role of the interpreter is to facilitate communication, and that the aphasia assessment is a unique interpreter-mediated setting in that well-intentioned attempts to facilitate communication can actually interfere with the assessment process.

Expansion of utterances

The most striking examples of situations in which utterances were expanded in the process of interpretation was in the tests that involved spontaneous speech, picture description, and the initial social interaction that often occurred at the beginning of the assessments sessions. Several examples of this ‘utterance expansion’ phenomenon were evident when the Cookie Theft Picture from the Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983) was administered as part of the Cantonese assessment:

Extract 8

3 Sarah  … so in this ah formal test .. ah if I could get you to … when you're translating

4  back to me the responses .. to say it word for word what she says

5  … and also if there's any incorrect pronunciation of the letters .. if you could

6 just let me know=

7 Ivy ((nodding)) OK

8 Sarah =about it as well … OK? … Mrs Pan can you tell me what is happening in

9  this picture?

10 Ivy  (ah … Mrs Pan, you look at this picture)

11 Mrs Pan  (yes) ((looks at picture))

12 Ivy  (which means you look at this picture, can you tell her what's

13   in it?)

14 Mrs Pan  (here is clip … clip … fell down … is it right?)

15 Ivy  um the boy .. ah the chair is nearly felling off

16 Sarah  ((writing)) - anything else?

Extract 9

24 Ivy  (and what? continue to talk)

25 Mrs Pan (talk about this?)

26 Ivy (you talk about it .. talk about whatever you see)

27 Mrs Pan (this … this… she holds a tub … a cup … a plate … there wiping)

28 Ivy  all right um .. this woman is holding a dish … is ah wiping …

Overall, the translation of the speaker's whole picture description done from the video suggests a greater degree of word-finding difficulty than does the original interpretation provided during the session. One of the clearest instances of this is the example above (Extract 8) from the beginning of the picture description, where Ivy's interpretation (line 15) includes the key nouns chair and boy although no Cantonese equivalents for these are uttered by Mrs Pan (line 14). In this initial utterance, she uses a word which translates as clip; this does not appear to be phonologically or semantically related to any identifiable target word. This word is not interpreted into English during the encounter. In Extract 9, Mrs Pan utters the semantically-related words tub and cup (line 27) before the (correct) dish. The words for tub and cup begin with the same initial phoneme in Cantonese, and this therefore seems likely to be an example of a phonemic paraphasia followed closely by a semantic paraphasia. At other points in the description (not shown in the extracts above) the words sink and girl appear in the English rendering provided by the interpreter, but are not found in Mrs Pan's Cantonese utterances. It seems likely that the fact that the interpreter could see the picture may have led her to interpret what Mrs Pan was obviously trying to say (the intended message), rather than adopting a more literal approach.

Clarifying questions and summary interpretations

This pattern was frequently observed in a number of tests administered during the recorded assessment sessions. It will be generally clear to the SLP that some negotiation of meaning is taking place when a series of turns takes place without any interpreting. The details of such negotiation will not be available to the SLP, however, unless the interpreter is able to re-construct the exchanges that took place (either voluntarily or when asked). Extract 10 (from the Vietnamese assessment) illustrates this phenomenon:

Extract 10

35 Simone  have you been here before?

36 Ian (have you ever stayed at this hospital before?)

37 Mrs Pham  (when?)

38 Ian (before)

39 Mrs Pham (when … I am here)

40 Ian (is that right?)

41 Mrs Pham (yeah)

42 Ian yes

Similar sorts of examples were evident in the Spontaneous Speech sub-test in the Vietnamese encounter, and in the Picture Description and Repetition sub-tests in the Cantonese encounter. In some cases, the interpreters reported back to the SLPs the details of the exchanges that had taken place. However, particularly in cases such as Extract 10, the aphasic utterances would have been impossible to re-construct by an interpreter working in real time and without recourse to a recording of the exchanges that took place.

Form of the interpreted utterance

The way in which an interpreter chooses to render an utterance may occasionally determine whether or not the SLP counts it as correct or not. Indeed, this happened on two successive items in the sentence completion task conducted with the Cantonese interpreter (Extract 5 above). As noted earlier, the Sentence Completion test items (e.g., “Grass is …”) became requests for information (e.g., “What's the grass like?”) in the Cantonese renditions. The aphasic speaker's first response is interpreted into English as “People grow the grass”, which is not accepted by the assessing SLP as it does not fit the sentence completion test frame. The subsequent attempt is interpreted into English as “… is growing outside”, and this time it is accepted by the tester as a possible sentence completion. Either coincidentally or deliberately, the interpreter has provided a response that fits the tester's English test frame, even though the speaker was not actually asked to complete a sentence in the Cantonese interpretation of the test. Whether or not the response is ultimately scored as correct or incorrect can therefore depend on whether or not the semantic content is rendered by the interpreter in a way which fits the English test frame. This is an example of a situation in which content validity is weakened at both the test administration and response levels.

Discussion

Summary of findings

The findings discussed above demonstrate that when aphasia assessments were conducted using English language testing instruments with items interpreted in real time by a trained interpreter, significant content validity problems arose. While different assessment sessions exemplified different potential problems, content validity issues of one type or another were apparent in each of the sessions. Furthermore, the problems were not confined to one or two sub-tests, but were evident at some point in a range of sub-tests, including: tests of connected speech elicited by interview and picture description, yes/no questions, sentence completion, sequential commands, repetition, word fluency, and responsive speech. Although some simple test items were apparently trouble-free (e.g., questions such as “what is your name/address?” and single word repetitions), it was apparent that many of the items on the English-language testing instrument did not lend themselves to administration in this way.

We sought to determine at what specific points these threats to content validity occurred. As noted above, the analysis demonstrates that this can happen (a) at the point of administration of the test item, or (b) at the point where the aphasic person's response is reported back to the SLP.

Role boundaries for interpreters

It is argued here that the different bodies of knowledge and normative roles of the two professionals mean that neither has a complete view of what is going on in the encounter. The testers rarely provided detailed descriptions of their particular points of focus prior to the administration of a test, and interpreters rarely alerted testers to the fact that a test was not going to “work” if it was translated. For the interpreter, not knowing the purpose of a given test also creates problems (Kambanaros & van Steenbrugge, 2004). If the linguistic elements embedded into the tests are not immediately apparent, they are unlikely to be preserved by an interpretation of semantic content except where the structural features of the two languages concerned allow this to happen by chance.

It is important to recognize that interpreters in all contexts naturally ask clarifying questions of one or other of their clients when something is not clear, as it is generally essential to allow them to fulfil their role. In fact, this is expressly stipulated in the Australian Institute of Interpreters and Translators (AUSIT) Code of Ethics (1996), which contains the following professional precept:

If anything is unclear, interpreters and translators shall ask for repetition, rephrasing or explanation (Item 5 (b) ii).

However, in the case of a language assessment, such clarification questions directed to the aphasic speaker can constitute “prompting”. If what is rendered back to the SLP is a “summary interpretation”, the degree to which the final answer had to be “elicited” from the person undergoing assessment with supplementary questions will be largely invisible to the assessing clinician.

In sum, the two participating professionals bring with them different knowledge schemata and adopt different orientations (i.e., a “linguistic” orientation or a “message” orientation) throughout the interpreter-mediated aphasia assessment. The interpreter often does not know precisely what the SLP is looking for, and therefore which elements of the test items or responses are crucial. The SLP does not know how the other language “works” at the levels of lexis, syntax, phonology, discourse, and pragmatics, and is thus unable to anticipate points at which crucial elements may be disrupted in the process of interpretation. The end result is that content validity is weakened without either participant necessarily being aware that this has happened.

Overcoming content validity problems

Pre-session briefings and planning test administration

A pre-session briefing (Isaac, 2002; Langdon, 2008) is likely to be helpful in a number of respects. Isaac (2002) stresses that such briefings should be conducted in a manner that reflects the collaborative partnership between the interpreter and the SLP. By discussing the nature of aphasia and the broad purposes of the assessment, the SLP may be able to convey to the interpreter the need to avoid asking the client clarifying questions in the course of the assessment, and to alert the SLP when responses are unclear or non-target-like, rather than repairing them in the process of interpreting. The interpreter, in turn, may be able to assist by pointing out elements of the tests that will not work in the target language. The SLP may then be able to remove or substitute objects or picture-based stimuli which are less common or unknown in the other language or culture. In a short pre-session briefing, however, the amount of tailoring that can take place is naturally very limited.

Yiu (1992) provides an insightful account of the process of adapting the WAB for use with speakers of Cantonese. For instance, the sentence completion task on the Cantonese Aphasia Battery uses three common expressions (e.g., “Spring, summer, autumn and ___” [winter]) and two popular Chinese proverbs (e.g., “A drunken man's desire is not the ___” [wine]). Adaptations of this sort require careful forethought and planning, as evidenced by the guidelines provided by Paradis (2004) for adapting the BAT for new languages. Complete re-writing of sub-tests is naturally well beyond the scope of pre-assessment briefings involving interpreters and SLPs.

A briefing session can, however, help the SLP and interpreter to agree on the details of test administration, and in the process overcome partially some of the problems identified above. One such issue is the lack of consistency when the test items are repeated. This could be addressed by the interpreter working from written items (as SLPs normally do) which have been translated prior to the encounter itself, or by the SLP alerting the interpreter when consistent wording is required so that he or she can make notes during the testing process. Simple measures such as the creation of an information gap between the interpreter and aphasic speaker on picture description tasks (i.e., concealing the picture from the interpreter) could assist the interpreter in achieving a more objective interpretation by removing the visual cues that can lead to expanded interpretations. The picture could be shown to the interpreter at the completion of the task, so that any additional insights with regard to what the aphasic speaker might have been trying to say could be discussed with the SLP.

A potentially useful strategy for tasks that call for an extended description or narrative monologue would be the adoption of a quasi-simultaneous interpreting mode, and strict interpretation into English of all questions addressed by the aphasic speaker to the interpreter, and of all responses (potential prompts) given by the interpreter. This would avoid the sorts of summary interpretations which tend to over-rate the speaker's actual performance; however, it may impose an unreasonably high cognitive load on interpreters if they are asked to preserve all repetitions, false starts, and revisions in their interpretations. Underlying these types of strategies is the more general need for the SLP to make the interpreter more aware of the overall purpose and the sensitive elements of the task (Isaac, 2002; Kambanaros & van Steenbrugge, 2004).

Limits of interpreting

In a briefing session with an interpreter and throughout the assessment itself, it is important for SLPs to remain realistic about the limits of what an interpreter (even at the highest level) can provide. In the encounters examined here, there was at times an implicit or explicit expectation from the SLP that a literal word-for-word interpretation would be provided by the interpreter. Evidence for this was apparent in the instructions given to interpreters (e.g., lines 3–4 in Extract 8 above), and in the verbatim recording of interpreted utterances, and similar expectations on the part of some SLPs were also reported by Roger et al. (2000). However, as noted earlier, grammatical disruptions and phonological irregularities are often not interpretable from one language to another, so that interpreters (in carrying out the strict interpreting task) have no choice but to repair grammatical deficiencies in the process of interpreting. Neologisms and certain paraphasias may be similarly uninterpretable. There are thus significant constraints that limit what can be achieved in this context through a reliance on so-called literal interpretations.

As noted by Gentile et al. (1996) and Roger and Code (in press), problems stemming from the “uninterpretability” of certain aphasic phenomena can sometimes be overcome by asking the interpreter to describe features of the person's speech which cannot be translated. An uninterpretable non-word, for example, could be described by the interpreter as having a similar sound to a likely target word in a given context. However, frequent suspension of testing during the session to allow for discussion and description is also of limited value as a compensatory strategy. Because of the different knowledge schemata of SLPs and interpreters, a common orientation to, and precise labelling of the details of form, phonology, and syntax would be difficult to achieve during the session. It would also mean that the person with aphasia is effectively put “on hold” while the SLP and interpreter engage in a two-way conversation, which is potentially both discourteous and distressing to those undergoing assessment, unless measures are taken to minimize their sense of exclusion from the interaction.

When it comes to the frequency (or existence) of certain grammatical constructions in different languages, it is unlikely that either a pre-session briefing or discussions during the course of the assessment of such issues will resolve them and allow the necessary adaptations or substitutions to be made. It is a very complex task, requiring a detailed understanding of the nature of the tests in question and the language for which they are being adapted. As Ardila (1995) points out, the inclusion of a particular task as part of a psychometric assessment assumes that the task is one that exists and has some relevance in the language in question. The assessments analysed here illustrate the way that such assumptions are built into aphasia tests.

Alternative approaches

Given the limits on what can be achieved using English-language tests and working with an interpreter, it appears that the use of the BAT or a test specifically written or adapted for use with the language in question is a much sounder option. However, the logistics of administering such tests remains a key problem in cases where the assessing clinician is not a speaker of the language in question. Depending upon the prevailing institutional cultures and policies in operation in various parts of the world, calling on friends, relatives, or other bilingual contacts may be seen as ethically unacceptable because of the emotional involvement (in the case of family members), as well as the potential for the dignity and privacy of the person with aphasia to be threatened by the involvement of others who are not bound by professional codes of ethics and confidentiality. Where professional interpreters are available, their involvement in administering such a test would thus be desirable. However, administering a series of aphasia tests that already exist in the target language clearly involves interpreters working well outside their usual professional role, as they would not in fact be interpreting at all. It would therefore be vital to obtain the interpreter's consent to act in this capacity.

It is common for a comprehensive aphasia assessment to be conducted over two sessions, necessitating two interpreter bookings. Rather than spreading the actual test administration over two sessions, an alternative schema would involve distinct activities in each session. The first would include a pre-session briefing followed by the administration of the selected tests. This session would be recorded, as has been recommended by several authors (Isaac & Hand, 1997; Paradis & Libben, 1987). The second session would take the form of a conference between interpreter and tester, where the aphasic speaker's performance on the tests would be discussed (with reference to the recording where necessary), to enable the tester to decide on therapeutic recommendations. The present study, in which the recordings of sessions were reviewed with the assistance of bilingual speech-language pathologists, highlighted the amount of extra information that could be extracted when segments of these recordings were available to be reviewed as many times as necessary.

Finally, some clinicians may decide to avoid the potential pitfalls of translated/adapted tests altogether, opting instead for a more discourse-based approach to assessment. Penn and Beecham (1992) assessed a multilingual aphasic speaker in four of his languages through three discourse tasks using trained first-language speakers as raters, whose judgements were found to be remarkably reliable. As the authors suggest, this approach may have wider applications in the assessment of aphasia in multilingual speakers in situations where SLPs with competence in the languages concerned are not available. Once again, if professional interpreters were willing to take on the role of trained rater, this would help to overcome the ethical barriers of involving laypersons in sensitive healthcare assessments.

Conclusions

In many English-speaking countries, there is no great shortage of trained clinical personnel, but there is a shortage of SLPs fluent in the various community languages spoken. This study illustrates the specific ways in which content validity is likely to be compromised when English aphasia tests are administered with the assistance of an interpreter without having been carefully adapted for use in another language. While this study does not claim to provide a definitive inventory of such problems, it hopes to alert SLPs to some of the vulnerabilities that are inherent in the process of administering aphasia tests in this way.

A major barrier to the use of the valuable testing material that already exists in many languages is the absence of individuals who are both fluent in the languages concerned and official sanctioned (from an ethical perspective) to take on a key role in assisting SLPs with such assessments. We have argued that accredited healthcare interpreters (where they are available) are ideally placed to fulfil this role, but whether or not they are prepared to take on such work will depend on their willingness to step clearly outside the conventional interpreter's role. Paradoxically, the recommendations outlined above may be easier to implement in countries or regions where interpreters are not required to hold formal accreditation, as they may be more at liberty to negotiate their individual role boundaries. While professional interpreter accreditation is certainly an asset in ensuring quality and ethical standards in healthcare interpreting, role definitions need to take account of the unique demands of particular contexts, of which the assessment of aphasia is a prime example.

Acknowledgements

The authors gratefully acknowledge the willingness of all participants (speech-language pathologists, interpreters, and speakers with aphasia) to take part in this study. The valuable insights provided by the bilingual speech-language pathologists as part of the data analysis are also acknowledged with gratitude. The authors also appreciate the constructive feedback offered by the two anonymous reviewers on an earlier version of this manuscript.

Appendix: Transcription conventions

..   two dots indicate a short pause (less than 1 second)

…   three dots indicate a longer pause (more than 1 second)

(yes)   italicized words in parentheses indicate translated utterances

((writes))   double parentheses indicate non-verbal actions

y::es   colons indicate lengthening of the preceding sound

=   an equal sign denotes continuation of a speaker's utterance