A transcription-less quantitative analysis of aphasic discourse elicited with an adapted version of the Amsterdam-Nijmegen Everyday Language Test (ANELT)

ABSTRACT

Background

For speakers with mild to moderate expressive aphasia the ultimate goal of aphasia therapy is to improve verbal functional communication, which may be assessed with the Amsterdam-Nijmegen Test for Everyday Language (ANELT; Blomert et al., 1995). The ANELT is based on a qualitative and transcription-less method of analysis: the scoring is based on personal judgement and directly made from the recording of the test. Previous research (Ruiter et al., 2011) has shown that a quantitative measure for the ANELT not only allows verbal effectiveness (i.e., the amount of essential information conveyed) to be measured more sensitively, but also allows derivation of a measure of verbal efficiency (i.e., average amount of essential information produced per time unit). Although the quantitative scoring further improved the construct validity of the ANELT, there is a limitation that hinders its clinical application: the quantitative measure requires orthographic transcription of the spoken responses to the test. That is, the quantitative scoring is transcription-based.

Aims

In order to work towards clinical applicability of the quantitative measure of the ANELT, this study addressed the potential of a transcription-less variant of the quantitative analysis, in which the amount of essential information is directly quantified on the basis of recording, as a valid and reliable procedure for the measurement of verbal effectiveness.

Methods & Procedures

A total of 56 speakers of Dutch participated: 31 neurologically healthy speakers and 25 persons with aphasia. Monologic discourse elicited with 10 scenarios from an adapted version of the ANELT (Ruiter et al., 2016) was analysed with both a transcription-based quantitative method and a transcription-less quantitative one. Resulting data were systematically compared on the following psychometric properties: internal consistency, inter-rater reliability, construct validity, convergent validity, and known-group validity.

Outcomes & Results

Internal consistency and inter-rater reliability were good and comparable between both scoring methods. Only for one scenario did the transcription-based scoring method yield higher agreement among the raters. With respect to validity, both scoring methods seem to yield measures of the same underlying constructs, show a strong and positive correlation, and allow differentiation between persons with and without aphasia.

Conclusions

Although future research is needed to develop norm scores and investigate other psychometric properties, the result from the comparison demonstrated the potential of the transcription-less quantitative method as a valid and reliable method to analyse monologic discourse elicited with the adapted ANELT.

KEYWORDS:

• aphasia
• functional communication
• transcription-less discourse analysis
• ANELT

Introduction

Aphasia, a communication disorder that results most frequently from stroke or other damage to the brain, has implications for the production of spoken discourse, even in mildly-impaired speakers (e.g., Fromm et al., 2017; Richardson & Dalton, 2015). Discourse is generally described as “a unit of language larger than a single clause, forming a meaningful unit of language, and used for a specific purpose or function” (Dipper et al., 2021, p. 1). Impaired discourse production can have a negative impact on everyday life participation (Croteau et al., 2020). Discourse measures may thus best predict everyday life communicative success. In addition to being ecologically valid, discourse measures seem to be sensitive to detect therapy responses (Richardson & Dalton, 2015; Wright, 2011).

The Amsterdam-Nijmegen Test for Everyday Language (ANELT; Blomert et al., 1995) elicits spoken discourse in Dutch persons with aphasia (PWA). In this test, PWA are presented with scenarios of everyday life situations, such as: “Suppose you have an appointment with your family doctor; however, something else has come up. Therefore you need to reschedule your appointment. What would you say if you called the doctor?” The test has a standardised procedure, norm samples, and two parallel versions of each 10 scenarios are available. The scenarios have a script-like character. That is, the experimenter is an involved listener who does not engage in conversation. As a result, the ANELT yields monologic discourse, which allows the investigation of macro-structural aspects of the discourse (Kong, 2016). As will be discussed more thoroughly below, these discourse aspects are strongly related to the goal-oriented nature of communication. That is, (spoken) language is typically motivated by a clear goal that the speaker is trying to achieve, such as to inform, persuade, instruct, or motivate the listener.

Because producing monologues in the ANELT involves macro-structure planning, which is related to successful communication, we consider the ANELT to be a test of functional communication. Although functional communication is considered one of the most important outcome measures of both aphasia recovery and therapy (Dalton & Richardson, 2019), it is debated how it should be defined and measured (Barnes & Bloch, 2019; Doedens & Meteyard, 2022). In this article, we define functional communication as the skill to get the message across independently, effectively (e.g., Doedens & Meteyard, 2020; Wallace et al., 2017) as well as efficiently (Kong, 2009; Ruiter et al., 2011) in communicative contexts that are representative of everyday life. Because of its focus on communicative success, functional communication is neither related to the mode of communication used to convey the message (American Speech-Language-Hearing Association, cited in Frattali, 1992, p. 64) nor - in case of spoken communication - to the well-formedness of the utterances used (Blomert et al., 1994). Since only spoken responses are taken into account, this test is a measure of verbal functional communication (Schumacher et al., 2020). The ANELT as published by Blomert et al. (1995; henceforward referred to as the ‘traditional ANELT’) does not include measures of verbal efficiency (i.e., average amount of essential information produced per time unit); instead, the focus is on verbal effectiveness (i.e., comprehensibility of the spoken message) only.

There are two important considerations in the analysis of monologic discourse elicited with the ANELT. Firstly, the operationalisation of the communicative aspects of interest and, secondly, whether it should be analysed qualitatively or quantitatively. In the next sections, we will discuss these considerations one after another, starting with the way of analysis for reasons of simplicity.

Qualitative versus quantitative analysis of elicited discourse with the ANELT

Discourse analyses can be split broadly into qualitative and quantitative methods. Qualitative analyses, in which the scoring is based on a personal judgement, are typically related to transcription-less methods of discourse analysis. That is, the (subjective) analysis or judgement is made directly from the audio recording (Armstrong et al., 2007). Quantitative analyses, on the other hand, seek to assess specific communicative behaviours in terms of a numerical value, which often requires transcription of the spoken responses first. Therefore, quantitative analyses are typically associated with transcription-based methods of discourse analysis.

The scoring of the traditional ANELT can be considered qualitative. The PWA’s verbal responses are scored on two scales: the Comprehensibility A-Scale (i.e., verbal effectiveness) and the Intelligibility B-Scale. Each ordinal scale has five points ranging from very bad to very good. In effectiveness and efficacy studies, the A-scale is often used exclusively as an outcome measure of verbal effectiveness (e.g., Bastiaanse et al., 2006). For that reason, we only discuss the ANELT A-scale here. The traditional ANELT A-scale is concerned with subjective judgement because no external criteria are provided for assigning scores. That is, the kind and amount of information that is considered essential in order to achieve the communicative goal in each scenario is not specified (for more detail, see Ruiter et al., 2011). Moreover, experimenters assign a score on the A-scale directly from the audio recording. Therefore, the scoring method of the traditional ANELT is transcription-less.

Although transcription-less qualitative analyses are less time-consuming than transcription-based quantitative ones, the latter methods seem to be more sensitive to detect changes in communicative behaviour over time. Evidence comes from Ruiter et al. (2011), who developed a transcription-based quantitative score of verbal effectiveness as measured with the Dutch version of the traditional ANELT. Their scoring system is based on the elements of meaning that are essential to achieve a communicative goal in each scenario. An adapted version of the Content Unit analysis (Yorkston & Beukelman, 1980) was used to establish verbal effectiveness in orthographic transcribed responses to the traditional ANELT. In line with the Content Unit analysis, (groupings of) information underlying propositions are labelled as Content Units (CUs). The list of CUs for the traditional ANELT includes a list of concepts (i.e., semantic elements) that at least 30% of the 24 non-linguistically impaired speakers of Dutch had produced to the traditional ANELT. Therefore, this quantitative scoring system is also criterion-based. The main results of Ruiter et al. suggest that the quantitative measure is more sensitive to detect change in PWAs’ verbal effectiveness over time as compared to the qualitative scoring procedure. In addition, the quantitative method allows derivation of a measure of verbal efficiency, which gives a more complete picture of PWAs’ verbal functional communication skills. In sum, the results obtained by Ruiter et al. suggest that the construct validity of the ANELT can be further improved by substituting the qualitative scoring method with a quantitative one.

As was mentioned above, not only the choice between a qualitative or quantitative analysis is of relevance. Which communicative aspect is of interest and how it should be operationally defined should be specified as well. These latter aspects will be addressed in the next section.

Communicative aspects of interest in spoken discourse

According to Dipper et al. (2021) there is little use of theoretical models that underpin the assessment and treatment of discourse production in aphasia. To fill that gap, these authors introduced a discourse production framework, called the Linguistic Underpinnings of Narrative in Aphasia (LUNA). The LUNA framework consists of four processing components, which may interact. The first component, the ‘pragmatics component’, includes the environmental, interpersonal, and interactional factors (e.g., formal versus informal setting, familiar versus unfamiliar conversational partner) that a speaker must take into account when producing spoken discourse.

Secondly, ‘macrostructure planning’ encompasses the creation of an organisational frame for the discourse, which represents the global meaning of discourse (e.g., topic, theme, or gist). Macrostructure planning is integrally related to the concept of genres, which can be described as ‘[..] how things get done, when language is used to accomplish them’ (Martin, 1985, p. 250). That is, discourse is structured according to particular generic frameworks, which strongly depend on their communicative purpose (i.e., what people want or need to do through communication). In order to achieve their desired communicative purpose(s), the following steps are required. First of all, it should be specified what key information must be included in the message and, secondly, in which order they should be conveyed. Thus, the output of macroplanning is an ordered sequence of intentional actions that are performed by means of verbal communication, also called speech-act intentions (Levelt, 1989).

In the third step, called the ‘propositional component’, the macrostructure is translated into a microstructure. At this level a preverbal message is created that will facilitate access to the linguistic system (Levelt, 1989). Each intended speech act is given informational perspective (i.e., topic, focus, or new), a propositional format, and all the features that are obligatory for a preverbal message are assigned. Thus, local decisions are made about entities to either include or exclude from propositions. According to Dipper et al. (2021) the propositional component is linked to Slobin’s (1996) notion of ‘thinking before speaking’ and Levelt’s (1989) notion of ‘conceptualisation’. The outcome of microplanning is - for each intended speech act - a preverbal message.

In the last step, called the ‘linguistic component’, each proposition (and the relations between them) is grammatically encoded, and words are accessed (Levelt, 1989).

The quantitative analysis for the traditional ANELT (Ruiter et al., 2011) predominantly focusses on the propositional component of the LUNA framework (Dipper et al., 2021). However, pinpointing the ANELT-CU as a pure measure of proposition-level would not be straightforward, since missing and/or incorrectly ordered concepts generally reduce overall narrative coherence. As such, proposition-level measures also relate to aspects of macrostructure planning (Richardson & Dalton, 2015).

Limitations to clinical implementation

As was discussed in the previous sections, the (criterion-based) quantitative scoring method (ANELT-CU, Ruiter et al., 2011) can be theoretically underpinned with the LUNA framework (Dipper et al., 2021). In addition, the ANELT-CU scoring method improves the construct validity of the traditional ANELT in comparison to the qualitative one (Blomert et al., 1995). For these reasons, the ANELT-CU may be beneficial in both academic and clinical practice in order to assess change in verbal effectiveness over time. There is, however, one important drawback to its clinical implementation, which is the fact that it is time-consuming. As the ANELT-CU requires transcription of the speech sample in order to count the produced CUs, time needed for scoring is almost doubled compared to the traditional ANELT. A transcription-less approach to this quantitative discourse analysis could improve its clinical applicability (Armstrong et al., 2007).

Research aims

In order to facilitate its clinical utilisation, we assessed the potential of a transcription-less quantitative analysis as a valid and reliable procedure for the measurement of verbal effectiveness with an adapted version of the ANELT (Ruiter et al., 2016). As will be extensively discussed in the following sections, modifications to the test items of the traditional ANELT were made, because some scenarios no longer seemed ecologically valid. Monologic discourse elicited with the adapted version of the ANELT (Ruiter et al., 2016) was analysed with both the transcription-based method (i.e., ‘ANELT-CU+Tr’) and the transcription-less one (i.e., ‘ANELT-CU-Tr’). By systematically comparing the resulting data, we investigated the following psychometric properties of the ANELT-CU-Tr:

1. Internal consistency: Do all scenarios of the adapted version of the ANELT (Ruiter et al., 2016), as measured with the ANELT-CU-Tr, measure the concept of verbal effectiveness in PWA and how is this related to the internal consistency of the ANELT-CU+Tr?

2. Inter-rater reliability: Does the ANELT-CU-Tr yield at least the same inter-rater reliability as the ANELT-CU+Tr in PWA?

3. Construct validity: To what extent can PWAs’ responses to the ANELT-CU-Tr be explained by the same common underlying dimension(s) as the ANELT-CU+Tr?

4. Convergent validity: Do the ANELT-CU-Tr scores of verbal effectiveness correspond to the transcription-based ones in PWA?

5. Known-group validity: Does the ANELT-CU-Tr allow experimenters to reliably tell PWA apart from neurologically healthy speakers (NHS)?

To conclude, the current study both replicates and expands the one conducted by Ruiter et al. (2011). Since some PWA that participated in the current study were only presented with the scenarios of ANELT parallel version I, focus is on this parallel version of the ANELT only (although the scription-less ANELT-CU scoring is available for parallel version II as well).

Materials and methods

Design and participants

The current study can be considered corpus-based research. In total, the data of 56 speakers of Dutch were analysed: 31 NHS and 25 PWA. The following three (selections of) corpora were used:

1) Corpus of neurologically healthy speakers of Dutch (n = 31) (Dassek, 2016; Zwartjens, 2017). These data were collected with a split-plot design with one within-participant factor Time: T1 and T2. The time interval between T1 and T2 was 8 weeks, in which the participants did not receive any language intervention. Both ANELT-CU parallel version I and II (Ruiter et al., 2016) were administered at each point in time (counterbalanced across participants). Since data collection was part of the Radboud University degree programme (i.e., Master’s specialisation Language and Speech Pathology, Faculty of Arts), no ethical approval was required (at that time). The participants’ age varied between 25 and 86 years (M = 49, SD = 17) and 16 of the 31 participants were male. Their level of education (Verhage, 1964) ranged from 3 to 7 (Mdn = 6). None of these participants had neurological damage, severe visual or hearing disorders (not correctable by standard glasses, contact lenses, or hearing aids) and none had participated in the Ruiter et al. (2011) study.

2) ‘Language in the brain’ corpus of Dutch-speaking persons with aphasia (n = 17). A selection of the data collected in the ongoing project ‘Language in the brain, from left to right’ was used. This project was started in 2017 by the third author (VP) and aims to investigate how the right-hemisphere language system works in persons who suffered damage to their language-dominant left hemisphere. Among other measurements of the structure and activity of the brain as well as tests of language performance, ANELT-CU parallel version I (Ruiter et al., 2016) was administered to participants at one point in time. An accredited medical research ethics committee (CMO region Arnhem-Nijmegen) approved the study (number NL58437.091.17).

The characteristics of the participants are presented in Table 1. Their age varied between 24 and 78 years, and 11 participants were male. For the current study, relevant inclusion criteria that all participants fulfilled were: native speaker of Dutch, no history of previous stroke, and (with normed aphasia tests) objectifiable symptoms of aphasia in the subacute phase. Although it was not necessary that aphasia symptoms were objectifiable in order to be included in the ‘Language in the brain, from left to right’-study, only the data of the participants who still presented aphasia symptoms were used in the current study. Relevant exclusion criteria were: not correctable visual or hearing disorders, neurogenic or psychiatric disorders (other than stroke), and cognitive disorders other than aphasia. The participants were able to provide consent and understand test instructions. This was operationally defined as a percentile score of at least 24 on specific subtests of the Dutch version of the Comprehensive Aphasia Test (CAT-NL; Visch-Brink et al., 2014) and the Dutch version of the Aachen Aphasia Test (AAT; Graetz et al., 1992).ⁱ

Table 1. Characteristics of PWA ‘Language in the brain’ corpus (n = 17)

		Age	TPO	Clinical impression	Aetiology ^c	Secondary	Education
PWA		(yrs) ^a	(yrs) ^b	severity of aphasia		diagnosis ^d	(Verhage, 1964)
31	f	61	2;4	residual	ICVA LH	n/a	4
32	m	58	~ 1;5	residual	ICVA LH	n/a	6
40	m	36	5;0	residual	ICVA LH	n/a	6
41	m	59	0;8	residual	ICVA LH	n/a	7
42	f	40	2;4	moderate	ICVA LH	AOS	6
43 ^e	m	76	md	severe	ICVA LH	n/a	md
45	m	69	25;3	residual	HCVA LH	n/a	4
46	f	24	2;9	severe	ICVA LH	n/a	5
47	m	59	1;7	moderate	ICVA LH	n/a	6
50	m	54	~ 3;0	mild	ICVA RH	n/a	6
51	f	70	6;7	moderate	ICVA LH	n/a	5
52	m	57	2;0	severe	ICVA LH	AOS	6
53	m	66	2;5	mild	ICVA LH	n/a	5
54	m	54	4;2	moderate	TACI LH	n/a	md
55	f	53	12;8	moderate	ICVA LH	n/a	5
56	f	54	23;8	moderate	ICVA LH	n/a	5
57	m	78	4;10	moderate	ICVA LH	n/a	5
M		56;11	6;4				Mdn 5
SD		13;11	7;8				Range 4 to 7

Notes. All participants were premorbidly right-handed, except for participant 50. ^{a, b} Years (yrs) refer to the age versus time post-onset (TPO) during the ANELT administration. ^c ICVA = ischemic cerebrovascular accident, TACI = total anterior circulation infarct, HCVA = hemorrhagic cerebrovascular accident, LH = left hemisphere, RH = right hemisphere. ^d AOS = apraxia of speech. ^e For PWA 43, the exact TPO is unknown, but well into chronic phase (at least 6 months post-onset). His level of education was also unknown (but probably 3 or below).

3) ‘SimpTell’ corpus of Dutch-speaking persons with aphasia (n = 8). The data of these eight PWA were collected in an ongoing pilot-study into the effect of a therapeutic app (SimpTell) on the verbal functional communication of persons with chronic expressive agrammatism (Ruiter et al., in preparation). In this study, a multiple single case design with pre- and post-therapy measurements (T1 and T2 respectively) was used in order to investigate whether SimpTell allows persons with chronic aphasia to enhance elliptical style and consequently to improve verbal functional communication (for the study’s rationale, see Ruiter et al., 2010). Among other tests of language performance, Ruiter et al.’s (2016) ANELT-CU parallel version I was administered at T1. The data of participant 3 and 7 were not included, because both participants had also participated in the ‘Language in the brain, from left to right’- study (see above). The SimpTell study was approved by the Ethics Assessment Committee of the Faculty of Arts and the Faculty of Philosophy, Theology and Religious Studies of Radboud University (ETC-GW 2019-8305) after this study was exempt from formal medical-ethical approval by an accredited medical research ethics committee (CMO region Arnhem-Nijmegen, 2019-5507).

The characteristics of the eight participants with aphasia are presented in Table 2. The participants’ age varied between 44;8 and 62;1 years and six of them were male. Education level data were not collected in this study. For the current study, relevant inclusion criteria that all participants fulfilled were: unilateral left hemispheric stroke, at least 6 months post-onset (chronic phase), native speaker of Dutch, and no history of previous stroke. Disordered sentence production (expressive agrammatism) due to aphasia was present in all participants as well. Relevant exclusion criteria were not correctable visual or hearing disorders, neurogenic or psychiatric disorders (other than stroke), and cognitive disorders other than aphasia. With exception of AAT Token Test version, the same criteria as in the project ‘Language in the brain, from left to right’ (described above) were used to ensure that participants were able to provide consent and understand test instructions.

Table 2. Characteristics of the PWA ‘SimpTell’ corpus (n = 8)

		Age	TPO	Syntactic	Aphasia severity (STT) ^d		Secondary	Aetiology^f	Handedness
PWA		(yrs)^a	(yrs)^b	structure^c	(raw score & interpretation)		diagnosis ^e
1	m	56;10	2;6	1	25.5	mild	AOS	TACI, LH	right
2	f	62;1	1;9	2	22.5	mild	n/a	ICVA, LH	right
4	m	46;6	5;2	2	32.5	no aphasia	AOS	ICVA, LH	right
5	m	57;8	6;10	1	n/a	n/a ^g	AOS	ICVA, LH	mixed
6	f	55;9	8;8	2	31.5	no aphasia	AOS	ICVA, LH	left
8	m	57;9	6;4	1	16.0	severe	AOS	ICVA, LH	right
9	m	49;6	1;0	1	18.5	moderate	AOS	ICVA, LH	mixed
10	m	44;8	17;2	1	25.0	mild	n/a	ICVA, LH	right
M		53;10	6;2		24.5	mild
SD		6;2	5;2		6.1

Notes. ^{a, b} Years (yrs) refer to the age versus time post-onset (TPO) during the pre-therapy measurements (T1). ^c Syntactic structure of participants’ speech output rated on the 6-point scale of the subtest Spontaneous Speech of the Dutch Aachen Aphasia Test (AAT; Graetz, De Bleser, & Willmes, 1992). 1 = mostly one or two word utterances; almost no inflection forms or function words; 2 = short, simple sentences, which are mostly syntactically incomplete; frequent absence of function words and inflected forms. ^d Severity of aphasia is expressed as the raw score on the shortened (36-item) version of the Token Test (STT; De Renzi & Faglioni, 1978). The notion’no aphasia’means that the PWA scored at or above the cut-off score for aphasia, which means that receptive language impairments could not/no longer be objectified with the STT. ^e Aphasia accompanied by Apraxia of speech (AOS); however, at least 95% intelligible speech output. ^f TACI = Total anterior circulation infarct, ICVA = ischemic cerebrovascular accident, HCVA = hemorrhagic cerebrovascular accident. ^g Because of colour blinedness the STT could not be administered in this participant. For that reason he was presented with some test items (only those that did not contain coloured pictures) of Subtest 9 and 10 of the CAT-NL (Visch-Brink et al., 2014), on which he scored 20/26 and 14/24 respectively. Even if he could have given a correct response to the test items that were not adminstered, he would have scored below cut-off (aphasia objectifiable, severity mild to moderate).

Assessment method: ANELT-CU

Modification of test items used in the traditional ANELT-CU. In the current study, both a transcription-based and transcription-less version of ANELT-CU (Ruiter et al., 2016) were used. Before describing both methods in more detail, we would like to point out that the ANELT-CU version used in the current study was different from the one used for the traditional ANELT in the 2011 study by Ruiter et al., because qualitative research into the representativeness and imaginability of the scenarios used in the traditional ANELT had revealed that NHS of Dutch (N = 54) found some scenarios no longer ecologically valid (Filipinski, 2014).

An example of a scenario that was found to be less imaginable is scenario 1 (Shoe) of parallel version I of the traditional ANELT (Blomert et al., 1995), given in (1). The reason why this scenario was less plausible is that it is unlikely that you will spend money on only half a repair of your shoe.

1. U gaat met deze schoen naar de schoenmaker [toon voorwerp: schoen]. Er is veel aan de hand met de schoen, maar om een of andere reden wilt u slechts één ding laten repareren. U mag kiezen wat. Wat zegt u?

‘You take this shoe [Show object: shoe] to the shoemaker. There is a lot going on with the shoe, but for some reason you only want one thing fixed. You can choose what. What would you say?’

In several Master’s and Bachelor’s theses (Aan de Stegge, 2015; Dassek, 2016; Giessen, 2015; Zwartjens, 2017) under supervision of authors MR, EL and TR, we worked towards adapted versions for 17 of the 20 ANELT scenarios (both parallel versions). Firstly, on the basis of Filipinski (2014), we modified the text of 17 scenarios, either by replacing some words or sometimes replacing entire test items by new ones. For example, scenario 1 (Shoe) of parallel version I was adapted by deleting the part of wanting to repair only half of the shoe, as given in (2).

• (2) U gaat met deze schoen [toon afbeelding 2: schoen met kapotte hak/losse zool] naar de schoenmaker om hem te laten repareren. Wat zegt u?

‘You take this shoe [Present picture 2: shoe with broken heel/loose sole] to the shoemaker to have it repaired. What would you say?’

In addition, the objects that are an integral part of some of the scenarios of the traditional ANELT (e.g., a broken shoe in scenario 1) were replaced by standard pictures of these objects, as the traditional ANELT is sold without the test objects, and experimenters have to select (or create) these materials themselves. Consequently, there is considerable variation in the objects used in the traditional ANELT.

Both the scenarios of the traditional ANELT and the modified scenarios were administered to 60 NHS of Dutch (Aan de Stegge, 2015; Giessen, 2015). Afterward, the participants rated imaginability and representativeness of both the old and new scenarios on a 6-point scale. Moreover, they were asked to indicate who they were to play in each scenario (i.e., clarity of instruction). None of the participants had participated in the study by Filipinski (2014). The participants’ age varied between 30 and 78 years (M = 51;9, SD = 11;4), and 30 were male. Their level of education (Verhage, 1964) ranged from 2 to 7 (Mdn = 6).

The results showed that 6 out of the 17 new scenarios were rated significantly more representative of current daily life and more imaginable than the scenarios of the traditional ANELT. The remaining 13 items were found to be as imaginable, representative for current daily life, and clearly formulated as the scenarios of ANELT traditional. In the remaining of the current study, the adapted scenarios of ANELT-CU (Ruiter et al., 2016) were used, but we would like to recall that a CU-based scoring system for the traditional ANELT is available as well (see Ruiter et al., 2011).

Quantitative scoring for the adapted ANELT scenarios. A quantitative scoring method was developed for the adapted ANELT scenarios (Dassek, 2016; Otters, 2019; Zwartjens, 2017). The procedure for developing the quantitative scoring method was similar to the one used by Ruiter et al. (2011). The list of essential CUs was derived from the orthographic transcriptions of the verbal responses that the 60 NHS of Dutch who had participated in the studies by Giessen (2015) and Aan de Stegge (2015) had given to the modified ANELT scenarios.

All preambles (i.e., the reason for starting communication) and requests (i.e., the communicative goal to be achieved) that were produced by at least 30% of the NHS were included in the scoring scheme. These alternative routes are indicated with ‘OR’ in Table 3. In each row the proposition underlying preambles and requests is divided into CUs. For ease of use, each preamble and request was also presented as a full sentence as well at the beginning of each row and CUs were systematically ordered according to the ‘who-doing-what-where-when’ format (or modifications of this form).

Table 3. ANELT-CU scenario 1 (Shoe) translated from Dutch (Ruiter et al., 2016)

PreambleThe heel/sole of the shoe needs to be repaired	● WHAT (CU 1)	● WHAT (CU 2)	Score 2/2
	- Heel/sole - That ^1	- Worn out/broken - In need of reparation
Request
I want to know how much it costs.	O WHO (CU 3)	O DOING (CU 4)	O WHAT (CU 5)	Score 1/ 3
	- I	- Knowing	- Costs
OR
The employee indicates how much it costs	O WHO (CU 3)	O DOING (CU 4)	O WHAT (CU 5)
	- You	- Indicating	- Costs
OR
The employee has to repair the shoe.	O WHO (CU 3)	● DOING (CU 4)	O WHAT (CU 5)
	- You	- Repairing	- Shoe/heel/sole - That
				Total score: 3/5

Note. ¹ (Only) in scenarios in which pictures are used, demonstrative pronouns are scored as a CU in the preamble. In those cases, the use of demonstrative pronouns should be combined with pointing to the picture. In scenarios without pictures, demonstrative pronouns are only scored as CUs if the antecedent is explicitly stated in one of the preceding utterances.

The ANELT-CU+Tr is illustrated with the spoken response that one PWA (from the SimpTell corpus) gave to ANELT-CU scenario 1, as given in (3).

• (3) Mijn zool is kapot eh vervangen.

‘My sole _{CU 1} is broken _{CU 2} uh replace _CU4’

Table 3 presents the quantitative scoring of this participant’s spoken response. Since three out of the 5 CUs were produced correctly, verbal effectiveness is 60%.

The 17 new ANELT-CU scenarios (Ruiter et al., 2016) were redistributed over the two parallel versions thereby seeking to divide the number of CUs to be achieved as well as the scenarios with pictures evenly. A short protocol was made for the scoring procedure.

Procedure

Monologic discourse elicited with parallel version I of the adapted ANELT scenarios (Ruiter et al., 2016) was analysed with both the ANELT-CU+Tr and the ANELT-CU-Tr. In line with the traditional ANELT (Blomert et al., 1995) each and every scenario could be played back only once. The same scoring scheme (see Table 3 for an example) was used for both methods of analysis.

The fifth author (JT) scored the responses of all NHS (n = 31) and PWA (n = 25) with both scoring methods. The second author (MO) scored the responses of 8 PWA from the ‘Language in the brain’ corpus (31-46). The transcription-less method was conducted first and the time interval between the transcription-less and transcription-based analyses was at least two weeks. In advance, both raters had been trained by the first author (MR) in applying both methods of analysis, using the responses that two NHS and two PWA had given to two scenarios. The data of these four speakers were not used in the remaining of this study. Moreover, the raters were provided with a short manual on how to apply both scoring methods.

Outcome measures and statistical analyses

The outcome measure for both the ANELT-CU-Tr and ANELT-CU+Tr was the CU score (either raw score or percentage) on parallel version I (Ruiter et al., 2016). The maximum raw score of verbal effectiveness was 66 CUs. All statistical testing took place at the 0.05 level of significance (two-tailed) using IBM SPSS version 25.

Statistical power

For several reasons we could not conduct an a priori power analysis, that is, to calculate the minimum sample size required to achieve a statistical power of at least 80% at an alpha level of .05. Since both the ANELT-CU-Tr and ANELT-CU+Tr for the adapted version of the ANELT (Ruiter et al., 2016) constitute new methods for language assessment the magnitude of a clinically relevant difference (i.e., effect size) was unknown. This also held for the variance of the scores that both methods of analysis would yield. Instead, we included all data available at that time in the ongoing studies.

Reliability

Internal consistency. Internal consistency was calculated using two indices: a) the Intra-Class Correlation Coefficient ICC(3,k), which is also called Cronbach’s alpha and b) item-total correlations. Criteria for internal consistency were: Cronbach’s α > 0.70 and item-total correlations ≥ 0.30 (Streiner et al., 2015). The degree of interrelationship among the items was calculated for both the ANELT-CU+Tr and the ANELT-CU-Tr applied to the PWAs’ responses (n = 24, as scenario 3 was missing for participant 40).

Inter-rater reliability. Inter-rater reliability was assessed in two ways. Firstly, a two-way mixed, absolute agreement, single measures Intra-Class Correlation Coefficient (ICC, Koo & Li, 2016) was used to assess the degree that coders MO and JT provided the same CU scores to the PWAs’ responses, both with respect to overall scores as well as for each scenario separately. For both methods of analysis, inter-rater reliability was calculated over eight PWA (subj. 31-46) from the ‘Language in the brain’ corpus. Due to missing data, the ICC for scenario 3 was calculated over seven participants.

Secondly, Otters (2019) assessed inter-rater reliability in five speech and language therapists (SLTs), trained in the transcription-based and transcription-less scoring by author MO. The SLTs had graduated 1 to 3 years before, were between 23 and 25 years old and were all female. Two-way mixed, absolute agreement, average measures ICCs (Koo & Li, 2016) were used to assess the degree that the raters provided the same CU scores to the responses of 4 PWA (participants 31, 32, 41 and 46 from the ‘Language in the brain’ corpus) to four scenarios (4, 5, 6, and 10).

The raters were not randomly chosen from a large population of raters, that is why we labelled them as a fixed factor. However, this does not preclude generalisation to other raters, as we are convinced that the raters could be replaced by other raters who should receive the same amount of training.

We followed the often used guidelines provided by Cicchetti (1994) for labelling the values of the obtained reliability coefficients: ICC’s < .40 were considered poor, between .40 and .59 fair, between .60 and .74 good, and excellent for values ≥ .75.

Validity

Construct validity. Principal axis factor analyses with oblique rotation (direct oblimin) were conducted to examine whether the ANELT-CU-Tr and ANELT-CU+Tr have the same underlying dimensions. Both the Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity were used to test whether the data were suitable for factor analysis. The diagonal of the anti-correlation matrix was also inspected to verify whether the KMO values for the individual scenarios were above the acceptable limit of .5 (Field, 2013). For factor extraction, we used Kaiser’s criteria combined with the criterion that the percentage of nonredundant residuals with absolute values greater than .05 should be less than 50%. The analyses were conducted on the scores of 24 PWA because participant 40 had missing data for scenario 3.

Convergent validity. Two-tailed Studentized Permutation Tests for paired data (Konietschke & Pauly, 2012) were performed to test difference in performance on the ANELT-CU-Tr and ANELT-CU+Tr in both groups (PWA and NHS). The test was performed with 1000 permutations per group. This test performs robustly against violations of the conventional t-test (Rietveld & van Hout, 2017). In addition, correlation between both tests was calculated for both overall scores as well as for each of the 10 scenarios separately. Spearman’s ρ correlation coefficient was used because the Shapiro-Wilk test had revealed that the distribution of verbal effectiveness departed significantly from normality for some outcome measuresⁱⁱ. Due to missing data for participant 40, Spearman’s ρ for scenario 3 was calculated over 24 PWA.

Known-group validity. Bootstrapped Welch’s t tests for independent samples were used to test whether the ANELT-CU-Tr was similar to the ANELT-CU+Tr in demonstrating different total scores for respectively the PWA (n = 25) and the NHS (n = 31). Bootstrapped Welch’s t tests, using 1000 bootstrap samples, were used as Levene’s tests had shown that the assumption of homogeneity was not met for all CU scores.

Results

Reliability

Internal consistency. For the ANELT-CU-Tr Cronbach’s α was .874, 95%CI [.782, .937], p < .01 and item-total correlations ranged from 0.452 to 0.888. This was quite similar to the ANELT-CU+Tr for which Cronbach’s α equalled .878, 95%CI [.789, .939], p < .01 and item-total correlations ranged from 0.494 to 0.900. For both methods of analysis Tukey’s test of additivity was not significant, which indicates that there was no interaction between the scenarios and the raters. These results indicate that the internal consistency of both methods of analysis is good.

Inter-rater reliability. As Table 4 shows, the single measures intra-class correlations (ICCs) for both the ANELT-CU-Tr and the ANELT-CU+Tr were in the good or excellent range except for scenario 10. For this scenario, the ICC for the ANELT-CU-Tr was only fair, ICC = 0.424, whereas it was excellent, for the ANELT-CU+Tr, ICC = 0.800. For both methods of analysis Tukey’s tests of additivity were not significant, which indicates that there was no interaction between the raters and the scenarios.

Table 4. Intra-class correlations (ICCs) for ANELT-CU-Tr and ANELT-CU+Tr

	ANELT-CU-Tr			ANELT-CU+Tr
Scenario	ICC	95%CI	p value	ICC	95%CI	p value
	Single measures (two raters)
Overall	.973	[.769, .995]	< 0.01	.983	[.903, .997]	< 0.01
1_Shoe	.655	[-.020, .920]	< 0.05	.709	[.040, .935]	< 0.05
2_Socker	.834	[.355, .965]	< 0.01	.974	[.876, .995]	< 0.01
3_Appointment	.857	[.401, .974]	< 0.01	.935	[.492, .989]	< 0.01
4_Glasses	.961	[.823, .992]	< 0.01	.875	[.500, .974]	< 0.01
5_Friend	.991	[.962, .998]	< 0.01	.991	[.958, .998]	< 0.01
6_Bouquet	.910	[.631, .981]	< 0.01	.945	[.762, .989]	< 0.01
7_Birthday	.768	[.218, .948]	< 0.01	.881	[.519, .975]	< 0.01
8_Child	.985	[.932, .997]	< 0.01	.973	[.873, .995]	< 0.01
9_Reminder	.897	[.524, .979]	< 0.01	.867	[.506, .971]	< 0.01
10_Local authority	.424	[-.270, .846]	ns	.800	[.330, .956]	< 0.01
	Average measures (five raters)
Overall	.960	[.918, .984]	< 0.01	.906	[.796, .963]	< 0.01
4_Glasses	.989	[.955, .999]	< 0.01	.889	[.562, .992]	< 0.01
5_Friend	.982	[.926, .999]	< 0.01	.895	[.586, .993]	< 0.01
6_Bouquet	.862	[.320, .991]	0.01	.930	[.719, .995]	< 0.01
10_Local authority	.861	[.470, .990]	< 0.01	.887	[.546, .992]	< 0.01

Notes. For explanation see text.

The average measures ICCS were in the excellent range (see Table 4). Tukey’s tests of additivity were significant for the transcription-based analysis of scenarios 5, 6 as well as the four scenarios together. This also held for the transcription-less analysis of scenario 4, indicating interaction between the raters and the scenarios.

These results indicate that coders typically assigned similar scores on the ANELT-CU+Tr and that – except for one scenario –the ANELT-CU-Tr yielded similar measures of inter-rater reliability as the transcription-based one.

Validity

Construct validity. Factor analyses were carried out for both the ANELT-CU-Tr and the ANELT-CU+Tr. The overall Kaiser-Meyer-Olkin (KMO) statistics for the ANELT-CUs were .75 and .72 respectively, and the KMO values for the individual scenarios of both ANELT-CUs were greater than 0.5; Bartlett’s tests were significant.

The analyses for the ANELT-CU-Tr and the ANELT-CU+Tr resulted in two factors with eigenvalues over Kaiser’s criterion of 1; these factors together explained respectively 63.14% and 63.59% of the variance (see Table 5). The average communality for the ANELT-CU-Tr was .55 for the ANELT-CU+Tr .56. The correlations between the extracted factors was -.50 for the ANELT-CU-Tr and -.49 for the ANELT-CU+Tr: both clear negative associations, which suggest for both ANELT-CUs dependence between the two extracted factors and the use of oblique rotation.

Table 5. Summary of the results obtained with the exploratory factor analyses for both the ANELT-CU-Tr and the ANELT-CU+Tr (n = 24)

	Rotated Factor Loadings
	ANELT-CU-Tr		ANELT-CU+Tr
Scenario	Factor 1	Factor 2	Factor 1	Factor 2
1_Shoe	.87	-.12	.84	.22
2_Socker	.86	-.11	.76	-.13
3_Appointment	.73	.36	.60	-.06
4_Glasses	.51	.22	.56	-.23
5_Friend	.45	.25	.52	-.29
6_Bouquet	-.18	.88	.39	-.34
7_Birthday	.10	.65	-.14	-.93
8_Child	.03	.57	.06	-.77
9_Reminder	.31	.51	.51	-.61
10_Local authority	.30	.48	.23	-.43
Eigenvalues	4.92	1.39	5.01	1.35
% of variance	49.21	13.93	50.07	13.52

Notes. Factor loadings over 0.40 appear in bold.

The loadings of the individual scenarios on these factors are presented in Table 5. Apart from scenarios 6 and 9, the individual scenarios correlated with the same factors in both scoring methods. More specifically, scenarios 1 to 5 seem to load on Factor 1 and 7 to 10 on Factor 2. Because of the (relatively) strong correlations between both factors in both methods of analysis, we interpret both factors to indicate ‘verbal functional communication skills’; however, they seem to differ with respect to ‘discourse complexity’. That is, scenarios 1 to 5 seem to require less macrostructure planning than scenarios 7 to 10. In the former scenarios, the intentional actions that need to be performed by means of verbal communication can be derived from the experimenter’s description of the scenario (i.e., to reschedule an appointment in scenario 3), whereas in the latter scenarios the intentional actions are less clearly specified. Participants are required to determine the communicative purpose first. For example, in scenario 7 participants need to decline an invitation to a birthday party. They therefore have to determine whether they want to tell the truth (i.e., not wanting to join the birthday party) or to tell a lie (e.g., not being able to make it) and whether the latter is socially acceptable. Discourse complexity seems to be further increased because these responses can be considered disagreement (dispreferred) responses, as the participants’ responses will not align with the description of the scenario given by the experimenter (Jones, 2019).

Convergent validity. The two-tailed Studentized Permutation Tests for paired data showed that the differences between both methods of analysis were not significant for both the PWA, t = 0.42, p = 0.616, relative effect 0.505 as well as the NHS, t = -1.02, p = 0.239, relative effect 0.486. Tukey’s tests of additivity were not significant. In line with these findings, both for the overall score as well as for each scenario separately, the ANELT-CU-Tr was significantly (p < 0.01) related to the ANELT-CU+Tr. The correlations ranged between .957 and 1.00, which indicates a very strong, positive relationship between both scoring methods.

Known-group validity. Differences between PWA and NHS on both scoring methods are presented in Table 6. Known-group validity was considered good for both the ANELT-CU-TR and the ANELT-CU+Tr as NHS produced significantly more CUs than PWA (p < .001).

Table 6. Overall percentage CUs (SD) for both neurologically healthy speakers (NHS, n = 31) and Persons with aphasia (PWA, n = 25)

Outcome measure	NHS		PWA		M difference	Bca 95% CI ^a	Bootstrapped Welch’s t test
ANELT-CU-Tr	63.64	(10.48)	43.99	(19.26)	- 19.65	[-27.96, -11.33]	t (1,35) = 21.003, p = .000
ANELT-CU+Tr	64.03	(10.25)	44.41	(19.16)	- 19.62	[-28.01, -12.19]	t (1,25) = 21.310, p = .000

Note. Degrees of freedom are rounded to the nearest integer.

^aBca 95% CI = Bias corrected accelerated 95% confidence interval

Discussion

The study reported here investigated the potential of a transcription-less quantitative analysis (ANELT-CU-Tr) as a valid and reliable procedure for the measurement of verbal effectiveness with an adapted version of the Dutch ANELT. Monologic discourse elicited with 10 scenarios from this test was analysed with both a transcription-less quantitative method and a transcription-based one (ANELT-CU+Tr). Resulting data were systematically compared on several psychometric properties.

Internal consistency and inter-rater reliability were good and comparable between both scoring methods, except for scenario 10 (Local Authority). With respect to validity, both scoring methods seem to yield measures of the same underlying constructs, show a strong and positive correlation, and allow differentiation between persons with and without aphasia. These findings are taken to suggest that the ANELT-CU-Tr can be reliably and validly used to analyse monologic discourse, which is in line with Armstrong et al. (2007). That is not to say, however, that the ANELT-CU-Tr – in its current form - can be implemented in clinical practice as a substitute of the traditional ANELT (Blomert et al., 1995) or the ANELT-CU+Tr (Ruiter et al., 2016). Nor does the ANELT-CU-Tr seem suitable for all aphasia syndromes. We will discuss these caveats below.

Firstly, the low (single measures) inter-rater reliability for scenario 10 (Local Authority) suggests that training could be enhanced, both in training material, guidelines, and practice. In scenario 10, the essential proposition can be encoded with other words than the ones that are given in the scoring scheme. We will illustrate this with scenario 1 (Shoe), because it was discussed extensively in the Method section. A PWA’s spoken response to scenario 1 could be: ‘Please, make sure sole no longer broken’. Such a response would adequately express CU4 and CU5 of the proposition The employee has to repair the shoe, even though the exact words are not given in the scoring scheme (see Table 3). Both the ANELT-CU-Tr and the ANELT-CU+Tr aim to yield scores of verbal effectiveness at the propositional level instead of linguistic encoding (propositional and linguistic component of the LUNA framework by Dipper et al., (2021) respectively)). Put differently, both scoring methods seek to quantify the amount of proposition or information content (i.e., degree of informativeness) in monologic discourse (cf., Kong, 2016). This is an essential aspect of both scoring methods, but apparently raters have more time to ‘think beyond the literal words’ when using the ANELT-CU+Tr (in comparison to the transcription-less one). Training could make it easier to apply the concept of propositional-level scoring in the transcription-less method. Guidelines for dealing with semantic and phonemic paraphasias could also be made more explicit.

The second aspect has to do with subgroups of PWA in which the ANELT-CU-Tr can be reliably and validly administered. As was pointed out by Kong (2016) analyses like the ANELT-CU may not provide specific information on the micro- and macro-linguistic structure of the produced discourse. Relevant for the discussion here is that according to the LUNA framework (Dipper et al., 2021) macroplanning not only involves specification of the key information to be included in the message, but also the order in which they should be conveyed. This latter aspect, which may be called coherence, is not analyzed with ANELT-CU. According to Linnik et al. (2015, p.776) coherence relates to the “unity, connectedness” of spoken discourse, including both the overall theme, goal, or plan of discourse (i.e., global coherence) and the conceptual links between individual propositions, which is called local coherence (Glosser & Deser, 1990).

The aphasiology literature is inconclusive about the extent to which coherence is preserved in PWA and the degree to which coherence production is related to severity of aphasia (for a discussion, for example see Linnik et al., 2015). If local coherence is impaired but PWA nevertheless produce the majority of the CUs, the ANELT-CU may yield an invalid measure of verbal functional communication. Despite the production of the relevant content units, comprehensibility of the message would be reduced due to impaired coherence production. Similarly, semantic jargon (e.g., ‘I toss a plate for sleeping’) and empty speech (i.e., spoken language with few meaningful words), typically associated with Wernicke’s aphasia, could yield invalid measures if some correct CUs are produced in an otherwise incomprehensible context. For these reasons, we argue that the ANELT-CU-Tr should only be used for PWA with mild to moderate aphasia and predominantly expressive disturbances (Weisenburg & McBride, 1935, cited by Ruiter et al., 2011), and not for speakers with severe expressive disturbances, such as Wernicke’s aphasia, who seem to be included in the study by Blomert et al. (1994). We would like to point out that the majority of the PWA that participated in the current study met this criterion. Only four of the 24 PWA had severe expressive language disorders (see Table 1 and 2). The rationale that the ANELT is more suitable for speakers with mildly to moderately expressive disturbances is in line with Schumacher et al. (2020). These authors administered both the English version of the traditional ANELT (Blomert et al., 1994) and the Scenario Test (van der Meulen et al., 2010) to 37 English-speaking PWA, who were all at least 1 year post-onset. The Scenario Test measures verbal and non-verbal functional communication in persons with moderate to (very) severe aphasia. Schumacher et al. found that the ANELT allowed for finer grading in PWA with ceiling scores on the Scenario Test, whereas the Scenario Test differentiated better between PWA with floor effects on the ANELT, suggesting that both tests are complementary. The Scenario Test is more suitable for persons with (very) severe aphasia and the ANELT for those with mild to moderate aphasia.

Thirdly, although the ANELT-CU-Tr is less time-consuming than the ANELT-CU+Tr, it yields a less complete picture of verbal functional communication. A previous study by Ruiter et al. (2011) revealed that both a measure of verbal effectiveness and verbal efficiency can be derived from the ANELT-CU+Tr, which is especially relevant when evaluating the effect of compensation therapy. The evaluation of changes in verbal efficiency over time requires a precise measure of PWA’s speech rate, operationalized as the (average) words produced per minute. This is needed to verify that the increase in average amount of CUs produced per minute (i.e., verbal efficiency) is larger than a possible increase in speech rate (Ruiter et al., 2010). Because an orthographic transcription is needed to calculate speech rate, this measure of verbal efficiency cannot be derived for the ANELT-CU-Tr. Although clinicians and researchers could still opt for the transcription-based ANELT when the calculation of verbal efficiency is essential, this drawback hinders clinical application of the ANELT-CU-Tr.

Lastly, in order to be clinically useful other relevant psychometric properties should be investigated, such as parallel-forms reliability of ANELT-CU version I and II and intra-rater agreement. Further research is also needed to establish normative scores. Critical differences are also needed for conducting studies into the efficacy and effectiveness of aphasia therapy. Only then the ANELT-CU-Tr may become a new standard for measuring verbal effectiveness in aphasic speakers with predominantly expressive disturbances.

In conclusion, although future research is needed to further improve clinical applicability of the ANELT-CU-Tr, the results obtained with the current study suggest that the ANELT-CU-Tr may provide a suitable alternative to the ANELT-CU+Tr and that this line of research seems a route to pursue.

Acknowledgements

We found like to thank all participants, Sharon aan de Stegge, Marjolein Bruijstens, Viola Bakx, Laura Bock, Sabrina Dassek, Nienke Dijkstra, Maartje Giessen, Franziska Filipinski, Renée Laurijssen, Lea Maessen, Tessa Spangenberg, Berber Spliethoff, and Judith Zwartjens for their contribution to this research project. We would also like to thank the two anonymous reviewers for their valuable and constructive comments on an earlier version of this paper.

Disclosure Statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The ‘Language in the brain, from left to right‘ study was supported by the Dutch Research Council (NWO) under Grant 451-17-003; The ‘SimpTell‘ study was supported by the Centre of LanguageStudies (Radboud University) under Grant RG2020-12.

Notes

1 For the CAT-NL, participants had to score at least 19 (out of 32) on subtest 9 [Comprehension of spoken sentences] and at least 15 (out of 32) on subtest 10 [Comprehension of written sentences]. In case the AAT was administered, participants had to score at least 34 (out of 60) on the first and seconds set of tasks of the subtest ‘Language Comprehension’, which consist of the comprehension of spoken words and sentences respectively. In addition, they had to score at least 31 (out of 60) on the third and fourth set of tasks of the subtest ‘Language Comprehension’ (comprehension of written words and sentences respectively). Lastly, the subtest ‘Token Test’ (50-item version) of the AAT was used as an inclusion criteria. Participants had to produce at least 10 correct responses to this subtest in order to be included.

² This held for six of the 11 outcome measures for the ANELT-CU+Tr and for four of the outcome measures for the ANELT-CU-Tr.