Translation, cultural adaptation, and validation of a screening test for cognitive and behavioural changes in amyotrophic lateral sclerosis

Abstract

Purpose

To describe challenges of translating and culturally adapting the Edinburgh Cognitive and Behavioural Amyotrophic Lateral Sclerosis (ALS) Screen into Norwegian (ECAS-N), evaluate its content validity; provide age- and education-balanced norms for verbal fluency and cut-off values for abnormal performance.

Materials and methods

Translation to Norwegian and back-translation to English complied with standard methods. Patients and ALS experts evaluated the relevance, comprehensiveness and comprehensibility of the translated ECAS (ECAS-N). Content validity indexes at the item level (I-CVI), scale level (S-CVI) and inter-rater agreement were calculated. Performance of controls determined norms for written and spoken verbal fluency (n = 559) and cut-off scores for abnormal performance (n = 85).

Results

High levels of content validity was achieved for all items of the ECAS-N, I-CVI, S-CVI and inter-rater agreement was 87.5%. Age- and education-balanced norms for written and spoken verbal fluency were produced. Cut-off scores of abnormal performance were slightly lower than the original ECAS.

Conclusions

The ECAS-N holds promise for detecting cognitive and behavioural impairment in Norwegian patients with ALS. Cut-off scores are situational and could slightly vary between different cultures. The ECAS-N can be used in international research, but researchers should be aware of the differences between the tests applied in the studies.

Implications for rehabilitation

• The ECAS-N, a Norwegian version of the English-language ECAS was produced for clinical use in Norway.

• Rigorous translation methods were used, and expert-judged modifications were made for Norwegian-specific culture and language nuances.

• Determination of ECAS-N cut-off scores for abnormal cognition was guided by a biostatistician, are slightly lower than the original ECAS, and are based on results from an objectively calculated number of healthy Norwegians.

• A nuanced picture of cognitive impairment in ALS drawn from a well-developed scale as the ECAS-N can contribute to a management process better tailored to patients’- and carers’ individual needs.

Keywords:

• ALS
• amyotrophic lateral sclerosis
• ECAS
• Edinburgh cognitive and behavioural ALS screen
• psychometric
• validation

Introduction

Amyotrophic lateral sclerosis (ALS) can present as a pure motor syndrome or coexist with cognitive and/or behavioural impairment resulting from frontotemporal dysfunction [1]. When present, cognitive and behavioural declines modify patient survival rate [2], increase caregiver burden [3], and alter decision-making regarding treatment [4]. Because of these serious consequences, healthcare professionals highly recommend early and repeated screening for cognitive and behavioural changes commonly associated with ALS [1].

Current evidence shows that the prevalence of cognitive changes accompanying ALS is varied, affecting between 30% and 62% of ALS patients [5]. In the last decade, convincing evidence has emerged about prominent changes in ALS patients’ language and executive functions, including social functioning [5–7]. Recent research also suggests that memory dysfunction may play a larger role in ALS than previously thought [5]. Behavioural changes in ALS may involve increased apathy, which is reported in about 30% of patients [8,9]. However, other behavioural changes may also occur, such as showing little concern for personal hygiene, increased irritability, and/or the development of new strange habits [9].

Despite this and other compelling evidence demonstrating that ALS is a complex, manifold disorder that also affects cognition and behaviour, an ALS-specific test for cognitive and behavioural screening is not yet available in the Norwegian language. To date, the cognitive and behavioural status of most ALS patients in Norway remains unknown, meaning a rapid, valid screening test for use in Norway is urgently needed. Recent systematic reviews suggest that the Edinburgh Cognitive and Behavioural ALS Screen (ECAS) [10] may be preferred over other screening tests for effectively, validly and reliably screening cognitive changes in patients with ALS [11,12]. Translation of the ECAS into multiple languages [13–16] will also facilitate international multicentre studies, which in the long run, can provide more detailed, nuanced information about cognitive dysfunction in ALS and better guide care for ALS patients with behavioural and cognitive changes. However, despite the need, translating and adapting psychometric instruments for different language and cultural groups may be viewed as a low priority. Moreover, translation procedures have pitfalls, which can undermine the validity of an instrument. For example, inadequate descriptions of translation methods and failure to comply with internationally recognised methods have been reported [17].

The aims of the present study were as follows: (1) to describe the challenges we encountered in the translation and cultural adaption of the ECAS into Norwegian (ECAS-N); (2) to evaluate the content validity of the ECAS-N; and for Norwegian speakers (3) to provide age- and education-balanced norms for verbal fluency and cut-off scores for identifying abnormal patient results. A properly translated and culturally adapted test can aid clinicians and researchers to more precisely characterise the ALS population and provide the most appropriate interpretation and generalisation of test scores.

Materials and methods

Study design

The design of this study complied with international guidelines for translation, cultural adaption, and evaluation of content validity for psychometric tests [18], as well as recommendations given in the ECAS manual for designing a study planning to acquire data for normative scores (https://era.ed.ac.uk/handle/1842/6592) [19].

Translation and cultural adaption

Approval for this translation project was granted by the ECAS’ copyright holder (Professor Sharon Abrahams, the University of Edinburgh, Edinburgh, United Kingdom). The original ECAS (cognitive screen and behavioural screen), along with its testing guidelines, was translated from English into Norwegian and culturally adapted in five steps [18]. The steps are described in detail in Table 1.

Table 1. Description of steps taken to translate and adapt the original version of the ECAS into Norwegian (ECAS-N)*.

Step No.	Aim	Contributors	Task
1	To produce two forward-translated preliminary versions of the ECAS-N	Two translators whose mother language was Norwegian	Work independently of each other to translate the ECAS into Norwegian.
2	To achieve consensus on one forward translated version of the ECAS-N	Two of the researchers and the two translators involved in step 1	Compare the two preliminary versions of the ECAS-N obtained in step 1 to each other. The aim was to identify similarities and discrepancies between the two preliminary versions of instructions, items, and response format in terms of wording, sentence structure, meaning, and relevance. Any identified ambiguities and discrepancies between the two preliminary versions of the ECAS-N were extracted and discussed.
3	To produce two backwards-translated English versions of the ECAS-N	Two translators whose mother language was English	Work independently of each other and without having knowledge of the original version of the ECAS.
4	To achieve consensus of one backwards-translated version of the ECAS-N and one preliminary version of the ECAS-N	Two of the researchers and all translators involved in steps 1–4	Compare the two backward-translated versions of the ECAS-N obtained in step 3. The aim was to identify similarities to and discrepancies between the two backward-translated versions instructions, items, and response format in terms of wording, sentence structure, meaning, and relevance. Any ambiguities and discrepancies between the two backward-translated English versions of the ECAS-N (from step 3) and the translated ECAS-N (from step 2) were extracted and discussed.
5	To reach an agreement with the copyright holders of the original ECAS that our pre-final, translated Norwegian version is ready for further psychometric testing	Copyright holders of the ECAS	Provide insight on the construct of the pre-final ECAS-N and clarify any questions that might arise.

ECAS: Edinburgh Cognitive and Behavioural ALS Screen; ECAS-N: Edinburgh Cognitive and Behavioural ALS Screen–Norwegian.

*Based on Sousa and Rojjanasrirat [18].

Translation and cultural adaption of the ECAS was done in collaboration with certified healthcare professionals, and professional translators. All translators were fluent in both English and Norwegian. This collaboration produced a Norwegian version of the ECAS containing words and sentences that covered idiomatic Norwegian expressions, both in spoken language and medical terms, with the added sensitivity to Norwegian-specific cultural nuances.

Determination of normative values for the ECAS-N

Age- and education-balanced normative values were calculated for the ECAS-N, including values for the total score, ALS-specific score, non-ALS-specific score and verbal fluency. Normative values for the translated ECAS-N were derived from a population of healthy native Norwegian speakers (i.e., control subjects).

Healthy control subjects

Based on the age range of the target ALS population, healthy individuals between 35 and 85 years old were eligible for inclusion. We excluded individuals who had severe difficulties in writing or reading, cognitive impairment at the time of testing due to a comorbid medical history, neurological disorders or comorbid psychiatric history. The participants declared all their diseases and recorded medications they currently used. Participants who used medication that influenced cognition (e.g., antidepressants) were excluded. Healthy age- and education-matched volunteers were recruited from various workplaces and social organizations, most of which were located close to where the study was conducted. Specifically, we aimed to obtain an equal distribution of controls with a lower and higher level of education across gender and those below and above the age of 60 years. The division of 60 years reflects that the disease usually presents clinically after 60 years [20].

Sample size estimations

In collaboration with a biostatistician, we estimated the sample size of healthy control subjects needed in order to produce a seven-category (from items 5 and 8 of the ECAS) scoring table for verbal fluency having age- and educational-balanced norms for verbal fluency. Assuming a mean sample of 10 subjects per group (age and education levels) for each of the seven categories, this would translate to 280 participants perversion of the ECAS-N (written and oral). Thus, a total sample size of 560 healthy control subjects was needed. For determination of cut-off scores, we adopted the method of Terwee et al. who proposed a sample of at least 50 people in any analysis of measurement properties [21].

Procedure for establishing normative values

For the verbal fluency tasks (item 5 and 8) in the ECAS-N, we followed recommendations given by Abrahams et al. [19]. This meant that the prevalence of words beginning with the letter provided and consisting of the specified amount of letters should have a frequency in the target language similar to that of letter S and 4-letter T words in English [19]. According to the Language Council of Norway, these conditions are met for letters S and 4-letter T words in Norwegian. Norwegian standards are available for spoken verbal fluency for the letter S, but these are standards for a significantly younger age group than those for people with ALS in Norway [22]. To our knowledge, Norwegian standards do not exist for spoken verbal fluency for the letter T nor written verbal fluency.

For normative values for verbal fluency, the healthy subjects were asked to perform two different verbal fluency tasks related to the letters S and T. Half of the healthy subjects were tested in written verbal fluency, while the remaining were tested in spoken verbal fluency; they did this once. To determine the cut-off scores for ECAS-N (ECAS total score, ALS-specific score, non-ALS-specific score), healthy subjects were then tested on the cognitive screen of the ECAS-N.

Statistical analysis

Descriptive statistics were used for summarising demographic data and to establish age- and education-balanced norms for verbal fluency and cut-off scores for the ECAS-N total, ALS-specific and non-ALS-specific scores. Verbal fluency indices (VFI) for the letters S and T were based on the number of words the test takers were able to write or speak within a predefined period, divided by the period of time used to write or speak these words. The method used for converting these values for use in score tables followed the translation guidelines provided by the copyright holders of the ECAS (https://ecas.psy.ed.ac.uk/ecas-international) [19]. Cut-off scores were defined as 2 standard deviations (SD) below the mean scores of the healthy control subjects. This is the parametric procedure recommended by Abrahams et al. [10].

Content validity evaluation of the pre-final version of the ECAS-N

To ensure that the ALS patient participants and their carers could easily understand the administration instructions, test items, and response format of the ECAS-N, we pilot tested the pre-final version of the ECAS-N. Involved in the pilot testing were also, a group of experts with thorough knowledge in cognitive impairment and neurological diseases like ALS. Content validity [23] was evaluated using a rating form for relevance and comprehensiveness and a different rating form for comprehensibility, both developed by the authors for this purpose.

Subjects with ALS and carers

Native-Norwegian-speaking patients with ALS were eligible for inclusion if they had received their positive diagnosis within 4 months of test administration. In addition, one carer chosen by the patient was included to participate in evaluating the behavioural parts of the ECAS-N. As with the healthy control subjects, we excluded ALS patients who had great difficulties in writing or reading, or who had a co-morbid medical history, neurological disorders, or a co-morbid psychiatric history. Patients and their carers were recruited in conjunction with their first consultation at an ALS-specific outpatient clinic.

Expert persons

Skilled resource staff, or professionals with expertise in different aspects of ALS, were recruited to generally oversee the translation in a valid and efficient way. We carefully selected these resource persons who were experts in the field of ALS research, ALS patient care or were experts in fields focusing on cognitive impairment in neurological diseases. Their duties were to facilitate the research experience; to provide relevant perspectives of those working in different healthcare disciplines; and to share their linguistic skills, knowledge of ALS, and experience to assist participants with ALS. A minimum of 10 patients, 10 carers, and 6 resource persons from the field of interest was necessary to evaluate content validity [18].

Rating scale for relevance and comprehensiveness

For assessing the relevance of the EACS-N, each item was evaluated using a categorical scale having four possible rating options: 1 = not relevant; 2 = unable to assess relevance; 3 = relevant but needs minor alteration; 4 = very relevant and succinct. A cut-off score of ≤2 for an item was defined as being irrelevant. Comprehensiveness was evaluated by asking the raters to list any shortcomings of the ECAS-N. This was a subjective measure. Ratings for all the ECAS-N items were collected on rating forms, later to be used in statistical analyses.

Rating scale for comprehensibility

For assessing the comprehensibility of the ECAS-N, the clarity of each item was subjectively evaluated using a binary scale (clear or unclear). Concrete suggestions for linguistic improvements were requested by the raters. Ratings for all the ECAS-N items were collected on rating forms, later to be used in statistical analyses.

Content validity assessment

The experts were asked to rate the clarity of the wording of the pre-final ECAS-N scales (cognitive and behavioural) and administration guidelines. To further evaluate the content validity of the pre-final version of the ECAS-N, these experts also assessed whether each item, response options and instructions for the cognitive screen, behavioural screen and guidelines were relevant for Norwegian patients with ALS, carers and test administrators. As part of the content validity assessment, the experts were asked to identify possible omissions (questions, items, instructions, etc.). For items judged unclear or irrelevant, they were asked to suggest improvements or modifications. This step was done to determine whether the items adequately measured all relevant dimensions of cognitive function. Problems identified during this evaluation process were discussed within the research group with the goal of reaching a consensus. When the agreement was reached among experts, their recommendations were used to guide decisions about retaining, revising, deleting or adding items. Amendments were reconsidered in relation to accepted guidelines for ECAS translation [19], and any ambiguities were discussed with the original authors of the ECAS before amendments were finalised.

Patients with ALS and carers were also asked to rate the clarity of each item of the pre-final versions of the ECAS-N cognitive screen and the ECAS-N behavioural screen, respectively. Both patients and carers were asked immediately after being tested (or interviewed in the case of carers) for their general input and were encouraged to suggest areas of item improvement.

Statistical analysis

In consultation with a biostatistician, we analysed data using SPSS (IBM Corp. Released 2019. IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY: IBM Corp). For the determination of ECAS-N relevance, we calculated the content validity index at the item level (termed I-CVI) and at the scale level (termed S-CVI). We calculated the I-CVI by summing the number of experts giving a rating of either 3 or 4 on the 4-point rating form for relevance and divided that number by the total number of experts. For calculating the I-CVI, we followed the advice of Lynn [24]. Since we had six experts performing this evaluation, the I-CVI is recommended to be no less than 0.83 [24]. Practically, this means that one item rated as “not relevant” will still mean that the ECAS-N would be determined to be content valid [24]. The average method was used to calculate the S-CVI [25]. The number of items rated relevant across experts (average I-CVA) divided by the number of experts constituted the S-CVI. A value of 0.90 or above is recommended for the S-CVI [25], which follows the standard criterion for acceptability for the S-CVI/avg. This standard translates to 100% agreement for at least 90% of the items. For comprehensibility, we accepted 80% agreement among the raters.

Ethical approval and informed consent

This study adheres to the Declaration of Helsinki [26] for experiments conducted with human subjects. All included patients, carers and healthy subjects signed voluntary informed consent. The study was considered by the Regional Ethical Research Committee (reference number 2015/1221) and approved by the Data Privacy Unit (reference numbers 2015/11598 and 2016/3166). All subjects’ data were de-identified.

Results

Translation and cultural adaption of the ECAS into Norwegian

The original English version of the ECAS [10] was translated into Norwegian and culturally adapted. We call this translated and culturally adapted version the ECAS-N. The ECAS-N can be found in the Supplemental material.

Normative data for verbal fluency and cut-off scores for abnormal performance

Table 2 presents demographic characteristics of the 559 healthy subjects who contributed data to determine normative values for the letters S and letter T verbal fluency test. The ages of these subjects were fairly well-balanced between subjects younger than 60 years and those 60 years or older, and those who attained a high school or elementary school level of education and those a university level.

Table 2. Selected characteristics of healthy subjects (N = 559) who contributed data for calculation of normative values for verbal fluency.

Characteristics	n (%)
Sex (male)	256 (46)
Age
<60 years	298 (53)
≥60 years	261 (47)
Highest educational level attained
High school/elementary school	266 (48)
University/college	293 (52)

For the letter S data, the VFI conversion tables were derived from 279 of the 559 healthy subjects, who provided spoken responses (mean no. “S” words produced = 4.05, SD = 2.33), and the remaining 280 subjects provided written responses (mean no. “S” words produced = 4.13, SD = 3.09). The table to convert VFI to a fluency score for the letter S is presented in the ECAS-N, item 5 that is Supplemental material.

For letter T data, VFI conversion tables were derived from 556 healthy subjects, 277 of whom provided spoken responses (mean no. “T” words produced = 11.50, SD = 10.56), and the remaining 279 subjects provided written responses (mean no. “T” words produced = 13.46, SD = 13.80). The table to convert VFI to a fluency score for the letter T is presented in the ECAS-N, item 8 that is Supplemental material.

Table 3 presents demographic characteristics of the healthy subjects who contributed ECAS-N data for the production of cut-off scores for abnormal performance. These cut-off scores were derived from 85 healthy subjects who completed all tasks of the ECAS-N (ALS-specific score: mean = 82.41, SD = 8.58; ALS non-specific score: mean = 30.42, SD = 3.42; ECAS-N total score: mean = 112.84, SD = 10.40).

Table 3. Selected characteristics of healthy subjects (N = 85) who contributed data for calculation of ECAS-N cut-off scores.

Characteristics	n (%)
Sex (male)	44 (52)
Age
<60 years	42 (49)
≥60 years	43 (51)
Highest educational level attained
High school/elementary school	42 (49)
University/college	43 (51)

ECAS-N: Edinburgh Cognitive and Behavioural ALS Screen–Norwegian.

Evaluation of content validity

Three aspects of content validity were evaluated: relevance, comprehensiveness and comprehensibility. Experts evaluated relevance (n = 6), comprehensiveness (n = 8) and comprehensibility (n = 8). Comprehensibility was also evaluated by patients with ALS (n = 13) and their carers (n = 12). The group of experts consisted of three physicians, one occupational therapist, one neuropsychologist, one registered nurse, one speech therapist and one patient research partner. All but one of the experts were native speakers of Norwegian, and the other was a fluent speaker.

Relevance and comprehensiveness

The experts rated each item of the pre-final version of the ECAS-N (cognitive screen, behavioural screen and guidelines) as being relevant for patients with ALS, carers and test administrators. The I-CVI was equal to 1 (range: 1–1) for all items of the ECAS-N, cognitive scale, behavioural scale and guidelines. The S-CVI for the total ECAS-N cognitive scale, ECAS-N behavioural scale, and guidelines were all 1 (range 1–1). No key concepts were noted as “missing” by the experts.

Comprehensibility

Comprehensibility was assessed in three rounds. To adapt the ECAS to the Norwegian language and culture, we found it necessary to revise items 3, 4, 12, and 15. In specific, we used Norwegian names in the story told in items 3 and 15. Regarding item 12, toast and barbers are not part of the Norwegian culture. Therefore, we changed the questions to be about porridge, which is commonly eaten in Norway, and hairdressers. Additionally, we carefully selected the words in item 4 to meet the requirements stated in the guidelines for ECAS translation: Items should include, in comparable numbers, nouns, verbs and compound words. Compound words should consist of noun-verb combinations. [19]. Consequently, we changed biscuits to oranges, skateboard to board games, and brought to followed. Further, the guidelines were elaborates on how to score patients’ sentence completion (item 12) and how to interpret compound words given by the patients (item 5 and 8). No items were deleted. After three rounds of review among experts, comprehensibility (clear or unclear) rating reached an inter-rater agreement of 87.5% on the revised pre-final version of the ECAS-N. Evaluation of comprehensibility of the ECAS-N by experts is presented in Tables 4 and 5 for the cognitive and behavioural screens (Table 4) and guidelines (Table 5).

Table 4. Comprehensibility ratings of the ECAS-N Cognitive and Behavioural screens*. [AQ

Item	Rating of comprehensibility†
	Round 1* n = 8	Round 2* n = 8	Round 3* n = 8
ECAS – cognitive screen
1 – Language – naming
Instructions, n
Response options, n
2 – Language – comprehension
Instructions, n	1
Response options, n	2	1
3 – Memory – immediate recall
Instructions, n
History, n	1
4 – Language – spelling
Instructions, n
Response options, n	4	1
5 – Fluency – letter S
Instructions, n
Table and calculation, n
6 – Executive – reverse digit span
Instructions, n
7 – Executive – alternation
Instructions, n	2	2
8 – Fluency – letter T
Instructions, n
Table and calculation, n
9 – Visuospatial – dot counting
Instructions, n
Response options, n
10 – Visuospatial – cube counting
Instructions, n	1
Response option, n
11 – Visuospatial – number location
Instructions, n	1	1
Response options, n
12 – Executive – sentence completion
Instructions, n	1	2
Response options, n
13 – Social cognition – part A
Instructions, n	1
Response options, n
14 – Social cognition – part B
Instructions, n	2	1
Response options, n
15 – Memory – delayed recall
Instructions, n	1
Table and calculation, n
16 – Memory – delayed recognition
Instructions, n	2	1
Response options, n	1
Table
Scoring table	3	1
ECAS – behavioural screen
Instructions, n	1
Response options, n
A – Behavioural disinhibition
Question, n	1
Examples, n	1
B – Apathy or inertia
Question, n	1
Examples, n
C – Loss of sympathy or empathy
Question, n
Examples, n	1	1
D – Perseverative, stereotyped, compulsive or ritualistic behaviour
Question, n
Examples, n	1	2	1
E – Hypo-orality and altered food preferences
Question, n	1
Examples, n	1
Psychosis screen
Instructions, n	1
Response option, n
Questions	1
Onset and duration of symptoms
Instructions, n
Response option, n	1
Questions, n

ECAS-N: Edinburgh Cognitive and Behavioural ALS Screen–Norwegian.

*The comprehensibility of the ECAS-N was rated by experts in three rounds. They rated the items as clear or unclear.

†Numbers represent the number of experts that rated the issue as unclear. Ratings were coded by colour as follows: ▪ Clear by all experts; no changes were required. ▪Unclear by ≥1 expert; they requested more detailed information about the purpose of the test, but no changes were required. ▪Unclear by ≥1 expert; changes were required.

Table 5. Comprehensibility of the ECAS-N and its guideline instructions*.

	Comprehensibility (clear/unclear)†
Item	Round 1* (n = 8)	Round 2* (n = 8)	Round 3* (n = 8)
Introduction	1	3
ECAS – cognitive screen
1 – Language – naming
Instructions	3	1
Scoring
2 – Language – comprehension
Instructions	1	1
Scoring
3 – Memory – immediate recall
Instructions	1	1
Scoring	2
4 – Language – spelling
Instructions	3	2	1
Scoring
5 – Fluency – letter S
Instructions	1
Scoring
6 – Executive – reverse digit span
Instructions	1	1
Scoring
7 – Executive – alternation
Instructions	2	3
Scoring	1
8 – Fluency – letter T
Instructions	1
Scoring
9 – Visuospatial – dot counting
Instructions	1	1
Scoring
10 – Visuospatial – cube counting
Instructions	1	1
Scoring
11 – Visuospatial – number location
Instructions	1	1
Scoring
12 – Executive – sentence completion
Instructions	2	3	1
Scoring	3	1	1
13 – Social cognition – part A
Instructions	1	1
Scoring
14 – Social cognition – part B
Instructions	1	1
Scoring	2
15 – Memory – delayed recall
Instructions	2	1
Scoring
16 – Memory – delayed recognition
Instructions	1	1
Scoring	3
ECAS – behavioural screen
Introduction	2
Instructions

ECAS-N: Edinburgh Cognitive and Behavioural ALS Screen–Norwegian.

*The comprehensibility of the ECAS-N was rated by experts in three rounds. They rated the items as clear or unclear.

†Numbers shown represent the number of experts that rated the issue as unclear. Ratings were coded by colour as follows: ▪ Clear by all experts; no changes were required. ▪Unclear by ≥1 expert; they requested more detailed information about the purpose of the test, but no changes were required. ▪Unclear by ≥1 expert; changes were required.

Ninety-two percent of the patients with ALS and 92% of the carers rated as clear the test’s instruction, response format/options, and items of the revised pre-final version of the ECAS-N. We interpret this to mean that the pre-final version of the ECAS-N was sufficiently well understood by the target population that it needed no further revision or modification.

Discussion

The work described in this study led to a translated version of the original ECAS into Norwegian (ECAS-N) that is adapted to a Norwegian cultural context, was evaluated for its content validity, and provides a country-balanced-norms of performance. The ECAS-N adds to the other language versions of the ECAS [13–16], which will facilitate international, multicentre studies of ALS. The challenge for translation of cognitive tests is that the tests cannot be just translated, but must be also culturally adapted. The adaption ensures that the cognitive performance will be tested correctly.

Several aspects of this translation effort are worth discussing. We recognised that simply translating the original ECAS in a literal way was insufficient to produce a valid screening tool for Norwegian-speaking patients with ALS. Rather, to produce a translation that is understood by native-speaking Norwegians, it was necessary to adjust its items and interview component to be in line with the nuances of the Norwegian language and culture. In this study, the conceptual, semantic, and content equivalency of the ECAS-N were rated as “good” by the target population (i.e., ALS patients and their carers) and experts in the field of ALS. We feel confident that the ECAS-N can be used routinely in clinical practice in Norway. Results of the present study emphasise that each screening test needs to have language-specific cut-off scores to confidently identify abnormal performance. Assuming that age and education level would result in important differences in cognitive performance, we calculated age- and education-balanced cut-off scores for the ECAS-N based on normative data from healthy persons age- and education-matched to the ALS population of patients in Norway.

Translation, adaption and evaluation of content validity

The ECAS was first developed in the United Kingdom (UK), and published in the English language [10]. Because it is based partly on common knowledge portrayed in pictures, patients who are assessed with the ECAS must be familiar with the objects portrayed in those pictures. Thus, being sensitive to cultural- and language-specific nuances is imperative in translation work [27].

The ECAS also provides norms for verbal fluency and cut-off scores for normal performance, but these norms are based on data from an English speaking population. Although the UK shares some cultural similarities with Norway, a direct word-for-word translation of the ECAS into Norwegian would fail to reflect nuances unique to native Norwegians, distinctions or the lack thereof that could potentially affect test-taker performance and lead to inaccurate or biased results. Although time-consuming, it is critical that researchers ensure that the psychometric instrument they are using to assess a given population has been cross-culturally adapted for that population [18]. This will ensure its validity for that particular population [18]. Also, as the consequences of using norms derived from people speaking other languages are not fully understood, country-specific norms need to be constructed [22]. Thus, we carefully considered these issues in our translation and adaptation.

Evaluation of content validity of the ECAS-N is also imperative to do in order to have a valid screening tool for detecting cognitive and behavioural declines. A clear structure and clear sentences used in the instructions, items and response format, ensure that the ECAS-N is easily understood by the patients with ALS and the carers and contribute to most equal completion and scoring among test administrators. Compared to the original ECAS, the ECAS-N offers clearer administrator instructions in the use of self-corrections and elaborates on how to score patients’ sentence completion (item 12). Additionally, the ECAS-N provides instructions on how to interpret compound words given by the patients (items 5 and 8). Compound words are common in Germanic languages, such as Norwegian, in which long words are constructed through linguistic compounding. Although compound words are also found in English, the original ECAS does not have clear instructions on how to interpret them, presumably because they are less common.

The present study was carried out rigorously in cooperation with the copyright holders of the original ECAS. Thus, we achieved a good translation that reflects the developers’ intention for the ECAS. However, future efforts to develop other translations should be aware that a rigorous methodological approach is a considerable investment of time and money, requiring careful planning and persistence [17,18]. Cultural adaption and changes are important to users but do not influence the essence of the test.

Sensitivity and specificity of cut-off scores

In the present study, the cut-off scores for the ECAS-N were lower (ALS-specific score and ECAS-N total score) than or equal (ALS non-specific score) to those reported for the original ECAS [10]. Cut-off values for the German-Swiss and Italian versions of the ECAS also differ from those of the original ECAS [28,29]. These discrepancies could be due to unequal composition of normative groups, use of different statistic procedures to estimate cut-off values, or simply country-specific differences in the characteristics of healthy people on which the norms are based. Country-specific cut-off scores are obviously needed and should be established before introducing a new translated version of the ECAS. We did this with the ECAS-N.

As some of the ECAS-N cut-off scores were lower than that of the original ECAS, using these scores may lead to some false-negative classifications, which can lower the sensitivity of the ECAS-N. Therefore, clinicians using the ECAS-N must scrutinise closely high-functioning individuals in which mild cognitive impairment might be masked by compensatory strategies.

Our study supports previously reported findings that advocate repeated longitudinal testing with the ECAS to help identify false-negative cases, as patients with ALS may continue to display more overt changes over time [30]. The most frequent of these changes are ALS-specific cognitive and behavioural declines [30]. Such persistent clinical attention may present opportunities for timely access to treatment and for making critical decisions before more debilitating impairment emerges [4].

A false negative or a false positive, classification may have a negative impact on patients' and carers` confidence about the utility of screening [31]. Therefore, both patients and carers should be fully informed about possible costs (i.e., limitations) and benefits before screening is done. Additionally, the ECAS-scores should always be interpreted in light of the information provided by people who know the patient well, in addition to other quality assessments conducted by trained observers.

In order to provide quality patient care and conduct cross-cultural research, clinicians and researchers involved in cognitive- and behavioural screening in patients with ALS should have access to reliable and validated tests in their native language and be culturally relevant. However, sensitivity is one of the other quality indicators of a good screening instrument [32,p.467]. Further psychometric testing of the ECAS-N among patients with ALS is important to produce a solidly reliable and valid ECAS in Norwegian.

Study strengths and limitations

The strengths of the current study lay primarily in the rigor in carrying out the translation and in the quality assessment achieved by complying with the standardised and exacting evaluation approach. This adopted approach was developed especially for translation, adaption and validation in healthcare research [18]. Another strength is that the expertise of a biostatistician guided the sample size calculation and analysis. The skilled judgement and experience of resource persons with expertise in neuropsychology and with patients with ALS strengthened the adjustments made in the ECAS-N item relevancy and clarity. Bias may be limited, as none of the experts was involved in the translation process, and they were all asked whether the translators failed to translate any items.

There are numerous methods for establishing cut-off values for health measurement scales [33]. Our choice to define cut-offs as values that are 2 SDs below the mean of healthy subjects is a traditional criterion, which is also in accord with the recommendations of the original version of the ECAS [10]. Calculation of a receiver operating characteristic curve is an alternative approach to deriving cut-off values [33]. Another limitation of our study is that cut-off scores were derived from a rather small sample of healthy subjects (n = 85), mainly recruited from a limited geographical area in Norway. A larger sample size might produce different means. Our sample for calculating cut-off scores was selected based on quota sampling procedures. This sampling method seemed reasonable because the proportion of samples for each category had about the same proportion that the ALS population had. We ensured that the sample would match a typical ALS population with slightly more men than women and would match well by age and education level, which we reasoned would capture important differences in cognitive and behavioural performance. A random sampling procedure may have sharpened the precision and representativeness of the final sample even more.

Acknowledgements

We thank Professor Abrahams and Professor Bak for their contribution in providing the translated Norwegian version of the ECAS. Moreover, the authors thank all the participants who gave their time for this study. We also thank the Department of Occupational Therapy and the Department of Neurology at Haukeland University Hospital for their valuable support throughout the process. Thanks also to colleagues for their dedicated work in translating the ECAS into Norwegian and testing, as well as those experts who contributed to the evaluation of content validity.

Disclosure statement

The authors report no potential competing interest with respect to the research, authorship, and/or publication of this article.

Additional information

Funding

The Western Regional Norwegian Health Authority supported this work

(grant number: 912158, 2017). Initiation grants were from the Norwegian Association of Occupational Therapy, the foundation ALS Norwegian support group, Department of Occupational Therapy and Department of Neurology, Haukeland University Hospital and National Advisory Unit on long-term mechanical ventilation.