Derivation of norms for the Dutch version of the Edinburgh cognitive and behavioral ALS screen

Abstract

Background: The Edinburgh cognitive and behavioral ALS screen (ECAS) was developed specifically to detect cognitive and behavioral changes in patients with amyotrophic lateral sclerosis (ALS). Differences with regard to normative data of different (language) versions of neuropsychological tests such as the ECAS exist. Objective: To derive norms for the Dutch version of the ECAS. Methods: Normative data were derived from a large sample of 690 control subjects and cognitive profiles were compared between a matched sample of 428 patients with ALS and 428 control subjects. Results: Age, level of education, and sex were significantly associated with performance on the ECAS in the normative sample. ECAS data were not normally distributed and therefore normative data were expressed as percentile ranks. The comparison of ECAS scores between patients and control subjects demonstrated that patients obtained significantly lower scores for language, executive function, verbal fluency, and memory, which is in line with the established cognitive profile of ALS. Conclusion: For an accurate interpretation of ECAS results, it is important to derive normative data in large samples with nonparametric methods. The present normative data provide healthcare professionals with an accurate estimate of how common or uncommon patients’ ECAS scores are and provide a useful supplement to existing cut-off scores.

Keywords:

• Amyotrophic lateral sclerosis
• Edinburgh cognitive and behavioral ALS screen
• cognitive dysfunction
• normative data

View correction statement:

Correction

Introduction

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative syndrome characterized by the progressive loss of motor neurons. Over the last decade, it has become increasingly clear that ALS has overlap with frontotemporal dementia (FTD) and many patients present with features of both diseases (1). Screening for cognitive and behavioral changes in patients with ALS has therefore become standard practice.

Considering that full neuropsychological evaluations (NPEs) are time consuming and unpractical in a multidisciplinary clinic, several brief neuropsychological screening instruments have been developed specifically for ALS (2–5). The Edinburgh cognitive and behavioral ALS screen (ECAS) is perhaps one of the most commonly used instruments and has been extensively validated and translated into multiple languages (4,6–13). Considering that there are differences with regard to the normative data of different language versions, validation of these versions is necessary. Therefore, the objective of this study was to derive normative data for the Dutch version of the ECAS.

Methods

Translation

The original English version of the ECAS was translated and subsequently back-translated following cross-cultural adaptation guidelines and ECAS guidelines (4). The Dutch version of the test is available as a Supplementary material.

Participants and procedures

This study was conducted in two parts. Participants for both parts of this study were recruited from a large ongoing national population-based prospective epidemiological study on ALS in The Netherlands (14). For the first part of the study, normative data for the Dutch version of the ECAS were derived in a cohort of 690 control subjects. In the second part, the ECAS scores were compared in a sample of 428 patients with ALS and 428 matched controls to evaluate the cognitive profile on the ECAS. Patients with ALS and control subjects were matched on age, sex, and level of education.

Patients had to fulfill the following inclusion criteria: (1) Dutch as first language and (2) the absence of preexisting conditions that could influence test performance; dyslexia, learning disabilities, substance abuse, psychiatric disorders, other neuromuscular diseases, cerebrovascular disease, epilepsy, neurodegenerative diseases, traumatic brain injury, and/or the use of psychoactive medication. Patients with ALS were diagnosed with possible, probable laboratory supported, probable or definite ALS according to the revised El Escorial criteria (15). Disease severity was assessed using the amyotrophic lateral sclerosis functional rating scale-revised (ALSFRS-R) (16). Additionally, the following information was collected for all subjects: age, sex, and level of education according to the International Standard Classification of Education (ISCED 1997). Baseline characteristics are provided in Table 1.

Table 1 Characteristics of the normative sample and the matched sample.

	Normative sample	Matched sample
	Controls (n = 690)	Patients (n = 428)	Controls (n = 428)
Age, m (sd)	63.3 (11.4)	63.7 (10.8)	63.7 (10.5)
Sex, n (%)
Female	279 (40.4)	164 (38.3)	164 (38.3)
Male	411 (59.6)	264 (61.7)	264 (61.7)
Level of education, n (%)
ISCED 0–4	427 (61.9)	304 (72.2)	304 (72.2)
ISCED 5–6	263 (38.1)	118 (27.8)	118 (27.8)
Age at disease onset, m (sd)		61.5 (11.4)
Age at diagnosis, m (sd)		62.8 (11.3)
Site of disease onset
Spinal		309 (72.2)
Bulbar		114 (26.6)
Missing		5 (1.2)
ALSFRS-R* m (sd)		37.6 (7.1)

* Raw total score

ALSFRS-R: amyotrophic lateral sclerosis functional rating scale-revised; ISCED: international standard classification of education; m: mean; n: sample size; sd: standard deviation.

Assessment and materials

The ECAS was developed specifically for administration by any healthcare professional. In the present study, the ECAS was administered to patients and control subjects by trained (research) nurses, neurology residents, neurologists, neuropsychologists, and neuropsychology trainees (4). Several participants completed more than one assessment with the ECAS. In both parts of the study, i.e. the derivation of normative data and the comparison of cognitive profiles between patients with ALS and controls, the first assessment with the ECAS was used.

ECAS

The ECAS is a brief multi-domain neuropsychological screening instrument designed specifically for the assessment of cognition, behavior, and the presence of psychotic symptoms in patients with ALS. The ECAS cognitive screen comprises two sections; an ALS specific section which assesses cognitive domains generally considered to be affected in ALS, i.e. language, verbal fluency, and executive function, and an ALS non-specific section which assesses cognitive domains which were considered to be preserved in ALS, i.e. memory and visuospatial abilities.

To accommodate for speech disabilities in the assessment of verbal fluency, comprising time-dependent tasks, verbal fluency indices (VFIs) (17) were calculated for both spoken and written verbal fluency. Following ECAS Administration and Guidelines Notes (4) conversion tables for the VFIs were constructed from the performance of the control subjects. Twenty eight control subjects were assessed with a written version of the verbal fluency task.

To avoid learning effects, various versions of the ECAS have been developed (18). In the present study, norms are derived for version A. Furthermore, in all sections of the present study, only version A was used.

Statistical analyses

Derivation of normative data

For the derivation of normative data, four methods were explored. First, normative data were derived using means and standard deviations for the sample stratified on age and level of education. Second, normative data were derived for the unstratified sample using multiple linear regressions with age, level of education, and sex as potential covariates in the model. For both methods using z-scores, abnormality was defined as z ≤ –2.000 and z ≤ –1.645, classifying the lowest 2.275% and 5.000%, respectively. Normality of scores was assessed with quantile–quantile (Q–Q) and density plots. Box–Cox procedures (19) were employed to estimate the optimal normalizing power transformation. Third, in case of non-normally distributed scores, normative data were derived using percentile rank scores (20,21) in a stratified sample. Finally, normative data were derived using the percentile rank method on the residuals of the multiple linear regressions, with age, level of education, and sex as potential covariates in the model. In contrast to z-scores, percentiles directly express how common or uncommon a patient's test score is in the normative population. Again, abnormality was defined as the lowest 2.275% and 5.000%.

Methods for the derivation of norms, their advantages and disadvantages, are explained in Supplementary material 1.

Case-control study of global cognitive functions using ECAS

For the comparison of global cognitive functions, the comparison of the ECAS total score, ECAS domain scores, and ECAS subdomain scores, patients and controls were matched on age (continuous), sex, and level of education, using a propensity score, a score that summarizes confounder information, and a nearest neighbor matching algorithm. Differences were tested with Wilcoxon rank sum tests. All statistical tests were corrected for multiple testing using the method proposed by Benjamini and Hochberg (22).

To assess whether the pattern of deficits in our sample of patients with ALS is similar to those reported in the literature, the frequency and percentage of patients with a score corresponding to a percentile equal or below 2.275 and 5.000 were reported using both z-scores and percentiles.

All statistical analyses were performed with R (23).

Ethical considerations

The Medical Ethics Committee of the University Medical Center Utrecht confirmed that the Medical Research Involving Human Subjects Act did not apply and granted a waiver. Informed consent was obtained from all participants.

Results

Derivation of normative data for the ECAS

For the derivation of normative data, it is pivotal that relevant characteristics of the control subjects match those of the patient sample. Normative data for the ECAS were derived from a large sample of control subjects whose characteristics approximately match those of the patient population. To assess whether differences in demographic characteristics between the control population and the patient population resulted in differences in norms, the analyses below were also performed in a sample of controls that was matched to the patient sample. However, no substantial differences were found (data not shown).

First, normative data for ECAS were derived in a stratified normative sample under the assumption of a normal distribution of scores using z-scores. Following Abrahams et al. (4), a conversion table for the VFIs (Table 2) and a stratified norm table were constructed with means, standard deviations, and cut-offs calculated by subtracting two standard deviations below the mean and 1.645 standard deviations below the mean, respectively (Table 3). Furthermore, to correct for non-normally distributed VFIs, these tables were constructed after log transformation of the VFIs (see Supplementary material 5).

Table 2 Normative data for free and fixed and spoken and written verbal fluency indices.

VFI conversion to score – letter free (T)			VFI conversion to score – letter fixed (N)
Spoken VFI	Written VFI	Score	Spoken VFI	Written VFI	Score
≥24.80	≥20.90	0	≥80.15	≥96.06	0
20.43–24.80	17.34–20.90	2	65.06–80.15	78.52–96.06	2
16.05–20.43	13.78–17.34	4	49.98–65.06	60.99–78.52	4
11.67–16.05	10.23–13.78	6	34.89–49.98	43.46–60.99	6
7.29–11.67	6.67–10.23	8	19.81–34.89	25.93–43.46	8
2.91–7.29	3.11–6.67	10	4.72–19.81	8.39–25.93	10
<2.91	<3.11	12	<4.72	<8.39	12

VFI: verbal fluency index.

Table 3 Normative data on the ECAS adjusted for age and level of education.

Age	<65								≥65
Level of education	ISCED 0–4				ISCED 5–6				ISCED 0–4				ISCED 5–6
	n = 198				n = 141				n = 229				n = 122
	m	sd	Cut-off		m	sd	Cut-off		m	sd	Cut-off		m	sd	Cut-off
			2.3%	5.0%			2.3%	5.0%			2.3%	5.0%			2.3%	5.0%
ECAS	112.08	(9.16)	94	97	118.69	(7.01)	105	107	106.95	(10.67)	86	89	114.94	(8.18)	99	102
ECAS ALS specific	82.78	(7.38)	68	71	87.45	(5.49)	76	78	78.61	(8.60)	61	64	85.20	(5.98)	73	75
Language	26.34	(1.68)	23	24	27.03	(1.28)	24	25	25.79	(1.84)	22	23	26.89	(1.23)	24	25
Fluency^a	19.62	(1.97)	16	16	20.58	(1.25)	18	19	19.18	(2.24)	15	16	19.97	(1.71)	17	17
Executive	36.82	(6.02)	25	27	39.84	(4.60)	31	32	33.64	(6.93)	20	22	38.34	(4.97)	28	30
ECAS ALS non-specific	29.30	(3.73)	22	23	31.23	(2.92)	25	26	28.34	(3.90)	21	22	29.77	(3.90)	22	23
Memory	17.63	(3.55)	11	12	19.38	(2.84)	14	15	16.87	(3.65)	10	11	18.02	(3.86)	10	12
Visuospatial	11.67	(0.79)	10	10	11.85	(0.38)	11	11	11.46	(1.05)	9	10	11.75	(0.52)	11	11

ALS: amyotrophic lateral sclerosis; ECAS: Edinburgh cognitive and behavioral screen; ISCED: International standard classification of education; m: mean; n: sample size; sd: standard deviation.

^aBased on traditional method for verbal fluency index conversion.

Second, a multiple linear regression-based approach for estimating z-scores was used. Several regression equations for the derivation of normative ECAS data were explored. The final regression equations are presented in Supplementary material 3. Inspection of density plots and Q–Q plots demonstrated that ECAS data were not normally distributed. Box–Cox procedures for finding the optimal normalizing power transformation confirmed that the data were not normally distributed (see Supplementary materials 2 and 3). Scores near the ceiling of the ECAS, its domains, and subdomains generally predominate, indicating that most tests are within the competence of most participants, resulting in non-normal distributions.

Given the skew of the normative data, use of z-scores is not useful when interpreting an individual’s score. Figure 1 demonstrates that the parametric percentiles, i.e. percentiles derived from the normal curve, classify patients quite differently than the nonparametric percentiles, i.e. the percentile rank scores.

Figure 1 Metrics of scores with and without the assumption of normality.

Third, normative data for ECAS were derived in the stratified normative sample using a percentile rank method. Stratified norm tables were constructed and for conversion of raw scores on each of the ECAS scales to percentiles (Supplementary material 4).

Finally, normative data for ECAS were derived using a percentile rank method on the residuals of the multiple linear regressions and a norm table was constructed for conversion of residual scores to percentiles (Table 4 and Supplementary material 5). These tables may be used for estimating cognitive deficits in patients with ALS.

Table 4 Normative data for the ECAS scores expressed as residual scores.

%	ECAS	ALS specific	ALS non-specific	Language	Fluency	Executive	Memory	Visuospatial
1	–31.77	–23.02	–12.24	–5.85	–8.02	–17.70	–12.58	–3.67
2	–26.29	–17.61	–10.06	–4.89	–7.06	–14.97	–9.93	–2.68
5	–16.44	–13.61	–7.00	–2.87	–3.30	–10.51	–6.48	–1.50
10	–11.17	–8.78	–4.71	–2.00	–1.50	–7.84	–4.54	–0.81
15	–8.07	–7.08	–3.53	–1.18	–1.24	–6.53	–3.12	–0.66
20	–6.40	–5.89	–2.46	–0.98	–0.75	–5.05	–2.35	–0.46
25	–4.62	–4.24	–1.71	–0.81	–0.41	–3.21	–1.62	0.05
30	–3.38	–2.81	–1.10	–0.27	–0.31	–2.17	–1.12	0.08
35	–2.04	–1.51	–0.71	–0.07	–0.20	–1.24	–0.63	0.13
40	–1.01	–0.48	–0.25	0.03	–0.08	–0.24	–0.19	0.19
45	0.25	0.25	0.31	0.10	0.06	0.46	0.22	0.24
50	1.27	0.91	0.70	0.17	0.33	1.25	0.68	0.25
55	2.16	1.82	1.08	0.31	0.45	1.73	1.16	0.27
60	2.97	2.63	1.46	0.75	0.55	2.26	1.38	0.30
65	3.91	3.32	1.79	0.88	0.64	2.74	1.71	0.32
70	4.91	4.22	2.10	0.96	0.70	3.53	1.97	0.35
75	6.05	5.16	2.41	1.04	0.79	4.07	2.37	0.39
80	7.33	5.92	2.86	1.12	0.87	4.87	2.62	0.43
85	8.30	6.91	3.21	1.20	1.01	5.58	3.11	0.48
90	10.14	7.98	3.77	1.69	1.79	6.45	3.59	0.54
95	12.12	9.49	4.54	1.92	2.59	7.65	4.31	0.63

ALS: amyotrophic lateral sclerosis; ECAS: Edinburgh cognitive and behavioral ALS screen.

Residual scores = Observed ECAS scores – Expected ECAS scores.

Expected scores can be calculated using the equations of the multiple linear regressions.

Comparison of ECAS scores between patients with ALS and controls

Patients with ALS and control subjects were matched using propensity score to obtain equivalent groups with regard to age, level of education, and sex. Demographic characteristics of the matched sample are provided in Table 1.

The comparison of ECAS scores, domain scores, and subdomain scores between patients with ALS and controls is presented in Table 5. Patients with ALS scored statistically significant lower on the ECAS total score, ECAS ALS specific score, and its subdomains, i.e. language, verbal fluency, and executive function. Furthermore, patients with ALS demonstrated statistically significant lower scores on the ALS nonspecific domain and the memory subdomain. These findings are in line with the results from the most recent meta-analysis of the cognitive profile of ALS (24).

Table 5 Comparison of ECAS scores between matched patients with ALS and controls.

Score	Patients				Controls				p
	m	sd	mdn	iqr	m	sd	mdn	iqr
ECAS	106.98	14.00	110.00	100.00–117.00	111.49	10.45	110.00	105.00–119.00	<0.001
ALS specific	79.53	10.90	82.00	74.00–87.00	82.28	8.22	82.00	78.00–88.00	0.001
Language	25.60	2.32	26.00	25.00–27.00	26.37	1.68	26.00	26.00–28.00	<0.001
Verbal fluency	19.18	2.99	20.00	18.00–20.00	19.73	1.99	20.00	20.00–20.00	0.018
Executive function	34.74	7.97	37.00	30.00–40.25	36.18	6.62	37.00	31.75–41.00	0.018
ALS non-specific	27.45	4.88	28.00	25.00–31.00	29.21	3.94	28.00	27.00–32.00	<0.001
Memory	16.02	4.59	17.00	14.00–19.00	17.58	3.74	17.00	16.00–20.00	<0.001
Visuospatial abilities	11.43	1.16	12.00	11.00–12.00	11.63	0.76	12.00	11.00–12.00	0.081

ALS: amyotrophic lateral sclerosis; ECAS: Edinburgh cognitive and behavioral ALS screen; iqr: interquartile range; m: mean; mdn: median; sd: standard deviation.

When patients obtain a score corresponding to the lowest 2.275% or 5.000% in the normative population, their scores are interpreted as an indication for the presence of a deficit. The percentages of patients with ALS who obtained a score lower than the respective cut-offs are presented in Table 6. For all scores, the percentage of patients with a score lower or equivalent to the cutoff exceeded the expected percentage. Furthermore, differences in classification due to the method of classification exist.

Table 6 Frequency and percentage of patients with cognitive deficits according to ECAS cut-off scores.

	z-Score				Percentile
Score	≤2.275%		≤5.000%		≤2.275%		≤5.000%
	n	%	n	%	n	%	n	%
ECAS	46	10.75	67	15.64	36	8.41	60	14.02
ALS specific	44	10.28	66	15.42	36	8.41	56	13.08
Language	42	9.81	48	11.22	25	5.84	58	13.55
Verbal fluency	40	9.34	47	10.98	21	4.91	43	10.05
Executive function	34	7.94	55	12.85	23	5.37	44	10.28
ALS non-specific	49	11.45	70	16.36	27	6.31	54	12.62
Memory	48	11.22	57	13.32	34	7.94	53	12.38
Visuospatial abilities	47	10.98	47	10.98	31	7.24	47	10.98

ALS: amyotrophic lateral sclerosis; ECAS: Edinburgh cognitive and behavioral ALS screen.

Discussion

The ECAS was developed specifically as a brief screening tool for cognitive deficits in patients with ALS (4). The objective of the present study was to derive normative data for the Dutch version of the ECAS in a large sample of control subjects and to compare the cognitive profile of patients with ALS with the profile of control subjects based on ECAS scores.

In this study, normative data for the ECAS were derived in a large sample of controls. Following the ECAS guidelines (4), the normative data for verbal fluency were derived using the means and standard deviations in an unstratified sample of controls. Scores were assigned to performance brackets and subsequently normative data were derived for the fluency tasks. This method, however, might not be the optimal method for the derivation of normative data for the ECAS fluency tasks. Instead parametric (25) or nonparametric methods that directly derive normative data for the ECAS VFI may result in more accurate normative data.

Since age and level of education are correlated to the performance on the ECAS, norms were derived and presented for stratified samples following ECAS guidelines (4). Results from the multiple linear model indicated that sex, might be correlated to performance on the ECAS as well. Moreover, these results demonstrated that the derivation of normative data is hampered by non-normally distributed scores. Scores near the ceiling of the ECAS, its domains, and subdomains generally predominate, indicating that most tests are within the competence of most participants, resulting in a non-normal distribution. Norms were, therefore, derived using both the traditional parametric method and nonparametric methods. The assessment of different methods for calculating normative data demonstrated that violations of assumption of normality have consequences for the classification of patients. Differences in classification appear to be more prominent in scores with a large range of scores, e.g. the ECAS total score while differences in classifications appear to be less pronounced in scores with a small range of obtained scores, e.g. the visuospatial score.

Given that the ECAS scores were negatively skewed, norm tables were constructed to convert scores to percentiles. Since level of education and age were correlated to performance on the ECAS, normative data should be stratified accordingly. Although the results of the multiple linear regression indicated that sex is associated with the performance on the ECAS, the derived normative data were not stratified by sex to avoid small sample strata in a stratified norm table. Stratification of the normative sample can be avoided when percentiles are derived using the residuals of the multiple linear regression model. Moreover, this method allows for a more precise estimation of the effect of age, as it is treated as a continuous variable rather than a categorical variable. The provided percentiles are the recommended metric for the clinical classification of patients with ALS. Percentile rank scores, however, are not feasible for use in studies that aim to compare groups of patients or compare scores over time, since percentiles are not suitable for further statistical analyses (26).

With regard to the comparison of global cognitive functions with ECAS scores between patients and controls, our results corroborate the pattern of cognitive impairment reported in the literature (24,27) with deficits on the ECAS total score, the ALS specific score, and the domains of language, verbal fluency, and executive function. Deficits on the ALS non-specific score and memory were also observed. Although memory is part of the ALS non-specific section of the ECAS, deficits in this domain are observed relatively frequent in patients with ALS (24). Nevertheless, it is important to note that patients with ALS with memory impairment often have deficits in multiple domains while isolated memory or predominant memory impairment, as is characteristic for Alzheimer’s disease, is rare (24). As such, the division between an ALS specific- and an ALS nonspecific score of the ECAS may still be valuable to distinguish FTD spectrum deficits from an Alzheimer profile. The percentage of patients who obtained a score lower than the cutoffs exceeded the expected percentage for all ECAS scores. These percentages, however, were not as high as those reported in previous studies (7,9,10). This might be explained in part by differences in patient samples and differences in the normative data.

The diagnostic accuracy of the Dutch ECAS as a screening instrument for cognitive deficits in patients with ALS and its role in the diagnostic pathway should be assessed in a future study with a full NPE as reference standard (28). For the assessment of cognitive function, the NPE is the most ideal reference standard. In clinical use, scores on the NPE are interpreted separately rather than as part of one composite score. These scores are more specific measures of cognitive function and might therefore be more informative for health care professionals, patients, and caregivers of patients with ALS. Although, the ECAS total score may be an accurate measure for screening for overall cognitive function in patients with ALS in a clinical setting, it cannot substitute for an NPE. With regard to this reference standard, it is important that it comprisse equivalent tests. Moreover, missing data on the reference standard due to impaired motor function and other kinds of nonrandom missing data should be avoided using tests that either do not depend on motor function or correct for motor function. Finally, the focus of a future study on the diagnostic accuracy of the ECAS should be on the predictive values, i.e. the probabilities that patients have a positive result on the NPE given a positive result on the ECAS and that patients have a negative result on the NPE given a negative result on the ECAS. These predictive values can be assessed for different cutoff values, i.e. percentiles of the ECAS.

Strengths of the present study were the sample size of the normative sample and the assessment of the distribution of the normative data. A limitation of the present study is that percentile norms were based on rather broad categories and that there is variation within these categories that could be explained by the differences between individuals in the same category. A more detailed analysis of the normative data, however, was not feasible with the present normative sample.

The derivation of country or language specific normative data is pivotal for an accurate interpretation of ECAS results. Moreover, normative data should be derived in large representative samples using nonparametric methods. The present study provides normative data for the Dutch ECAS and thus provides health professionals with accurate estimates of how common or uncommon patients’ ECAS scores are.

Declaration of interest

L.A. Bakker, L.A. Spreij, M. Verhaegen, J. de Vocht, Dr. C.D. Schröder, Dr. T.C.W. Nijboer, Prof. Dr. J.M.A. Visser-Meily, and Prof. P. van Damme have nothing to disclose.

Additional information

Funding

Prof. Dr. L.H. van den Berg reports grants from ALS Foundation Netherlands, grants from The Netherlands Organization for Health Research and Development (Vici scheme), grants from The Netherlands Organization for Health Research and Development (SOPHIA, STRENGTH, ALS-CarE project), funded through the EU Joint Programme – Neurodegenerative Disease Research, (JPND), personal fees from Baxalta, personal fees from Biogen, personal fees from Cytokinetics, other from Prinses Beatrix SpierFonds, other from Latran Foundation, outside the submitted work. M.A. van Es received grants from the Netherlands Organization for Health Research and Development (Veni scheme), The Thierry Latran foundation, Joint Programme – Neurodegenerative Disease Research (JPND) and the Netherlands ALS Foundation (Stichting ALS Nederland). He received travel grants from Baxalta and serves on the biomedical research advisory panel of the motor neurone disease association (MNDA).