Defining cognitive impairment in amyotrophic lateral sclerosis: an evaluation of empirical approaches

Abstract

Objective: Amyotrophic lateral sclerosis (ALS) is a multi-system disorder characterized primarily by motor neuron degeneration, but may be accompanied by cognitive dysfunction. Statistically appropriate criteria for establishing cognitive impairment (CI) in ALS are lacking. We evaluate quantile regression (QR), that accounts for age and education, relative to a traditional two standard deviation (SD) cutoff for defining CI. Methods: QR of cross-sectional data from a multi-center North American Control (NAC) cohort of 269 healthy adults was used to model the 5^th percentile of cognitive scores on the Edinburgh Cognitive and Behavioral ALS Screen (ECAS). The QR approach was compared to traditional two SD cutoff approach using the same NAC cohort (2SD-NAC) and to existing UK-based normative data derived using the 2SD approach (2SD-UK) to assess the impact of cohort selection and statistical model in identifying CI in 182 ALS patients. Results: QR-NAC models revealed that age and education impact cognitive performance on the ECAS. Based on QR-NAC normative cutoffs, the frequency of CI in the 182 PENN ALS patients was 15.9% for ALS specific, 12.6% for ALS nonspecific, and 15.4% for ECAS total. This frequency of CI is substantially more conservative in comparison to the 2SD-UK (20.3%–34.6%) and modestly more conservative to the 2SD-NAC (14.3%–16.5%) approaches for estimating CI. Conclusions: The choice of normative cohort has a substantial impact and choice of statistical method a modest impact on defining CI in ALS. This report establishes normative ECAS thresholds to identify whether ALS patients in the North American population have CI.

Keywords:

• Cognition
• ECAS
• normative data
• neuropsychology
• ALS-FTD
• dementia
• biomarker
• risk

Amyotrophic lateral sclerosis (ALS) is a multi-system disorder primarily characterized by progressive lower motor neuron (LMN) and upper motor neuron (UMN) degeneration (1), but there is also increasing evidence for cognitive and behavioral impairment (2). Indeed, current international consensus criteria for ALS frontotemporal spectrum disorder (ALS-FTSD) recommend two axes for the diagnosis of ALS including (I) defining the motor neuron syndrome and (II) defining neuropsychological deficits (3). Cognitive impairment in ALS is an important clinical consideration given its association with poor prognosis including reduced survival and functional decline (4).

The Edinburgh Cognitive and Behavioral Assessment Screen (ECAS) is an established brief neuropsychological assessment designed specifically for ALS that accommodates physical challenges (e.g., limited ability to speak or write) that potentially confound traditional neuropsychological testing in this population (5). It is recommended as one of the principal tools for assessing cognition in ALS-FTSD (3). Administration of the ECAS takes approximately 25 min to complete and uses a 136-point scale to evaluate several domains of cognition including language, executive functioning, and verbal fluency that are commonly believed to be most often affected in ALS, collectively contributing to an “ALS Specific” composite score, as well as visuospatial and memory domains that are thought to be less frequently affected in ALS and contribute to an “ALS Non-Specific” score. The ECAS has been validated against more extensive neuropsychological tests (6,7) and multimodal studies, suggesting that ECAS performance relates to biologically plausible regional neuroimaging (8) and neuropathological burden (9) in ALS patients. Although the ECAS has been translated into several languages (10–13), normative data to empirically define cognitive impairments in North American ALS patients are lacking, and only rare prior studies have considered age and education when defining thresholds of cognitive impairment (14,15). Moreover, prior approaches to defining impairment cutoffs on the ECAS have typically been based on data from relatively small series of healthy adults and have been based on statistical models that may violate important assumptions about the psychometric properties of ECAS scores.

In this report we present and compare three sets of normative thresholds for defining cognitive impairment, calculated based on different reference populations [N = 40 single-center United Kingdom controls vs. N = 269 multi-center North American Controls (NAC)] and statistical methods: the most common approach of “2 standard deviations below the mean” (2SD) vs. quantile regression with covariate adjustment (QR) is summarized in Table 1. Moreover, we illustrate how the choice of normative cutoffs impacts the classification of impairment (or not) in a cohort of 182 ALS patients.

Table 1 Study populations from which normative data were collected, statistical methods, and threshold strategies used to define cognitive impairment on the ECAS.

	2SD-UK	2SD-NAC	QR-NAC
Study population	N = 40	N = 269	N = 269
	UK Controls	North American Controls	North American Controls
		(CALSNIC, CRiALS, PENN)	(CALSNIC, CRiALS, PENN)
Statistical method	Mean ± 2SD	Mean ± 2SD	Quantile regression
Threshold strategy	2SDs below the mean^a	2SDs below the mean	5th percentile, adjusting for age and education

SD: standard deviation; NAC: North American Controls. ^aCutoff values previously published⁵.

Methods

Control participants and sources of normative data

Data to establish normative ECAS performance were pooled together from three North American sources which we collectively refer to as the North American Control (NAC) cohort: the research cohort at the Penn Frontotemporal Degeneration Center and Comprehensive ALS Center at the University of Pennsylvania (PENN); the multi-center study conducted by the Clinical Research in ALS study at the University of Miami (CRiALS); and the multi-center study conducted by the Canadian ALS Neuroimaging Consortium (CALSNIC) (16). This resulted in a total of 269 healthy adults; each source of data is described in detail below and demographic characteristics are summarized in Table 2.

Education was recorded differently across studies, and for simplicity in interpretation, we categorically defined education for statistical models as individuals who completed a four-year college degree or higher level of education (≥College) or did not (<College).

Table 2 Summary demographics and ECAS performance from three cohort studies of healthy adults, constituting our North American Controls (N = 269).

Demographics	Overall		CALSNIC		CRiALS		PENN
Total N (% female)	269 (56%)		147 (54%)		40 (70%)		82 (52%)
Age, mean (SD)	55.5 (11.5)		56.7 (9.9)		50.2 (12.8)		56.0 (13.0)
≥College education, %	60%		50%		80%		70%
ECAS Performance	Mean	Median	Mean	Median	Mean	Median	Mean	Median
	(SD)	[25–75 %tile]	(SD)	[25–75 %tile]	(SD)	[25–75 %tile]	(SD)	[25–75 %tile]
Language	26.5	27	26.4	27	26.6	27	26.7	27
(Max score = 28)	(1.8)	[26, 28]	(2.0)	[26, 28]	(1.7)	[26, 28]	(1.5)	[26, 28]
Fluency	18.6	20	18.0	20	18.9	20	19.5	20
(Max score = 24)	(4.0)	[16, 20]	(4.3)	[16, 20]	(3.0)	[18, 20]	(3.8)	[18, 22]
Executive	39.8	41	39.3	41	39.9	41	40.7	42
(Max score = 48)	(4.5)	[38, 43]	(4.9)	[38, 43]	(4.4)	[38, 43]	(3.6)	[38, 43]
Memory	17.1	17	16.8	17	18.2	19	17.1	17
(Max score = 24)	(2.9)	[15, 19]	(3.0)	[15, 19]	(2.4)	[16.75, 20]	(2.7)	[15, 18.75]
Visuospatial	11.8	12	11.8	12	11.8	12	11.8	12
(Max score = 12)	(0.5)	[12, 12]	(0.4)	[12, 12]	(0.5)	[12, 12]	(0.6)	[12, 12]
ALS Specific	84.9	87	83.7	86	85.4	87	86.9	89
(Max score = 100)	(8.1)	[81, 90]	(8.9)	[80, 89]	(6.6)	[81, 9]	(6.7)	[84, 91]
ALS Nonspecific	29.0	29	28.7	29	30.0	31	29.0	29
(Max score = 36)	(3.0)	[27, 31]	(3.1)	[27, 31]	(2.4)	[28, 32]	(2.8)	[27, 30.75]
Total	113.9	115	112.4	114	115.4	117	115.9	116
(Max score = 136)	(9.3)	[110, 120]	(10.5)	[108, 120]	(7.6)	[110, 121]	(7.3)	[112, 122]

CALSNIC

147 healthy adults recruited across Canada and the U.S. and with no history of neurological disease, head trauma, or significant psychiatric history. The CALSNIC research protocol was approved by a Health Research Ethics Board convened at each of the participating universities including University of Alberta, University of British Columbia, Université Laval, University of Calgary, the Montreal Neurological Institute and Hospital at McGill University, Western University, University of Toronto, and University of Miami. All participants at each of these institutions provided written consent following an approved informed consent procedure.

CRiALS

40 healthy adults recruited from across the U.S. with no history of neurological disease, no significant psychiatric history, and either have no family history of ALS or FTD or do not carry the genetic mutation(s) known to cause ALS/FTD in their family. The Clinical Research in ALS (CRiALS) research protocol was approved by an Institutional Review Board convened at the University of Miami and all participants provided written consent following an approved informed consent procedure.

Penn

82 community-dwelling healthy adults who self-report to have no history of neurological or significant psychiatric condition were screened to have a Mini Mental Status Examination score of 27–30. The PENN research protocol was approved by an Institutional Review Board convened at the University of Pennsylvania and all participants provided written consent following an approved informed consent procedure.

ALS participants

To illustrate the impact of different ECAS normative cutoffs on identifying cognitive dysfunction in ALS, we evaluated 182 ALS patients at PENN who were diagnosed by a board-certified neurologist according to published consensus criteria (1). Demographic and clinical characteristics are described in Online Supplementary Table 1. In addition to a clinical diagnosis of ALS, further inclusion criteria required completion of the ECAS along with three clinical assessments comprised of (1) a seated forced vital capacity (FVC) assessment of respiratory function; (2) the Revised ALS Functional Rating Scale (ALSFRS-R); and (3) Penn Upper Motor Neuron Score (PUMNS). All patients provided informed consent under a protocol approved by an Institutional Review Board convened at the University of Pennsylvania.

ECAS

All healthy adult control participants and ALS patients completed the North American ECAS (Version 1). Administration forms, published guidelines, and instructions are available online (https://ecas.psy.ed.ac.uk/ecas-international/#American). The North American ECAS was lightly adapted from the original version, as described in detail elsewhere (5), using the established Guidelines for Translation (https://ecas.psy.ed.ac.uk/wp-content/uploads/2017/11/Guidelines-for-Translation.pdf) that include using more culturally appropriate language materials in the memory recall (e.g., “shopping trolleys” → “shopping carts”, “Primose Woods” → “Marigold Woods”) and executive sentence completion (e.g., “postman” → “mailman”) tests. Scoring was also adapted to allow responses consistent with North American cultures. For example, in the naming test, we accepted “coyote” for fox, “vest” for waistcoat, “camper” or “RV” for trailer, and “porcupine” for hedgehog. All evaluators were trained in the administration and scoring of the ECAS, which assesses the participants’ oral response performance in each of the five domains (language, fluency, executive, memory, and visuospatial function).

Statistical approaches

All statistical analyses were performed using R software (Version 3.5.1). Visual assessment of histograms and Q-Q plots for each domain as well as the composite scores on the ECAS suggested that most scores are not normally distributed (see Online Supplementary Figure 1) and Kolmogorov–Smirnov tests relative to a normal distribution centered on the mean of each score confirmed lack of normality for each domain (all p < 0.1e⁻⁸). Therefore, we compare the most common approaches for defining cutoffs based on 2 standard deviations (2SD) below the mean across both sources of normative data relative to a quantile regression (QR) approach that does not make Gaussian assumptions and further accommodates adjustments for age and education.

We additionally summarize descriptive results using both parametric (e.g., mean) and non-parametric (e.g., median) statistics to facilitate comparison of previous reports.

Methods for defining thresholds of cognitive impairment

Parametric UK norms (2SD-UK)

The most widely applied approach for defining cognitive impairment on the ECAS is based on a previously established < 2 standard deviation cutoff relative to the mean in each domain and composite subscore from a sample of 40 Edinburgh-based healthy adults (5).

Parametric North American (2SD-NAC)

For comparison purposes, we also implemented a established < 2 standard deviation cutoff relative to the mean in each domain and the composite subscores, by using our sample of 269 North American healthy adults.

Quantile regression North American

Quantile regression has one key benefit over traditional parametric approaches for defining normative cutoffs. It makes no assumption about the data distribution and is, therefore, more appropriate for skewed data distributions that may have floor or ceiling effects. In addition, it can account for heteroscedasticity in which the variance of an outcome (e.g., ECAS score) is not uniform across a factor (e.g., age or education), whereas the 2SD method does not allow for covariate adjustments except through stratification (17). QR-NAC cutoffs were calculated using the R quantregGrowth package (18) for the 5^th percentile of performance in each ECAS domain and composite subscores for the 269 healthy adults.

Results

Normative ECAS performance in the North American Control Cohort

The NAC study population comprised 269 healthy adults, including 150 (56%) females. An assessment of demographics revealed, on average, that 60% of adults completed college or a higher degree (range 50–70% across studies) and participants ranged in age from 24 to 83, with a mean ± SD of 55.5 ± 11.5 years (Table 2). CALSNIC healthy adults were significantly less educated than CRiALS and PENN healthy adults (χ²=11.0; p = 0.004), and CRiALS healthy adults (ß= −6.1; p = 0.001) were significantly younger than CALSNIC and PENN healthy adults. To further evaluate heterogeneity across cohorts, we compared performance across ECAS subtest using Kruskal–Wallis tests and only observed differences in the CALSNIC healthy adults having lower S-Word verbal fluency, and the CRiALS cohort having higher delay recall and recognition performance (see Online Supplementary Table 2). These observations highlight the importance for considering age and education in statistical models and are consistent with demographic differences across studies with the proportionally less educated cohort (CALSNIC) having lower verbal fluency and the younger cohort (CRiALS) having higher memory performance.

An evaluation of ECAS performance revealed median [25^th–75^th percentile] of ECAS total, ALS specific, and ALS nonspecific scores of 115 [110–120], 87 [81–90], and 29 [27–31], with maximum possible scores of 136, 100, and 36, respectively (Table 2). Age- and education-adjusted estimates of 5^th percentile thresholds for cognitive impairment are summarized for all ECAS scores, including each cognitive domain, in Table 3 (see also Figure 1 and Online Supplementary Figure 2). Importantly, the QR-NAC approach demonstrates a significant decline in healthy adult performance associated with age on each composite score (Figure 1, Table 3), with higher educational attainment (completing college) increasing the threshold for cognitive impairment by 1–4 points. Moreover, for individual cognitive domains, the 5^th percentile of performance also declines with age for memory, fluency, executive, and language performance, but not the visuospatial domain in which performance approached ceiling across all ages (Online Supplementary Figure 2).

Figure 1 Quantile regression model for ECAS Total, ALS Specific, and ALS Nonspecific scores across age, including 5^th percentile cutoff (red) as well as for the comparison of two standard deviation approach for North America (2SD-NAC; orange) and United Kingdom (2SD-UK; blue).

Table 3 Comparison of ECAS cutoffs determined using two standard deviation approach with published United Kingdom values (2SD-UK) (5) and, in the North American Controls, using a two standard deviation (2SD-NAC) and quantile regression (QR-NAC) statistical approaches.

ECAS Score	2SD-UK	2SD-NAC	Education	QR-NAC (age)^a
				35	40	45	50	55	60	65	70	75
Language	26	23	<College	24	24	24	24	24	23	23	22	22
(Max value = 28)			≥College	24	24	24	24	24	23	23	22	22
Fluency	14	11	<College	12	12	12	11	11	9	8	6	5
(Max value = 24)			≥College	14	14	14	13	13	11	10	8	7
Executive	33	31	<College	33	32	32	31	31	30	30	29	29
(Max value = 48)			≥College	32	31	31	30	30	29	29	29	28
Memory	13	11	<College	14	13	13	12	11	11	10	9	9
(Max value = 24)			≥College	15	15	14	13	13	12	11	11	10
Visuospatial	10	11	<College	11	11	11	11	11	11	11	11	11
(Max value = 12)			≥College	11	11	11	11	11	11	11	11	11
ALS Specific	77	69	<College	74	73	72	71	70	68	67	66	65
(Max value = 100)			≥College	77	76	75	73	72	71	70	69	67
ALS Nonspecific	24	23	<College	26	25	25	24	23	23	22	22	21
(Max value = 36)			≥College	27	26	25	25	24	24	23	23	22
Total	105	95	<College	102	101	99	98	96	94	92	90	87
Max value = 136)			≥College	106	105	103	101	100	98	96	93	91

^aQuantile regression technique used to estimate the 5th percentile values adjusted for age and education.

Prevalence of cognitive impairments in PENN ALS cohort

The prevalence of cognitive impairment in ALS varies depending on the population used to generate normative data as well as the approach used to determine thresholds for defining cognitive impairment (Figure 2 and Online Supplementary Table 3). When comparing the statistical approaches (2SD and QR) developed across NAC and UK cohorts, the 2SD-NAC and QR-NAC cutoffs suggest that ∼16% of PENN ALS patients exhibit cognitive impairment based on the ALS-specific and ECAS total composite scores, substantially fewer than that estimated using the 2SD-UK approach for ALS-specific (32.4% impaired) or ECAS total (34.6% impaired). The discordance between cutoff approaches is even more apparent in the language domain with an estimated ∼63% impairment relative to ∼19% and ∼20% impaired in the PENN ALS cohort based on the 2SD-NAC and QR-NAC approaches, respectively. Fluency and executive domains also suggest a lower frequency of cognitive impairment in the PENN ALS cohort using the 2SD-NAC and QR-NAC approach relative to the 2SD-UK approach.

Figure 2 Prevalence of impairment on the ECAS in 182 ALS patients using three different sets of normative cutoffs. QR-NAC: Quantile regression North American Controls; 2SD-NAC: 2 standard deviation cutoff North American; 2SD-UK: 2 standard deviation cutoff United Kingdom.

In contrast to the NAC vs. UK comparisons above, the differences between the QR and parametric statistical approaches yield relatively similar estimates when evaluating the prevalence of cognitive impairment in the PENN ALS cohort using NAC reference populations. These include the ALS specific score −15.9% (QR-NAC) versus 16.5% (2SD-NAC) and the individual cognitive domains that contribute to this score: Language − 19.8% (QR-NAC) versus 19.2% (2SD-NAC); Executive − 11.0% (QR-NAC) versus 11.5% (2SD-NAC); and Fluency − 9.3% (QR-NAC) versus 14.8% (2SD-NAC). The greatest divergence between QR-NAC and 2SD-NAC approaches was for visuospatial impairment, 22.5% (QR-NAC) versus 32.4% (2SD-NAC), that together with memory (QR-NAC = 8.8% vs. 2SD-NAC = 9.9%) contributes to a discrepancy in rates of impairment on the ALS nonspecific score − 12.6% (QR-NAC) versus 14.3% (2SD-NAC).

Discussion

Increasing interest in frontotemporal spectrum impairment in ALS and the overlap between ALS and FTD have necessitated development of tools to quantify the nature and frequency of cognitive dysfunction in ALS. Although the ECAS, a brief assessment tool, has emerged as the dominant instrument for evaluating cognition and behavior in ALS (19), limited reference data (at least within North America) have hampered progress. In particular, efforts are underway to better understand genotype-phenotype relationships, including predictors of cognitive impairment (20). Indeed, such large-scale studies principally rely on the ECAS for quantifying cognitive impairment since longitudinal assessment using a full neuropsychological battery may be impractical, especially as accrual of motor deficits with disease progression undermines the validity of established pen-and-paper neuropsychological tests. Without appropriate normative data for the ECAS, these efforts may be less successful due to potentially significant misclassification of cognitive impairment. Moreover, the recently revised Airlie House guidelines for the design and implementation of ALS clinical trials recommend that investigators stratify for cognitive impairments (21); it is therefore essential to appropriately and accurately define whether individual patients are impaired or not.

Here we present normative data for the North American ECAS based on a moderately sized, multi-center dataset and recommend (a) that it is essential to have culturally appropriate healthy control data for categorizing cognitive impairments; (b) the use of quantile regression as appropriate for estimating normative thresholds (e.g., 5^th percentile) of data that are not normally distributed; and (c) adjusting for age and education, factors known to associate with cognitive function. Applying these findings to the ALS patient cohort at PENN, we have estimated the prevalence of cognitive impairment in ALS to be ∼16%. This is substantially less frequent than what we and others have previously estimated based on published norms, which suggested typically up to ∼50% (3,22), and one estimate as high as 70% (23), of ALS patients have cognitive impairment. Of note, these prior estimates that are inflated in comparison to our observations were derived from neuropsychological tests that, unlike ECAS, do not control for motor confounds. Future work will be necessary to validate the QR-NAC approach by comparing these suggested cutoffs relative to other more extensive “gold-standard” neuropsychological assessments. However, although such validation approaches are desirable, they are also highly challenging because the vast majority of “gold-standard” neuropsychological assessments are typically confounded by motor features. Critically, our observation of a reduced frequency of cognitive impairment in ALS appears to be less related to the implementation of a QR statistical approach than to the selection of a culturally and demographically comparable normative healthy control cohort. We discuss the interpretation of each of these factors in detail below.

Although there is increasing evidence for language impairment in ALS, the current findings suggest that estimates of the rate of language impairment vary widely depending on the statistical approach and control population employed to generate normative data, ranging from ∼19% for the 2SD-NAC and QR-NAC approaches to as high as ∼63% of PENN ALS patients using the 2SD-UK approach. These findings therefore emphasize that sociodemographic differences across cultures are a critical consideration when defining normal language performance. As the ECAS becomes increasingly used internationally, it may be valuable to account for potential cultural and regional confounds in subsequent versions of the test, such as including cross-linguistic naming of objects (24), differences in orthographic-phonemic mapping that may influence spelling across languages, and implementing norms across several languages (25). Although letter-guided verbal fluency constitutes a “domain” distinct from language in the ECAS, it is linguistically mediated and thus susceptible to cross-linguistic differences, such as variable letter frequency (26). It will, therefore, likely be important to re-calibrate the “verbal fluency indices” across ECAS versions. Nevertheless, it is valuable to note the overall relatively high rate of impairment on language skills probed by ECAS. This has been noted by other investigators (27), but may be underappreciated. In an exploratory comparison of coefficient of variation across language domain subtests, it appears that spelling has a disproportionately high level of dispersion in both the NAC cohort (14%) and the ALS cohort (23%) in comparison to naming (9% NAC; 16% ALS) and comprehension tests (3% NAC; 6% ALS). Although this suggests that spelling may be a driver of poor performance in the language domain for both healthy adult controls and ALS patients, it is not clear why this is the case. Future item analyses may help elucidate the contribution of this domain for defining language impairment and provide insight into the reliability of spelling for identifying language impairment. Lastly, although semantic deficits have been reported in ALS (28), there is increasing evidence for grammatical and discourse deficits in ALS (29–31). Supplementing the current language domain with tests of grammar and discourse in subsequent ECAS versions may provide a more accurate picture of language impairment in ALS.

Our use of the QR-NAC approach, in addition to accounting for non-normality, benefits by also accounting for cognitive performance across age. Importantly, ECAS total score cutoffs range from 106 to 87 points depending on the age of the individual, highlighting the importance of considering age in the determination of impairment by using a dynamic rather than static cutoff strategy. Memory appears to be the domain most affected by age, as well as the ALS nonspecific score, which suggests that these components are more sensitive to age-related or non-FTD contributors to cognitive decline. Indeed, memory decline in aging may reflect mild cognitive impairment or prodromal Alzheimer disease (AD) (32), and AD co-pathology has been noted in autopsy-confirmed series of ALS patients (33). In this context, it is important to consider that ALS cognitive impairment (ALSci), as defined using current clinical criteria (3), may be associated with the ALS-FTSD spectrum as a prodromal phase of ALS-FTD or ALS-dementia. Importantly, the QR-NAC approach suggests that the highest frequency of impairment based on composite scores (e.g., ALS specific, ALS nonspecific, total) was the ALS specific composite, which comprises the more predominantly affected executive, fluency, and language domains.

Beyond adjusting for age, the level of educational attainment has a modest impact on normative cutoff values. In particular, attainment of a college degree appears to have the greatest impact on verbal fluency and executive domains that contribute to the ALS-specific score, and only a 1-point impact on memory domains, whereas language and visuospatial performance remain constant across education groups. There are at least two possible reasons for this observation. One possibility for a lack of contribution of education to language and visuospatial domains may simply be a function of these domains having the most limited statistical variance. Indeed, we observed the highest proportion of cognitively impaired ALS patients in the language (particularly spelling) and visuospatial and domains and, although the QR-NAC approach does not have normality assumptions like the SD approaches, a lack of variance on these subtests may inflate estimates of impairment and underestimate the contribution of other factors like education. Another possibility, mentioned above, is that the limited range of probed language areas may be relatively insensitive to education. Yet an alternative possibility is that verbal fluency, executive, and memory domains are indeed preferentially impacted by educational status. For example, models of cognitive reserve suggest that factors like education may help stave off the clinical manifestations of underlying pathologies in aging individuals (34) and evidence suggests that reserve may selectively impact executive functions but not visuospatial functions in healthy aging (35) and frontotemporal degeneration (36).

There are several caveats to consider. First, behavior is an important component of ALS-FTSD and, although this report establishes the frequency of cognitive impairment in ALS, it does not establish normative data for defining behavioral impairment in healthy adults or ALS patients. Second, our estimates of normative cognitive performance are based on a cross-sectional evaluation, which represents a snapshot of a progressive disease in which cognitive impairment may only emerge as disease unfolds (37). Although initial efforts have parametrically defined longitudinal change on the ECAS for healthy adults (38), it is critical for future longitudinal investigations to define age-related rate of cognitive decline in healthy adults and in a manner that accounts for sociodemographic and cultural differences, to determine how the tempo of disease progression in ALS differs from normal age-related cognitive decline. Third, a limitation of pooling normative healthy adult data across cohorts is that educational attainment was not captured in a harmonized manner and therefore we were limited to dichotomizing education as college completion versus less than a college education. However, cognitive performance may be more sensitive to finer grained educational differences (e.g., high school, graduate degree) and this is an important consideration for future investigations. Likewise, our control cohort had a greater frequency of college completion (∼65%) than the general population; although the current study is the largest ECAS normative study to date, future studies with increased sampling of the general population may better capture the influence of education on ECAS performance. Fourth, the visuospatial domain is the only domain in which we observed a higher rate of cognitive impairment in ALS patients relative to the 2SD-UK approach. Although this is less exaggerated using the QR-NAC approach, which is less susceptible to ceiling effects than the 2SD-NAC approach, this observation highlights the importance for considering the psychometric properties of the ECAS and subsequent ECAS versions may benefit by revising this domain to lower the current ceiling properties. Lastly, we defined thresholds of cognitive impairment using a commonly implemented 5^th percentile cutoff for the QR approach and 2SD cutoff for the other approaches, but it is possible that alternative less conservative thresholds may be more appropriate.

In summary, estimates of the frequency of cognitive impairment in ALS vary widely depending on the choice of cohort and statistical approach used to define thresholds for defining cognitive impairment. Although our estimates are based on application of norms to ALS patients in the PENN cohort, we suggest that the use of quantile regression models, as employed in our QR-NAC approach, provides a robust strategy to define cognitive impairments that is statistically appropriate and accounts for potential confounding factors such as age and education.

Acknowledgements

The authors extend thanks to research coordinators at the University of Pennsylvania including Laura Baehr, Laura Hennessy, and Hayley Donaldson for their efforts with participant recruitment and evaluation. CALSNIC additionally thanks the research team for participant recruitment and evaluations. In addition, CRiALS extends thanks to the research team at the University of Miami for participant recruitment and evaluation (Anne-Laure Grignon, Danielle Dauphin, Danielle Sheldon, Sumaira Hussain, Anne Cooley, Jessica Medina, Ashley Manso).

Disclosure statement

Dr. McMillan receives research funding outside the submitted work from Biogen, Inc. and provides consulting services for personal fees from Invicro and Axon Advisors on behalf of Translational Bioinformatics, LLC. He also receives an honorarium as Associate Editor of NeuroImage: Clinical.

Dr. Johnston receives research funding from Annexon, Alexion, ALS-Pharma, Biogen, Cytokinetics, Mallinckrodt, Mitsubishi-Tanabe Canada, Orion, and serves as a paid consultant to Biogen and Mitsubishi-Tanabe Canada.

Dr. Genge provides consulting services for the following companies: AB Sciences, Akcea, Alexion, AL-S Pharma, Anavex, Avexis, Bayshore, Biogen, Clene, CSL Behring, Cytokinetics, QurAlis, Mitsubishi Tenabe Pharma America, Novartis, Orion, Revalesio, Roche, Sanofi Genzyme, and Wave life sciences. She is also CRU Medical Director, PI or sub-PI on trials sponsored by the following companies: AB Sciences, AL-S Pharma, Acceleron, Amicus, Alnylam, Bioblast, Biogen, BMS, Boston Biomedical, Cytokinetics, Sanofi Genzyme, Grifols, Ionis, Lily, Mallinckrodt, Medimmune, Novartis, Orion, Orphazyme, Pfizer, Ra Pharmaceuticals, Roche, Teva, and UCB.

Dr. Benatar reports personal fees from Roche and Biogen.

Drs. Wuu, Rascovsky, Cosentino, Grossman, Elman, Quinn, Rosario, Stark, Granit, Briemberg, Chenji, Dionne, Korngut, Shoesmith, Zinman report no conflicts of interest.

Additional information

Funding

This work was supported by Amyotrophic Lateral Sclerosis Association; Fondation Brain Canada. Dr. McMillan reports grants from the National Institutes of Health [NIH; AG017586; AG066597]. Dr. Kalra reports grants that support CALSNIC including the Canadian Institutes of Health Research, ALS Society of Canada, Brain Canada Foundation, and Shelly Mrkonjic ALS Research Fund. Dr. Benatar reports grants from Muscular Dystrophy Association [Grants #4365 and #172123], the ALS Association [Grant #2015], and the National Institutes of Health [NIH; NS105479]; philanthropic support from the ALS Recovery Fund and the Kimmelman Estate during the conduct of the study.