Evidence of validity and reliability of the Colombian version of Addenbroke’s Cognitive Examination Revised (ACE-R)

Abstract

Objective

The purpose of this study is to provide evidence that supports the validity and reliability of the Colombian version of the Addenbrooke’s Cognitive Examination Revised (ACE-R) in comparison to the MMSE at assessing and finding patients with Mild Cognitive Impairment (MCI). Additionally, the study aims to determine the optimal cut-off scores based on the age of a population with a low education level.

Method

This study included 314 individuals (235 participants diagnosed with MCI and 79 cognitively healthy) who live in two different rural departments (states) in Colombia. The participants were recruited for this study through community clubs for the older adults. Most of the individuals were female (236), the average age was 65.95 years of age (SD= 7.8), and the average education level was of 3.78 years (SD = 1.79). It is important to note that the sample only included people with a maximum of 6 years of schooling.

Results

A ROC analysis indicated that the ACE-R is more effective than the MMSE at evaluating and finding MCI individuals within the three groups. The cut-off points for the Under 60 years of age group was 83.50 (sensitivity 0.880% and specificity 0.632%); 61–69 years of age 80.50 (sensitivity 0.714% and specificity 0.677%); and Over 70 years of age was 79.50 (sensitivity 0.750% and specificity 0.659%). The internal consistency analysis with MacDonald’s Ω determined reliability indicators ≥70 in the ACE-R, except for the age range of 61 to 69 years.

Conclusion

The Colombian version of the ACE-R demonstrates to be a valid and reliable global cognitive screening tool. It is effective at discerning MCI individuals from healthy within a group of participants with a low education level.

Keywords:

• Mild cognitive impairment
• dementia
• aged
• Latin America
• neuropsychology

Introduction

Mild cognitive impairment (MCI) is considered an intermediate state between the cognitive changes associated with normal aging and mild dementia (Petersen, 2004), with the prevalence between 10 and 20% in people older than 65 years (Petersen et al., 2010).

Dementia and its impact is predicted to increase from 50 million people to 82 million people in the next decade (World Health Organization [WHO], 2020) as a result of the global population aging process (Bonilla-Santos et al. 2021; Fang et al. 2014; Pan et al. 2022; Prince et al. 2013; Wang et al. 2017) and the increasing need for dependence of older adults who will be affected by this condition (Calderón et al. 2021; Hildreth & Church, 2015; Waheed et al. 2020).

Latin America and the Caribbean are among the regions that will be most affected, with an estimated increase of 7.6 million by 2030 (Organización Panamericana de la Salud, 2023). These regions exhibit a high prevalence of dementia compared to Europe and the United States, due to significant risk factors including genetics and social determinants of health (Parra et al. 2020; Santamaria-Garcia et al. 2023). In Colombia, the prevalence of dementia is 9.4% (>60 years), and it increases to 57.4% in people over 85 years of age, particularly among those with low income and low education levels (McEniry et al. 2019; O’Donovan et al. 2020; Paredes-Arturo et al. 2019; Pedraza et al. 2017; Vargas et al. 2014).

The availability of validated screening tools for assessing cognitive functioning in populations with low education levels and exposure to social adversities is indispensable as it can reduce healthcare costs. Therefore, highly sensitive neuropsychological instruments with diagnostic specificity are required (Broche-Pérez & López-Pujol, 2017; Carnero-Pardo, 2014; Lorentz et al. 2002; Matias-Guiu et al. 2015; Siciliano et al. 2016; Torralva et al. 2011; Wang et al. 2017) to identify dementia and its associated factors promptly (Porto et al. 2022). For multicentred dementia studies, the use of valid and reliable tools that ensure decision-making (Aiello et al. 2022; Tsoi et al. 2015) that can be applied in any globalized context becomtes a necessity (Santamaria-Garcia et al. 2023; Zugman et al.2023).

Addenbrooke’s Cognitive Examination (ACE) and its subsequent versions (ACE-R, ACE-III, M-ACE, and Mini-Addenbrooke’s Cognitive Examination) have demonstrated high sensitivity and specificity for the early detection of mild cognitive impairment and dementia. This instrument has been validated and translated into several languages (Mirza et al. 2017; Ospina, 2015; Waheed et al. 2020) and exhibits adequate psychometric properties (Calderón et al. 2021; Custodio et al. 2020; Junco & Prieto, 2014) (see Table 1). However, most studies have focused on patients in clinical or hospital care settings, with participants typically having a minimum of 9 years of schooling. Therefore, its interpretation for diagnosing mild cognitive impairment in populations with less than five years of schooling and deeper social problems is limited (Custodio et al. 2017; Ospina, 2015; Xiang et al. 2021)

Table 1. Education level of participants worldwide assessed with the ACER-R.

Study	Country	Schooling (mean years/SD)
Pigliautile et al. (2011)	Italy	7,1 ± 3,7
Pigliautile et al. (2015)	Italy	9,4 ± 4,80
Siciliano et al. (2016)	Italy	10,87 ± 5,15
Alexopoulos et al. (2010)	Germany	11,02 ± 2,63
Yoshida et al. (2012)	Japan	11,01 ± 2,70
Fang et al. (2014)	China	10,07 ± 4,41
Wang et al. (2017)	China	11,30 ± 1,40
Pan et al. (2022)	China	11,67 ± 4,01
García-Caballero et al. (2006)	Spain	13,13 ± 2,36
Matias-Guiu et al. (2015)	Spain	7,5 ± 4,6
Carvalho and Caramelli (2007)	Brazil	8,5 ± 4,3
Torralva et al. (2011)	Argentina	12,9 ± 4,6
Custodio et al. (2012)	Perú	11,7 ± 2,8
Muñoz-Neira et al. (2012)	Chile	10,80 ± 5,03
Broche-Pérez and López-Pujol (2017)	Cuba	8,59 ± 3,67
Ospina (2015)	Colombia	8,20 ± 4,80

In the specific context of the low-income Colombian population, which encompasses a wide array of social, environmental, and economic challenges, this article aims to assess the validity and reliability of the Colombian version of the Addenbrooke’s Cognitive Examination Revised (ACE-R) and determine the optimal in a population with less than 6 years of schooling.

Method

This study was carried out throughout the years 2019, 2020 and 2021, framed within the South Colombian study of dementia (see Figure 1).

Figure 1. Selection of study population.

Assessment procedure

The research protocol included a variety of neuropsychological tests for the early evaluation of Alzheimer’s Disease. All study subjects joined the projects and followed the same stages until they were finally classified as part of the cognitively healthy control group or group with possible MCI. The stages consisted of 1. clinical interview and 2. application of cognitive screening tests (MMSE) and (ACE-R), depression screening (Yesavage Geriatric Scale – GDS), screening scale for the diagnosis of multi-infarct dementia― the Hachinski Ischemic Scale and the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD), Colombian version.

MCI diagnosis was determined through clinical consensus when the patient met the criteria stipulated by Petersen (2004) and Winblad et al. (2004): (a) changes in cognition recognized by the participant under evaluation and/or by a close informant; (b) MMSE score below 1.5 standard deviations according to the normative data for age and schooling; (c) Objective memory impairment (single domain amnestic MCI), memory impairment and another cognitive process (multiple-domain amnestic MCI), impairment of a cognitive process other than memory impairment (single domain non-amnestic MCI), impairment of two or more cognitive processes other than memory impairment (multi-domain non-amnestic MCI) assessed through CERAD domains (the participant having 1.5 standard deviations from what was expected for age and schooling based on the CERAD normative data for Colombia was taken into account); and (d) autonomy at performing functional activities of daily living assessed using the Lawton and Brody Scale (Scores of 8 for women and ≥ 5 for men). The Reiserbeg Global Scale of Impairment (GDS) was also used to assess the stage of cognitive deficit in (1) absence of cognitive impairment, (2) very mild cognitive impairment, (3) mild cognitive impairment, (4) moderate cognitive impairment and (5) absence of Alzheimer’s dementia according to the NINCDS-ADRDA criteria, and absence of vascular dementia according to the NINDS-AIREN criteria. Participants with a history of (1) neurological disease, (2) traumatic brain injury and (3) neurodevelopmental and learning disabilities, were excluded from the study.

The Colombian version of the ACE-R was adapted and validated in 2015 by Ospina, to create a reliable, low-cost, and easy-to-administer screening test that could assess multiple cognitive domains in the Colombian population. The adaptation of the original version was accomplished through translation and extensive revision by experts in neurology and neuropsychology with knowledge of the two languages involved (English and Spanish). The modifications made for the Colombian version were for the attention and orientation domain within the three-word register item; based on the word typification analysis, apple, ball, and key were the words included. In the memory domain, the address item was modified according to how it is used in a conventional and generalized manner in addresses in Colombia. The retrograde memory questions were also modified to be consistent with the schooling average of adults and older adults of the population in the study―that is, these questions would be known to the general population so that the act of recalling this information could be assessed, thus avoiding a bias due to lack of knowledge. No modifications were made for the fluency domain. In the language domain, the comprehension, repetition of words and writing items remained the same, as no significant linguistic or cultural differences could be found with respect to the original test. Changes were made to the sentence reading item, as sentences became meaningless when literally translated into Spanish. For this reason, they were changed to sentences that had been widely adopted in other tests validated in Colombia, such as the MMSE and CERAD: ‘If I don’t go down, then you go up’ and ‘The man is walking down the street’. In the naming item there are some drawings that differ from those in the original, such as the lion, the pig, the chicken, the guitar, and the shovel. The pencil, the watch, the camel, the crocodile, the anchor, the crown, and the accordion remained unchanged. For the comprehension and semantic associations item, modifications were made to add a different degree of difficulty to each question. In the reading item, after a consensus was reached by a group of experts that included neurologists, linguists and neuropsychologists, the words Hectare, Water, Access, Success, and That (Hectárea, Agüita, Acceso, Éxito, Aquello) were chosen. The author did not deem it necessary to make any modifications in the visuospatial skills domain.

Participants

A total of 314 subjects are included in the study, who were mostly female (75.2%), with a maximum level of 6 years of schooling and who belonged to two rural departments located in the Andean region: Huila (64.6%) and Caquetá (35.4%), in Colombia. The average age was 65.95 years of age (SD = 7.8), 74.8% of the participants had a diagnosis of MCI and 25.2% were cognitively healthy.

Data analysis

A descriptive analysis of the sample obtained and classified the variables ‘presence/absence of mild cognitive impairment’, ‘age’ and ‘educational level’. Subsequently, a Kolmogorov–Smirnov analysis was performed to determine whether the measurements were parametrically distributed. Age was stablished as a categorical variable, considering the ranges under 60 years, from 61 to 69 years and over 70 years. The comparison between cognitively healthy and MCI participants was made with the Mann–Whitney U test on independent samples, the Kruskal–Wallis test by age ranges in MCI and the Levene test was applied to determine the equality of variances. To establish the sensitivity, specificity and cut-off points of each test separate analyses were carried out using ROC curves by age range. MacDonald’s Ω statistics were used to determine the internal consistency of the ACE-R test. For statistical decision-making, a significance of less than 0.05 was used. The analyses were performed with the SPSS v26 ® and Jasp ® software. It was impossible to match any of the measured variables to normality criteria. Therefore, we determined that the data showed non-parametric behavior.

Ethics

During the execution of this project, the fundamental base was the knowledge and compliance of the different ethical standards at international and national level regarding the study of humans. This included the Declaration of Helsinki, the International Ethical Guidelines for Biomedical Experimentation in Human Beings, the Standards of Good Clinical Practices, Resolution No. 008430 of 1993 of the Ministry of Health and Social Protection of Colombia as well as the Law that dictates the Code of Ethics and Bioethics of the Psychologist in Colombia, Law 1090 of 2006. This project included the Endorsement of the Ethics, Bioethics and Research Committee 002-006 University Hospital.

Results

Regarding the description of the sample, a mean age of 65.95 years of age was identified, and the corresponding mean level of education was 3.78 years (SD = 1.79). Of these, 11.8% reported a history of dementia in their family and less than 5% reported some major somatic comorbidity―except for diabetes―which was present in 12.7% of those evaluated (Table 2).

Table 2. Characteristics Of the sample assessed.

Group	f	%	Other medical history	f	%
Cognitively healthy control	79	25.2	Cerebrovascular disease
MCI	235	74.8	No	309	98.7
Department of origin			Yes	4	1.3
Huila	203	64.6	Cardiopathy
Caquetá	111	35.4	No	296	94.3
Sex			Yes	18	5.7
Male	78	24.8	Hepatopathies
Female	236	75.2	No	298	94.9
Laterality			Yes	16	5.1
Right	309	98.4	Diabetes mellitus
Left	4	1.3	No	267	85.0
Ambidextrous	1	0.3	Yes	47	15.0
Is there a history of dementia in your family?
No	278	88.5
Yes	36	11.5
Have you experienced any thinking or memory disorders?			Have you had any memory loss in the past year?
No	111	35.4	No	176	56.1
Yes	203	64.6	Yes	138	43.9

Moreover, there are no significant differences when comparing age and sex, but there are in terms of education (age ranges 61–69); however, it was identified that, according to the age ranges (under 60 years, 61–69 years and over 70 years), reported schooling decreases. Therefore (and in line with the available empirical evidence), this study defines the age ranges as criteria for establishing internal consistency indices (sensitivity and specificity) (Table 3).

Table 3. Classification of study groups by sex, age, and level of education.

	Age			Years of schooling
	M.C.I.	C.H.	p value	M.C.I.	C.H.	p value
	Mean (S.D.)	Mean (S.D.)		Mean (S.D.)	Mean (S.D.)
Under 60 years of age	56.46 (3.22)	56.52 (2.87)	.93	3.72 (1.77)	4.88 (1.33)	.06
61–69 years of age	64.39 (2.24)	64.95 (2.57)	.20	3.54 (1.90)	4.69 (1.35)	.01
Over 70 years of age	75.98 (4.90)	75.17 (4.65)	.59	3.27 (1.77)	4.08 (1.51)	.13
Women	62.92 (7.72)	67.17 (8.12)	.08	3.56 (1.79)	4.28 (2.02)	.10
Men	64.00 (6.71)	66.46 (8.64)	.10	3.66 (1.82)	4.56 (1.22)	.08

M.C.I.: Mild Cognitive Impairment; C.H.: Cognitively Healthy. p value was made from the U Mann Whitney.

The difference in medians in total MMSE and ACE-R between cognitively healthy control patients and patients with MCI, indicates that the assessment subscales presented significant differences, excluding the attention subscale (Table 4).

Table 4. Differences between cognitively healthy control patients and patients with MCI.

Variable	Cognitively healthy control (n = 79)				MCI (n = 235)				Z	Sig.
	X	SD	Min.	Max.	X	SD	Min.	Max.
ACE-R Total score	83.19	12.95	24.00	96.00	75.03	12.73	20.00	98.00	−6.65	<0.001
ACE-R Orientation/10	9.80	.74	5.00	12.00	9.23	1.53	.00	13.00	−4.115	<0.001
ACE-R Attention/8	6.78	1.44	3.00	8.00	6.78	1.66	1.00	8.00	−0.554	0.579
ACE-R Memory/26	21.32	3.19	13.00	26.00	17.64	4.89	2.00	26.00	−6.129	<0.001
ACE-R Fluency/14	9.76	2.16	3.00	14.00	8.24	3.05	.00	27.00	−4.451	<0.001
ACE-R Language/26	23.66	2.07	15.00	26.00	21.29	3.46	9.00	26.00	−5.797	<0.001
ACE-R Visuospatial/16	14.87	8.65	9.00	89.00	11.91	2.94	4.00	20.00	−5.724	<0.001
MMSE Total	26.95	2.32	20.00	30.00	25.29	3.71	6.00	30.00	−3.894	<0.001

An ROC analysis was conducted to simultaneously compare the behavior in the ACE-R and MMSE in terms of area under the curve, cut-off point, sensitivity, and specificity for each of the age segments defined. When comparing the areas under the ROC curves covered by the two instruments, statistically significant differences were obtained (z = 3,288; diff. 0.105; C.I. 95% = 0.041–0.168; p value = 0.001) (Table 5).

Table 5. ROC analysis to differentiate performance of ACE-R and MMSE.

Age segment	Area under the curve	C.I. (95%)	Cut-off point	Sensitivity (%)	Specificity (%)
Under 60 years of age
MMSE	.582	.442–.723	26.50	.520	.439
ACE-R	.769	.667–.871	83.50	.880	.632
61–69 years of age
MMSE	.644	.545–.743	26.50	.619	.591
ACE-R	.728	.637–.819	80.50	.714	.677
Over 70 years of age
MMSE	.675	.524–.827	26.50	.583	.588
ACE-R	.687	.493–.881	79.50	.750	.659
Total
MMSE	.645	.577–.714	27.50	.519	.711
ACE-R	.750	.688–.812	80.50	.785	.633

Both tests exhibited optimal indicators for their use in MCI discrimination, being higher for ACE-R in all three groups (Figure 2).

Figure 2. Characteristics of the ROC curves of the ACE-R and the MMSE by educational level and total population.

Regarding the differences in means by age group in patients with MCI, significant differences were identified in almost all factors measured, except for the ACE-R attention (Supplementary Table S1).

Finally, an analysis was performed using MacDonald’s Ω, which determined the internal consistency of the ACE-R instrument with the entire sample and subsamples. The tests above yielded reliability indicators of ≥ 70 except for the age range of 61 to 69 years (Table 6).

Table 6. Reliability analysis of the ACE-R by educational level and total population.

Internal Consistency	Age range			Total
	Under de 60 years of age	61–69 years of age	Over 70
MacDonald’s Ω (CI 95%)	.791 (.720–.862)	.687 (.605–.769)	.813 (.753–.872)	.784 (.747–.822)

Discussion

The objective of the study is to provide evidence of the validity and reliability of the Colombian version of the Addenbrooke Cognitive Test-Revised (ACE-R) in comparison with the MMSE at discriminating patients with Mild Cognitive Impairment (MCI) from healthy ones. In addition, the study aims to determine the optimal cut-off scores analyzed by age ranges for a population with a very low education level in Colombia According to the ROC analysis, the ACE-R outperformed the MMSE in detecting mild cognitive impairment across all three age groups.

This study determinates the ACE-R cut-off points by age as follows: for the group under 60 years of age, the score was 83.50 (with a sensitivity of 0.880%, and a specificity of 0.632%); for the group between 61 and 69 years of age, the score was 80.50 (with a sensitivity of 0.714% and a specificity of 0.677%); and for the group older than 70 years of age, the score was 79.50 (with a sensitivity of 0.750% and a specificity of 0.659%). The cut-off points that this study reports differ from Ospina (2015), who determined a cut-off point of 87 for healthy subjects from Colombia with a mean of 10 years of schooling and with higher specificity and sensitivity than ours.

Regarding the ACER-R validation studies in Latin America and the Caribbean, our study consists of a population in conditions close to illiteracy and that have been exposed to social adversity, such factors might lead to the differences in the specificity and sensitivity and the cut-off points with respect to the studies carried out in Cuba (Broche-Pérez et al., 2018), whose average education was 8.59 (±3.67), Chile (Muñoz-Neira et al. 2012), 10.80(±5.03), Perú (Custodio et al. 2012) 11.7(± 2.8) Argentina (Torralva et al. 2011) 12.9(±4.6); Thus, the low-income Colombian population, with a wide range of social, environmental and economic difficulties, specifically makes the task of ensuring equivalency processes difficult. The evidence of validity and reliability of the ACE-R in the present study shows a lower internal consistency (≥70, except for the age range of 61 to 69 years) than the one obtained in the adaptation and validation carried out by Ospina (2015). This can be explained by the heterogeneity of the participants, particularly due to the differences in age and low educational level (Nieto et al. 2016).

Therefore, it is necessary to increase the use of the ACE-R in a population with such characteristics, given the implications of these difficulties for most of the Colombian adult population. This explains why the adaptation processes in this type of context cannot be extrapolated to those of other countries for example, Spain (García-Caballero et al. 2006; Matias-Guiu et al. 2015), Germany (Alexopoulos et al. 2010) or Japan (Yoshida et al. 2012), where the older adult population has on average a higher educational level and has not been exposed to similar vulnerable conditions.

In this study, the performance in terms of the cut-off point, sensitivity and specificity of the ACE-R exhibit optimal ­indicators, which allows discrimination by age segments and shows the best indicators in the age group under 60 years of age. This is of special interest for the processes of cognitive evaluation through ACE-R in the Colombian population, given the variety of social, environmental, and economic problems that impact the access and the permanence in school systems, which undoubtedly have age implications.

There is no doubt that dementia and cognitive decline represent one of the major progressive public health problems worldwide. In this sense, it is of special interest to have instruments with adequate psychometric properties that respond to the specific cultural needs of the context and that offer pertinent and relevant data on cognitive functioning in the shortest possible time (Parra et al. 2020).

In this sense, this study provides a screening tool that has evidence of validity and reliability and allows the identification of possible cognitive impairments within groups of Colombian adults with a low education level who also live in adverse conditions. This is of special interest due to the use of the ACE-R in multicenter studies (Aiello et al. 2022; Tsoi et al. 2015), which have considered including populations with low education and socioeconomic precariousness (Santamaria-Garcia et al. 2023).

This study reports several limitations, including the fact that the participants were recruited from community centers. There are differences in the years of schooling mean between the range 61–69 years of age of MCI group and the healthy ones, and there are no biomarkers (Bonilla-Santos et al. 2021) that correlate with the results neuropsychological tests. Lastly, considering that the ACE-R is a sensitive screening test for the identification of MCI, the limitation in specificity should be reported. Besides, the validation analyzes are not carried out with the dementia clinical group.

Additional information

Table Classification of study groups by age and level of education.

Acknowledgements

We thank the older adults and their caregivers for their participation and willingness to conduct the research.

Disclosure statement

No potential conflict of interest was report by the authors.

Data availability statement

Data generated within the framework of the current studies are available upon reasonable request to the corresponding author.

Additional information

Funding

This work was supported by The Science, Tecnology, and Innovation Fund- Huila of the General System of Royalties through the Ministry of Science, Technology, and Innovation of Colombia [BPIN 2020000100011] Universidad Surcolombiana and Universidad de la Amazonia, and [CONADI Universidad Cooperativa de Colombia in agreement with Universidad Surcolombiana] under Grant [INV2384].