Comparative diagnostic accuracy of ACE-III and MoCA for detecting mild cognitive impairment

Abstract

Objective

The aim of this study was to validate the reliability of the Chinese version of Addenbrooke’s Cognitive Examination III (ACE-III) for detecting mild cognitive impairment. Furthermore, the present study compares the diagnostic accuracy of ACE-III with that of Montreal Cognitive Assessment (MoCA).

Methods

One hundred and twenty patients with MCI and 136 healthy controls were included in the study. All patients were evaluated by the Chinese version of ACE-III, MoCA and MMSE.

Results

Subjects in the control group showed better performance in ACE-III total score and its subdomain scores than those in the MCI group. There was a significantly positive correlation between ACE-III total score and MoCA score. Meanwhile, there was also a significantly positive correlation between ACE-III total score and MMSE score. For ACE-III total score, a cut-off point of 85 yielded a sensitivity of 97.3% and a specificity of 90.7%. The AUC for ACE-III total score was 0.978. For MoCA, a cut-off point of 23 yielded a sensitivity of 86.5% and a specificity of 97.7%. The AUC for MoCA was 0.961. There were no significant differences in diagnostic accuracy between ACE-III and MoCA.

Conclusion

The present findings support that both ACE-III and MoCA are useful for detecting MCI in early stages.

Keywords:

• Addenbrooke’s Cognitive Examination III
• Montreal Cognitive Assessment
• mild cognitive impairment
• cognitive screening
• Chinese

Introduction

Mild cognitive impairment (MCI), which is considered as a transitional phase between healthy aging and dementia, refers to impairment in cognition above that which is seen in normal aging, but not severe enough to impair daily function.¹ There is an annual progression rate of 5–15% to dementia in patients with MCI than that in the general population.^2,3 On the other hand, studies have found a reversible rate of 30–50% to normal cognitive function in patients with MCI.⁴ The dichotomy between progression rate to dementia and reversible rate to normal suggests that early interventions play a pivotal role in preventing the progress of MCI. Hence, it is important to screen and detect MCI in early stage.

Cognitive assessment by appropriate screening test is central to the diagnosis of MCI. The Mini-Mental State Examination (MMSE) is the most commonly used test in clinical practice. However, MMSE has weaknesses, such as lack of episodic and semantic memory and executive functioning tasks, existence of ceiling and floor effects. Hence, MMSE has limitations in screening multiple cognitive domains and detecting MCI.

The Addenbrooke’s Cognitive Examination III (ACE-III) was reported as a screening tool to detect cognition.⁵ ACE-III involves five cognitive domains: measuring attention/orientation (18 points), memory (26 points), fluency (14 points), language (26 points), and visuospatial function (16 points)–100 in total. Higher scores indicate higher levels of cognitive function. ACE-III has been translated into several languages.^6–8 The Chinese version of ACE-III has been proved to be a reliable assessment tool for dementia.⁹ The ACE-III total score and its subdomain scores make it useful for detecting MCI.¹⁰ Relatively few studies have assessed the utility of ACE-III for diagnosing MCI.^11–14 To the best of our knowledge, there has been no report about the utility of the Chinese version of ACE-III in the diagnosis of MCI.

The Montreal Cognitive Assessment (MoCA) are widely used for screening MCI in China and other countries.¹⁵ We suspected that the ACE-III may be effective to establish a cognitive profile and for the assessment of MCI. The purpose of this study was to compare the diagnostic ability of the Chinese version of ACE-III and MoCA in patients with MCI and controls. Furthermore, we assessed the correlation between ACE-III and MoCA.

Materials and methods

Study design

This is a prospective, single-center, cross-sectional study. The present study was approved by the ethics committee of Geriatric Hospital of Nanjing Medical University. All the participants are governmental staffs. All participants gave their written informed consent to participate in the study. The present study was conducted in accordance with the Declaration of Helsinki.

Study sample

All participants were recruited from the memory disorders clinic and the physical examination center of Geriatric Hospital of Nanjing Medical University during the study period from August 2016 to July 2017. They were divided into two groups: MCI (n=120) and healthy control (n=136). All patients with MCI fulfilled the Pertersen criteria for MCI. Participants in control group were volunteers who fulfilled the following inclusion criteria: (a) no neurological or systemic diseases potentially affecting cognitive function; (b) no current psychiatric disorders; (c) no history of abusing alcohol or other substances; (d) absence of memory or other cognitive complaints; (e) absence of visual, motor, or auditory limitations impairing the administration of the test.

Neuropsychological assessment

In all participants, cognition was evaluated by the Chinese version of ACE-III, MoCA, and MMSE. In addition, the following instruments were used: Clinical Dementia Rating (CDR) and Activities of Daily Living Scale (ADL). CDR was used to determine the severity of cognition impairment. ADL was used to determine activities of daily living. Items overlapping across the different cognitive tests (for example, drawing a clock or specific questions regarding orientation) were only administered once. In order to prevent biases, all the rating scales were performed by two physicians of neurology, who are blinded to grouping. In the two physicians of neurology, one is in charge of MoCA, MMSE, CDR, and ADL. The other is in charge of ACE-III.

We have introduced the translation and adaptation of ACE-III from English to Chinese in previous study.⁹ Addenbrooke’s Cognitive Examination III includes five domains, each reflecting a specific cognitive function and contributing equally to the total score. The total score is 100 points, which is allocated to the five cognitive domains separately as follows: 18 points for attention and orientation, 26 points for memory, 14 points for fluency, 26 points for language, and 16 points for visuospatial abilities. A higher score indicates better cognitive function. Some items of the original ACE-III were adapted for better understanding during administration within the Chinese population. We replaced “lemon” with “orange” in the item of three words for registration and recall. The name and address in anterograde memory, recall, and recognition was replaced by a Chinese name and address. The last question in retrograde memory was replaced by “who is the only female emperor in Chinese history?” We asked subjects to speak as many words as possible with the Chinese character “che” during 1 min instead of the letter “P” in the item for verbal fluency. All the words and sentences were replaced by Chinese phrases in the repetition item. The comprehension part of the domain “language” has four questions. The first one was replaced by “which thing is related to monarchy?” The second question was revised into “which animal lives in Australia?” The third question was revised into “which animal lives in south pole?” The fourth question was revised into “which thing is used for parking a boat?” Finally, we asked the patients to read out eight Chinese characters instead of five English words in the reading item.

Statistical analysis

The statistical analysis was conducted using SPSS Statistics 16.0. The chi-squared test and one-way ANOVA were carried out to determine the statistical significance in demographics, MMSE score, MoCA total score, MoCA delayed recall score, ACE-III total score, and its subdomain scores between the two groups. A two-tailed Pearson’s correlation analysis was performed among multiple neuropsychological tests. Receiver operating characteristic (ROC) curves and areas under the curves (AUC) were estimated to evaluate the discriminating capacity of neuropsychological tests between the MCI and the control group. The method proposed by DeLong et al was used to compare the ROC curves of each test.¹⁶ Youden J index was used to estimate the optimal cutoff points.¹⁷

Results

Clinical and demographic and profile

In total, 256 participants were included in the study. They were divided into two groups: MCI (n=120) and healthy control (n=136). Table 1 displays clinical and demographic characteristics in the MCI group and the control group. As shown in Table 1, there were no significant differences in sex between the MCI group and the MCI group. There were significant differences in age, education level, MMSE score, MoCA total score, MoCA delayed recall score, ACE-III total score, and ACE-III subdomain scores. The average age of patients in the MCI group is about 7.5 years old than that of subjects in the control group. The average education level of patients in the MCI group is about 4.0 years more than that of subjects in the control group. Patients in the MCI group had lower MMSE score, lower MoCA total score, lower MoCA delayed recall score than subjects in the control group. As expected, subjects in the control group showed better performance in ACE-III total score and its subdomain scores than those in the MCI group.

Table 1 Clinical and demographic characteristics (mean ± SD)

	MCI (n=120)	Control (n=136)	p-value
Age, years	76.87±7.87	69.24±11.41	<0.05
Male, %	54.17	54.41	>0.05
Education, years	11.31±3.23	14.29±2.50	<0.05
MMSE score	25.60±1.94	28.58±1.29	<0.05
MoCA total score	20.19±3.06	26.51±2.08	<0.05
MoCA delayed recall score	1.19±1.39	3.22±1.44	<0.05
ACE-III total score	73.48±8.87	91.34±4.66	<0.05
ACE-III orientation/attention score	16.34±1.57	17.54±0.75	<0.05
ACE-III memory score	19.36±4.75	24.35±1.71	<0.05
ACE-III language score	18.47±3.81	23.71±1.95	<0.05
ACE-III verbal fluency score	7.61±2.42	10.61±2.04	<0.05
ACE-III visuospatial score	12.08±3.30	15.29±0.098	<0.05

Abbreviations: MCI, mild cognitive impairment; MMSE, mini-mental state examination; MoCA, Montreal Cognitive Assessment; ACE-III, Addenbrooke’s Cognitive Examination III.

Correlation between ACE-III and MoCA

A two-tailed Pearson’s correlation analysis was performed among ACE-III total, ACE-III subdomain scores, MoCA total score, MoCA delayed recall score, and MMSE score. There was a significantly positive correlation between ACE-III total score and MoCA score (rs=0.794, two-tailed, p<0.05). It suggested that ACE-III total score increased when MoCA total score increased (Table 2). Meanwhile, there was also a significantly positive correlation between ACE-III total score and MMSE score (rs=0.759, two-tailed, p<0.05). It suggested that ACE-III total score increased when MMSE score increased (Table 2).

Table 2 Correlations between tests

	MoCA total score		MoCA delayed recall score		MMSE score
	r	p	r	p	r	p
ACE-III total score	0.794	<0.05	–	–	0.759	<0.05
ACE-III orientation/attention score	0.483	<0.05	–	–	–	–
ACE-III memory score	0.563	<0.05	0.517	<0.05	–	–
ACE-III language score	0.641	<0.05	–	–	–	–
ACE-III verbal fluency score	0.610	<0.05	–	–	–	–
ACE-III visuospatial score	0.625	<0.05	–	–	–	–
MoCA total score	-	-	–	–	0.642	<0.05
MMSE score	0.642	0.000	–	–	–	–

Abbreviations: MMSE, mini-mental state examination; MoCA, Montreal Cognitive Assessment; ACE-III, Addenbrooke’s Cognitive Examination III.

ACE-III and MoCA: discriminant ability between groups

In order to assess the discriminant ability of ACE-III between control group and MCI group, we estimated the ROC curves for ACE-III. Sensitivity and specificity were calculated with a ROC curve. As shown in Table 3, for ACE-III total score, a cut-off point of 85 yielded a sensitivity of 97.3% and a specificity of 90.7%. The AUC for ACE-III total score was 0.978. For MoCA, a cut-off point of 23 yielded a sensitivity of 86.5% and a specificity of 97.7%. The AUC for MoCA was 0.961. For MMSE, a cut-off point of 28 yielded a sensitivity of 78.4% and a specificity of 86.0%.

Table 3 Sensitivity, specificity, optimal cutoff point, and Area Under ROC Curve (AUC) of tests

	Youden index	Sensitivity (%)	Specificity (%)	Optimal cut-off point	AUC
MMSE score	0.655	83.8	81.7	≤28	0.891
MoCA total score	0.853	97.8	87.5	≤23	0.965
MoCA delayed recall score	0.612	77.9	83.3	≤3	0.830
ACE-III total score	0.887	91.2	97.5	≤85	0.980
ACE-III orientation/attention score	0.420	66.2	24.2	≤18	0.752
ACE-III memory score	0.505	77.2	73.1	≤24	0.844
ACE-III language score	0.588	84.6	74.2	≤22	0.876
ACE-III verbal fluency score	0.471	52.9	94.2	≤11	0.824
ACE-III visuospatial score	0.532	82.4	70.8	≤15	0.826

Abbreviations: ACE-III, Addenbrooke’s Cognitive Examination III; MoCA, Montreal Cognitive Assessment; MMSE, mini-mental state examination.

Further, the diagnostic accuracy of ACE-III was compared with that of MoCA and MMSE by Area Under ROC Curve (AUC). As shown in Figure 1, the AUC for ACE-III (0.980) and MoCA (0.965) was superior than that for MMSE (0.891) (ACE-II vs MMSE, p<0.05;MoCA vs MMSE, p<0.05). It suggested that both ACE-III and MoCA had higher diagnostic accuracy in detecting MCI than MMSE. The AUC for ACE-III total score was larger than that for MoCA, but there were no significant differences between the two tests (p>0.05). Furthermore, there were no significant differences in pairwise comparison of ROC curves among ACE-III subdomain scores. There were no statistically significant differences in pairwise comparison of ROC curves between ACE-III memory score and MoCA delayed recall score (p>0.05).

Figure 1 Receiver operating characteristic (ROC) curve of tests.

Discussion

The aim of the present study was to compare the diagnostic ability of the Chinese version of ACE-III and MoCA in patients with MCI and controls. The results showed that the Chinese version of ACE-III had a good correlation with MoCA and excellent discriminant ability for MCI.

In the present study, the patients in the MCI group got higher score than healthy control in MMSE score, MoCA total score, MoCA delayed recall score, ACE-III total score, and its subdomain scores. It suggested that MMSE, MoCA, and ACE-III could differentiate MCI from healthy control.

The present study suggested that the accuracy of ACE-III was equivalent to that of MoCA. Our results showed that the AUC of ACE-III was larger than that of MoCA, but there were no significant differences between the two tests. Another study also demonstrated a high diagnostic accuracy of ACE-III for distinguishing MCI from controls.¹⁸ Matías-Guiu et al detected significant differences between the AUC of ACE-III and MoCA, which suggested that ACE-III was more accurate than MoCA in screening for AD.¹⁹ It may be ascribed to the different participants and sample size. Further study were needed to compare the diagnostic accuracy of ACE-III and MoCA in screening for MCI.

The present study suggested that both ACE-III and MoCA had acceptable sensitivity and specificity in differentiating MCI from control subjects. In other words, both ACE-III and MoCA were effective in screening for MCI, which confirmed the usefulness of ACE-III and MoCA in previous studies.^18,20,21 Compared to ACE-III, MoCA achieved higher sensitivity and lower specificity in the present study. In order to detect MCI in early stage, higher sensitivity was more important. Therefore, MoCA might be more suitable for screening for MCI than ACE-III. However, a recently small-size study suggested that the Portuguese version of ACE-III held sensitivity and specificity values higher than MoCA in all the domains.²² Hence, further studies were needed to compare sensitivity/specificity of MoCA and ACE-III.

In our study, the cut-off point of ACE-III for MCI was 85, which was higher than that (82) in previous study.²² It may be ascribed to the longer education years in the present study. On the other hand, the cut-off point of MoCA for MCI was 23. A meta-analysis in a sample of 9350 suggested that the best cut-off point of MoCA for MCI among people aged over 60 was 24/25, with the sensitivity of 80.48% and specificity of 81.19%.²³ In the present study, we failed to detect significant differences in diagnostic ability among ACE-III subdomain scores. Meanwhile, there were no significant differences between ACE-III memory score and MoCA delayed recall score.

Our results suggested that ACE-III total score and its subdomain scores were significantly positively correlated with MoCA. What is more, the correlation coefficient between ACE-III total score and MoCA total score was higher than that between ACE-III total score and MMSE score. Our results were in accordance with previous report.²¹

There are some limitations of the present study. First, all participants were assessed in a single session. It may produce interference between multiple scales, especially in the memory tasks. Second, the present study was carried out in an official hospital and the participants were all governmental staffs with a relatively high education level. Hence, further studies about Chinese version of ACE-III should be performed in other population with relatively lower education level.

Conclusion

In summary, the present study demonstrates that both ACE-III and MoCA are useful for accurate detection of MCI in elderly individuals. Furthermore, they support using Chinese version of ACE-III as both a screening tool and a means of describing cognitive profiles.

Author contributions

BRW, HFZ, YDZ, and JQS participated in study design and data interpretation. YDZ and JQS played key roles in the development of all components of the manuscript. BRW and JQS drafted the manuscript. HFZ and YDZ revised the manuscript. CX and YS participated in data acquisition, data interpretation, and manuscript drafting. All authors gave final approval of the version to be published. All authors agreed to be accountable for all aspects of the work.

Acknowledgments

The study was funded by National Natural Science Foundation of China (81500916), Youth Medical Talents Program of “Science and Education Strong Health Project” of Jiangsu Province (QNRC2016079), Medical Innovation Team Program of “Science and Education Strong Health Project” of Jiangsu Province (CXTDA2017030) and Cardre’s Health Care Project of Jiangsu Province (BJ18027).

Disclosure

Dr Bian-Rong Wang reports grants from Cardre’s Health Care Project of Jiangsu Province (BJ18027), during the conduct of the study. Dr Ying-Dong Zhang reports grants from Medical Innovation Team Program of “Science and Education Strong Health Project” of Jiangsu Province (CXTDA2017030), during the conduct of the study. Dr Jian-Quan Shi reports grants from National Natural Science Foundation of China (81500916), Youth Medical Talents Program of “Science and Education Strong Health Project” of Jiangsu Province (QNRC2016079) and Medical Innovation Team Program of “Science and Education Strong Health Project” of Jiangsu Province (CXTDA2017030), during the conduct of the study. The authors report no other conflicts of interest in this work.