Clinical value of the Montreal Cognitive Assessment (MoCA) in patients suspected of cognitive impairment in old age psychiatry. Using the MoCA for triaging to a memory clinic

ABSTRACT

Objectives: Diagnostic pathways are limited. A validated instrument that can triage patients when they are suspected of mild dementia (MD) is necessary to optimise referrals.

Method: The MoCA is validated for identifying MD and mild cognitive impairment (MCI) in a cohort of patients suspected of cognitive impairment (CI) after initial assessment in old age psychiatry. The reference standard was the consensus-based diagnoses for MD and MCI, adhering to the international criteria and using suspected patients, but without CI as comparisons (NoCI).

Results: The mean MoCA scores differ significantly between the groups: 24(SE: .59) in NoCI, 21(SE: .31) in MCI and 16,7(SE: .45) in MD (p < .05). The AUC of MD against non-demented (MCI + NoCI) was 0.83(95%CI: 0.78–0.88) resulting in 90% sensitivity, 65% specificity, 50%PPV and 94%NPV at a “best” cutoff of <21 according the Youden index and respectively 0.77(95%CI: 0.69–0.85), 56%, 73%, 90%, 28% for CI (MD + MCI) against NoCI at <21.

Conclusion: 90% of individuals with a MoCA of <21 will have CI (MD + MCI), while 94% with a MoCA of ≥21 will not have dementia. The MoCA can reduce referrals substantially (50%) by selecting who don’t need further work up in a memory clinic, even if they were suspected of CI after initial assessment.

KEYWORDS:

• MoCA
• triaging
• validation
• memory clinic
• old age psychiatry

Introduction

Diagnosing, as well as the guidance and treatment of dementia, including Behavioural and Psychological Symptoms of Dementia (BPSD), is often done in old age psychiatry which, at least in the Netherlands, make up to 25% of all memory clinics (Verhey et al., 2010). Here, patients with a wide variety of etiologies of possible cognitive impairment (CI) are presented—including major depressive, schizophrenic- and bipolar-disorders. More referrals to memory clinics and old age psychiatry should be expected due to demographic reasons and more awareness of CI (Alzheimer’s disease International, 2016) alongside the trend of earlier assessment with less pronounced symptoms (Grimmer et al., 2015). A validated short tool to assess patients that are suspected of CI to objectify the complaints, before further referral, is necessary to triage who is indeed in need of an elaborate diagnostic investigation for dementia. This could help to relieve the pressure on diagnostic pathways (Alzheimer’s disease International, 2016; Davis et al., 2015), which are costly and scarce in most countries (Alzheimer’s Disease International, 2018). Especially as doctors without an objective test rather refer too early than too late to avoid a missed diagnose and this raises the false positive referrals.

According to the Cochrane review, “the MoCA may help identify people requiring specialist assessment and treatment for dementia” (Davis et al., 2015, p. 5). General practitioners in the Netherlands are advised to use the Montreal Cognitive Assessment (MoCA) especially for patients with “possible CI” but less so for “not likely” or “likely” CI patients (Janssen et al., 2017). Screening older patients with the MoCA is often recommended as subjective cognitive complaints agree poorly with objective cognitive deficit (Pendlebury et al., 2015) but results in too many false positives in old age psychiatry (Dautzenberg et al., 2020). Using an objective test (the MoCA) only for suspected patients concurs with the above need for triaging possible impaired patients and is especially welcome in old age psychiatry, as the (subjective) cognitive complaints are numerous due to age (60+), psychiatric comorbidity (including psychotropic medication) causing CI next to CI as a primary reason for referral.

The MoCA is a widely used short screening tool for mild cognitive impairment (MCI) and mild dementia (MD) (Alzheimer’s disease International, 2016; Davis et al., 2013; Nasreddine et al., 2005), validated in multiple settings and languages (Mocatest.Org). However, many of these studies were designed with a case–control set-up using healthy, community-based individuals as controls (Davis et al., 2015), which can result in spectrum-bias (Dautzenberg et al., 2020; Davis et al., 2015; Noel-Storr et al., 2014), overestimating specificity. In literature, lower cutoff scores are repeatedly suggested for clinical use, especially with MD (Carson et al., 2018; Davis et al., 2015; Elkana et al., 2020; Gil et al., 2015; Larner, 2012; Lee et al., 2008; O’Caoimh et al., 2016; Pugh et al., 2018; Rossetti et al., 2011; Waldron-Perrine & Axelrod, 2012).

A test needs to be validated in its corresponding clinical setting (Noel-Storr et al., 2014), as the prevalence of the index disorder and the clinical setting influences results of the validation of tests.

Our aim was to test the criterion validity of the MoCA for MD after initial assessment in old age psychiatry, in order to examine the added value of the MoCA for triaging patients for further specialised work-up. These patients were suspected of cognitive problems on clinical judgment without a cognitive test. To our knowledge, this is the first time the MoCA has been validated for this use in old age psychiatry. Our reference standard consisted of a consensus-based diagnosis adhering to international criteria resulting in patient groups with MCI, MD, and patients suspected of MCI/MD—but ruled out of having cognitive impairment (NoCI) from the same cohort.

Patients and methods

Samples

All newly referred patients for diagnostic purposes from the North-West part of Utrecht (the Netherlands) to our old age psychiatry memory clinic between 2008 and 2018 were eligible for the study if they were capable of giving written informed consent. This clinic offers services to 57,000 inhabitants of 60+ in the North-West side of the city and its rural surroundings and is one out of four memory clinics in the bigger metropolitan area. Therefore, patients with severe dementia (Global Deterioration Scale (GDS) ≥6) (Reisberg et al., 1982) or BPSD as a reason for referral, as well as compulsory referrals, were not eligible (n = 1337). Exclusion criteria included patients with a diagnosis of severe mid-stage dementia (GDS ≥ 5) to prevent inclusion of the extreme of the spectrum—as this could lead to spectrum bias (Noel-Storr et al., 2014)—or other obvious causes of CI, such as; a recent history of substance abuse (<2 years), a delirium (<6 months), or an acquired brain injury including CVA or TIA (n = 174). Only those patients that were referred to our memory clinic after the initial assessment at our old age psychiatric service were included (n = 292) (Figure 1).

Figure 1. Flowchart suspected patients. C.I.: Cognitive Impairment; MCI: Mild Cognitive Impairment; No-CI: No Cognitive Impairment; GDS: Global Deterioration Scale; BPSD: Behavioural and Psychological Symptoms of Dementia.

All of these patients followed a comprehensive cognitive diagnostic route for CI using a consensus based diagnosis following international criteria as a reference standard with a neuropsychological assessment, and when applicable CT/MRI-imaging and Cerebrospinal Fluid (CSF) Analysis (Dautzenberg et al., 2020; Nederlandse Vereniging voor Klinische Geriatrie, 2014). They were classified as MD, MCI or NoCI. We further differentiated these groups by the most likely cause by DSM IV (American Psychiatric Association, 2000) and clustered the neurodegenerative (MCI-N.D.) and psychiatric causes (MCI-Psy) for the MCI-group. We did not differentiate the MCI into non/amnestic uni- or multi-domain.

The comparisons consisted of NoCI patients from this cohort. Therefore avoiding spectrum-bias due to healthy controls and avoiding selection-bias by including naturalistic possible etiologies to comply with the Standards for Reporting of Diagnostic Accuracy dementia (STARD-Dem) (Noel-Storr et al., 2014).

The Committee for Research and Ethics of the institution approved this study (CWO-nr 1606).

All participants gave their informed consent. Data are available on request.

Measurements

Initial assessment

This was completed by an old age psychiatrist and included; a laboratory test (Table 1), medical and functional history from a next of kin and an investigation of Instrumental Activities of Daily Living (IADL) completed by a psychiatric nurse practitioner during a home visit. The 15-item Geriatric Depression Scale (GDS15) (Yesavage & Sheikh, 1986) and the Global Assessment of Functioning (GAF) (American Psychiatric Association, 2000) were also taken during this time. If this resulted in suspicion or doubt of CI, the patients were referred to the memory clinic.

Table 1. Details of the diagnostic tests.

The neuropsychological assessment consisted of the following assessments
Full test	Subtest
Dutch reading test for adults to estimate premorbid intelligence (“Nederlandse Leestest voor Volwassenen” NLV ), proverbs, Zung 12; Self-rating Depression scale (ZDS), Raven Coloured Progressive Matrices Questionnaire for orientation and personal and non-personal episodic memories “Toutenburger Vragenlijst” Visual Association Test (VAT) 15 words imprinting and recall or recognition Copying of Drawings; Meander of Luria, Complex figure of Rey, House, Cube, Greek cross. D-KEFS | Trail Making Test A and B (TMT) Hooper Visual Organization test (VOT-short version) Calculation, spelling and reading Binet- Bobertag story Fluency- test category (and letter) Groninger Intelligence test (GIT) Clock reading and writing	Wechsler Adult intelligence scale; WAIS IV (Symbol substitution, Numerical series/ Digit Span, Agreements/ Similarities, Figures; Figure Weights), Wechsler Memory scale IV; WMS IV (numerical series) Behavioural Assessment of the Dysexecutive Syndrome (BADS; Key search test and Zoo-plan test)
Laboratory tests consisted of:
Full Blood count; Erythrocyte sedimentation rate (ESR); Potassium (K); Sodium (Na); Creatinine (creat); Calcium (Ca); Urea (Ur); Aspartate transaminase (AST); Alanine transaminase (ALT); Gamma-glutamyltransferase (yGT); Alkaline phosphatase (ALP); Glucose non-fasting (Glu); Thyroid Stimulating Hormone (TSH); Albumin; Vitamin B1,B12,D; Folic acid; Albumin (Alb); Total protein; Magnesium (Mg); Syphilis

Diagnostic test

All of the participants were assessed with a MoCA as soon as possible but within 3 months of initial assessment. This was done by a trained psychiatric nurse practitioner at the old age psychiatry clinic independent of the decision to refer to the memory clinic.

The MoCA consists of one page that covers the cognitive domains of executive function and; visuospatial abilities, naming, short term memory, attention and working memory, language, concentration, verbal abstraction and orientation. It can be applied within 10 min and the maximum score is 30 which indicates no errors were made. Correction for low education effects were made, according to the instructions, by adding one point to the total of patients with 12 years of education or less. Suggested cutoff for the diagnosis of dementia was a score of 21 (<21), for MCI < 26. These cutoffs gave the best Youden index for this population (Dautzenberg et al., 2020).

Reference test

The reference test was the diagnosis determined at multidisciplinary meetings, these meetings included an old age psychiatrist, a neuropsychologist and a geriatrician. The diagnosis of MD, MCI or NoCI was supported with (at least) a 4 h neuropsychological assessment. This included multiple tests in the domains of memory, attention, executive function, fluid intelligence and language capacities (Table 1). The diagnoses were made in consensus and in accordance with the DSM IV (American Psychiatric Association, 2000), the MCI criteria as proposed by an international consortium (Gauthier et al., 2006; Winblad et al., 2004), or the Dutch guideline on dementia (Nederlandse Vereniging voor Klinische Geriatrie, 2014). This guideline covers the criteria of -DSM IV for dementia, -NIA-AA / NINCDS-ADRDA for Alzheimers disease (McKhann et al., 2011), -NINDS-AIREN / AHA-ASA for Vascular dementia (Gorelick et al., 2011; Román et al., 1993), -Frontotemporal dementia (FTD) according The Lund and Manchester Groups (Gorno-Tempini et al., 2011; Neary et al., 1994), and the Consensus for Dementia with Lewy Body (DLB) (McKeith et al., 2005). The results of the MoCA were not used to diagnose MCI or Dementia.

Statistical analyses

Statistics

Demographic and clinical variables were compared within patients suspected of MD, MCI or NoCI using Statistical Package for the Social Sciences (SPSS, version 22; SPSS Inc., Chicago, IL); Chi2 test to compare Sex and education. ANOVA to compare age, GAF, GDS15, and MoCA scores followed with a Least Significant Difference (LSD) post Hoc test. Using Receiver Operating Characteristic (ROC), analysis of the Area Under the Curve (AUC) was calculated as a measure for the diagnostic accuracy of the MoCA.

We calculated three different ROC curves, as the MoCA can be used for different tasks: (1) to find dementia (MD versus MCI + NoCI); (2) to rule out Cognitive Impairment (MD + MCI versus NoCI); and; (3) to detect MCI (MCI versus NoCI) (as CI is a multidimensional state, one may also want to identify who is at risk for developing dementia by focusing on MCI).

Positive and negative predictive values (PPV, NPV) were calculated for the “optimal” cutoff scores as calculated by the Youden’s J index. Boxplots were calculated to understand the distribution of the total MoCA scores for the main diagnostic groups and for their underlying DSM IV diagnosis to further explore the origin of the false positive (FP) and false negative (FN) results.

Results

Study groups

Out of 2206 patients referred to the old age psychiatry clinic, 1337 were deemed ineligible for this study as they were not capable of giving informed consent. The exclusion criteria listed above was applied to exclude the extremes of the spectrum (n = 174). Of the remaining 695 patients, 292 were suspected of CI and underwent further assessment at our memory clinic. All were included in calculating the diagnostic accuracy of the MoCA in this setting (Figure 1). This resulted in 83 MD, 153 MCI and 56 NoCI patients. The different underlying disorders are shown in the flowchart (Figure 1). The average time between the initial assessment and the assessment of the MoCA was 21.5 and 60.8 days for diagnosing CI at the memory clinic.

Demographic and clinical findings

The key demographic and clinical characteristics of the study group are displayed in Table 2.

Table 2. Key demographic and clinical characteristics.

	Dementia (a)	MCI (b)	NoCI (c)	Total referred (d)	Statistic difference p < 0.05
Variable / n	83	153	56	292
Age (SD)	77.3 (7.5)	73.9 (8.0)	71.0 (7.2)	74.3 (8.0)	a>b>c
Range	59–94	53–93	58–85	53–94
Education  < 12 (%)	52	53	44	51	No sig.
Sex F (%)	63	57	56	59	No sig.
GAF (SD)	52 (10.2)	57 (12.8)	54 (11.4)	55 (12.0)	a < b
GDS15 (SD)	6.6 (4.9)	7.7 (4.7)	8.9 (4,3)	8.4 (4.5)	No sig.
MoCA (SD)	16.7 (4.1)	20.9 (3.8)	23.9 (4.3)	20.3 (4.7)	a < b < c
Range	5–26	3–28	12–30	3–30

Education and sex were compared between a,b,c, with a Chi2 test.

Groups a,b,c and were compared with ANOVA.

MCI: Mild Cognitive Impairment; NoCI: No ognitive Impairment; GAF: Global Assessment of Functioning; GDS15: Geriatric Depression Scale 15 question version; MoCA: Montreal Cognitive Assessment.

The male-female ratio did not differ significantly between the groups. The significant differences in age were representative of the demographics of an old age psychiatry setting. The GAF score was the highest in the MCI group, as they were the least afflicted. Of the MCI patients 50% (n = 75) had no psychiatric disorder besides the MCI.

As would be expected, the mean MoCA scores differed significantly (p < .05) between the three groups: a mean of 24(SE: .59) in NoCI, 21(SE: .31) in MCI and 16.7(SE: .45) in the MD group (Table 2). The distribution of the MoCA scores for the main diagnostic groups and their DSM IV etiologies (including the prevalence) are presented in Figure 2.

Figure 2. Boxplot median scores. Upper part Main groups: NoCI: No Cognitive Impairment (n = 83: white boxes). MCI: Mild Cognitive impairment (n = 153: striped boxes). MD: Mild Dementia (n = 56:cross boxes). Lower part DSM IV groups: MD-A: Alzheimer’s Dementia (n = 27). MD-V: Vascular Dementia (n = 15). MD-L: Dementia Lewy-body (n = 9). MD-F: Frontotemporal Dementia (n = 4). MD-M; Dementia mixed causes and Not otherwise specified (n = 28). MCI-ND:MCI due to neurodegenerative process (n = 75). MCI-D: MCI and depression (n = 41). MCI-B: MCI and bipolar disorder (n = 14). MCI-S: MCI and schizophrenia (n = 12). MCI-M: MCI remaining or mixed causes including Not otherwise specified (n = 11). NoCI-D: NoCI and depression (n = 23). NoCI-B: NoCI and Bipolar disorder (n = 9). NoCI-S: NoCI and schizophrenia (n = 7). NoCI-P: NoCI and personality disorders (n = 6). NoCI-M: NoCI and remaining or mixed causes (n = 11). Star outlier  =  3.0 × IQR (Interquartile range). Point outlier  =  1.5 × IQR (Interquartile range).

ROC analysis

The ROC curves of the three different comparisons are presented in Figure 3(a–c) and their AUC in Table 3 along with the sensitivity, specificity, PPV and NPV of the MoCA scores of <26 (original cutoff) and <21 (best Youden score for MD). The sensitivity and specificity for the cutoffs from 26 through 18 are presented in Table 4.

Figure 3. (a–c). Results of ROC analysis.

Table 3. Area Under the Curve between variations of groups and their sensitivity, specificity, PPV and NPV at cutoff scores 26 and 21 with the best Youden index (n = 292).

Groups						CutOff  < 26				CutOff  < 21
			AUC	SE	Sens	Spec	PPV	NPV	Sens	Spec	PPV	NPV
Dem	vs	NoDem	.830	.026	98	17	31	95	90	65	50	94
Dem	vs	MCI	.810	.029	98	6	36	83	90	63	56	92
CI	vs	NoCI	.770	.040	95	47	88	68	56	73	90	28
MCI	vs	NoCI	.707	.048	94	47	83	72	37	73	79	30

Dem: Dementia; NoDem: No Dementia (MCI + NoCI); MCI: Mild Cognitive impairment; NoCI: Suspected patients no Cognitive Impairment; CI; Cognitive Impairment (Dem + MCI).

AUC: Area Under the Curve; SE: Standard Error; Sens: sensitivity; Spec: specificity; PPV: positive predictive value; NPV: negative predictive value.

Table 4. Sensitivity and Specificity at MoCA scores from 26 to 18.

Cut-off value^a	Sensitivity	Specificity
		MCI + NoCI No Dementia	MCI	NoCI
Dementia
18	54%	86%	83%	93%
19	62%	84%	82%	89%
20	78%	77%	75%	82%
21	90%	65%	63%	73%
22	91%	55%	50%	67%
23	95%	43%	35%	67%
24	98%	34%	24%	64%
25	98%	24%	12%	55%
26	98%	17%	6%	47%
CI (Dem + MCI)
18	30%			93%
19	33%			89%
20	43%			82%
21	56%			73%
22	65%			67%
23	75%			67%
24	84%			64%
25	91%			55%
26	95%			47%
MCI
18	17%			93%
19	18%			89%
20	25%			82%
21	37%			73%
22	50%			67%
23	65%			67%
24	77%			64%
25	88%			55%
26	94%			47%

^a (MoCA-D below score)

Dem: Dementia; MCI: Mild Cognitive impairment; NoCI: Referred patients no Cognitive Impairment; HC: Healthy Controls; CI; Cognitive Impairment (Dem + MCI).

The cutoff scores with the highest Youden index were <20, <21 for MD, <24 for CI and <25 for MCI.

Only 50% of those with a positive MoCA (score <21) had MD (PPV), but 94% of those with negative tests were correctly identified as not having dementia (NPV) (Table 3). Given the a priori likelihood of MD (28%) in this sample, a NPV of 94% represents a considerable improvement over chance. When using the MoCA for detecting CI (MD + MCI), 90% of the positive tests (<21) correctly identified CI. In clinical practice, a cutoff of <21 resulted in 90% of those with a positive MoCA having CI and 94% of those with a score of ≥21 not having Dementia.

In example assessing 100 patients suspected of MD after initial assessment at a cutoff <21 would result in a 50% reduction of referrals compared to triaging only by initial assessment. The amount of FP would be 25 (of whom were 20 MCI), and 3 FN.

We further explored the distribution of the MoCA scores with a boxplot of the main groups and the MoCA scores by DSM IV diagnosis (Figure 2). Of the demented patients, all of the DLB and mixed causes, and 75% of the vascular and Alzheimer patients scored <21. Three out of five patients with Alzheimer’s that scored ≥21 appeared to have very high education (PhD degree). Of the FTD patients (n = 4) 75% scored ≥21. The median MCI MoCA score was 21. Looking at the aetiology of the MCI group, the neurodegenerative patients were responsible for most of the false positives (FP). More or less 50% of the depressed, bipolar and the schizophrenic patients diagnosed with MCI scored <21.

Discussion

Our aim was to test the criterion validity of the MoCA for MCI and MD in patients suspected of CI and intended to be referred for a comprehensive diagnostic route in an old age psychiatry memory clinic.

We did this because, to our knowledge, no previous study has looked at the criterion validity of the MoCA being used as an add-on i.e., as a (secondary) objective test, after initial assessment in this setting. This is important as it involves a considerable and growing number of patients seen each year and because it is likely that the performance of the MoCA is different across settings. Besides, a lot of the former studies were carried out with healthy controls as comparisons causing spectrum-bias.

As would be expected, the mean MoCA scores differed significantly between patients with MD, MCI and NoCI. However within all three groups, the range was substantial—particularly within the MCI group—making it difficult to differentiate between the three groups using an individual MoCA score as some scores overlap into the other groups. As can be seen in the boxplot (Figure 2(b)), the range has not merely a psychiatric cause as the MCI neurodegenerative group (MCI-ND) have an even wider range.

The mean scores of the MD and MCI groups were comparable to those reported in the literature and demonstrate that our results have external validity (Mocatest.Org). Our control group scores were lower than those in the original and most other validation studies that used healthy controls, but we showed in an earlier study that the use of healthy individuals as controls resulted in a high mean MoCA score, leading to an unrealistically good specificity and PPV (Dautzenberg et al., 2020; Noel-Storr et al., 2014). Our mean MoCA scores were very similar to all patient groups referred to a memory clinic, this included the comparison group (Larner, 2012). Another explanation for the lower scores of our comparison group, and hence a lower specificity, is the psychiatric “comorbidity” which is known to decrease the MoCA score on its own (Blair et al., 2016; Ramírez et al., 2014; Wu et al., 2017).

Our “low” NoCI specificity of 47% concurred with another memory clinic study, where the comparison group consisted of referred subjects with memory loss complaints including psychiatric illnesses (Smith et al., 2007).

Testing the MoCA in our memory clinic setting revealed a good (Fischer et al., 2003) AUC (0.83) when differentiating between demented and non-demented, but with mediocre specificity (65%). This implies that the MoCA could accurately find most demented patients in a group suspected of CI (sensitivity 90%, <21), but a substantial amount of non-demented patients also scored below this cutoff (of whom 79% are MCI), making it unsuitable for diagnostic purposes but good as a screening tool for MD. This is also demonstrated in the poor PPV of 50 at a cutoff <21.

When wishing to use the MoCA to identify those in need of further cognitive work-up (triage), a high NPV is needed to safely exclude patients who do not need further diagnostic work-up.

Given the results of our study, we recommend using the MoCA to exclude MD if someone scores 21 or above. Taking clinical and demographic factors such as FTD or very high levels of education into account (respectively 4.9% and 3.7% of our MD patients), the chance of this patient having MD is very low (NPV > 94%). Although the absolute numbers of these outliers in our study were low, it confirms that MoCA tests results of patients with FTD or high education are prone to be false negative.

The overlapping range of MoCA scores between groups in this study could be explained by individual differences such as FTD or PhD degrees (resulting in higher scores in the MD group), and poor motivation/concentration/attention due to mania or severe depression and schizophrenia (resulting in some lower scores in those with psychiatric illnesses) (Blair et al., 2016; Ramírez et al., 2014; Wu et al., 2017; Yoon et al., 2017). This underscores the importance of taking demographic and clinical factors into account when interpreting the MoCA results and not simply relying on the score, which is further emphasised by the finding that the MoCA score range of the NoCI in this study (12–30) is smaller compared to our previous study (5–30) where the results of the initial assessment were not taken into account (Dautzenberg et al., 2020).

It is reported that half of the patients with mild depression referred with cognitive complaints scored below 26 on the MoCA in a memory clinic (Blair et al., 2016). Another study reported that admitted schizophrenic patients had a mean MoCA score of 22 and 70% scored <26 (Wu et al., 2017). Their MoCA score was independent of their clinical state. A negative correlation between the cognitive part of the PANSS (assessing symptoms of schizophrenia) and the MoCA was found in another study with a mean MoCA of 23 (Ramírez et al., 2014). Our results, as underscored in the boxplot, are in line with these studies and showed the individual effect of psychiatric comorbidity.

If one excludes all psychiatry, as often happens in studies, the higher scores of the comparisons will result in a better specificity, but would no longer represent the clinical reality. Referrals with cognitive complaints during, or possibly due to, psychiatric illnesses is the clinical reality and need to be differentiated. As neurodegenerative causes could still be a comorbidity or even the cause of this psychiatric illness considering their age. Excluding these patients could lead to a delayed diagnosis as (especially) depression or psychosis can be seen during early stage dementia. To find the optimal cutoff value we used the objective Youden J index, although the object and the setting can result in a different “best” cutoff score. For differentiating between MD and no-dementia, cutoffs of <21 and <20 result in the same Youden score—however, the <21 cutoff has a sensitivity of 90% compared to 78% at <20, favouring the former when used as a screener. When identifying MCI, we favour a cutoff of <26, with a sensitivity of 94%, compared to a cutoff of <25 with a sensitivity of 88% despite the latter having a better Youden index by 2%.

Our study showed that the MoCA was excellent at confirming normal cognition amongst patients suspected of CI and thereby very helpful in triaging, i.e., the decision if they indeed need to be referred to a memory clinic. Depending on the accessibility of further diagnostic workup, one can vary the cutoff score and thereby change the amount of FP and FN. Being aware of the patient’s high education level or FTD-symptoms would even lower the FN as shown in this study.

A strength of our study was that the cohort consisted of patients were the clinician wanted further diagnostics. Not merely the patients (lack of) subjective complaints were decisive, nor psychiatric comorbidity for in- or exclusion. This cohort design comes with a limitation; all MoCA scores were included independent of the compliance during the MoCA assessment. Clinical judgment could also be used to lower the FP, especially those lacking motivation during the assessment. Again, one should be cautious of not missing MD with depressed or psychotic symptoms. Even if one could rule out all psychiatric causes of MCI before referral, our findings showed that 50% of the MCI-ND scored below 21. Despite their low MoCA scores, these patients still clinically didn’t have dementia, as they were mostly IADL independent (GDS score of 3).

Because by Dutch law only a psychiatrist can initiate compulsory referrals and our old age psychiatry led memory clinic offers also non-pharmacological home therapies this results in more advanced dementia referrals (sever dementia, BPSD and compulsory referrals), including from other memory clinics, to our clinic. Hence the fast numbers of excluded patients with a clear diagnosis of sever dementia. This could be an explanation why, after applying the exclusion criteria of this study, the prevalence of Alzheimer’s dropped from 61% at referral to old age psychiatry to 33% (23/83) in the study population. This could be a possible limitation of our study as we did not include all patients and that this (may have) influenced our findings, as we deliberately excluded all obvious and known causes and severe CI, e.g., BPSD and severe dementia (GDS ≥ 5). However, this may also be considered a strength of this validation of the MoCA where only patients suspected of CI—excluding the extremes of the spectrum as STARDdem dictates—were included. We believe that this is closer to the clinical reality as a triage tool has no added value for patients with obvious clinical symptoms of severe dementia. They don’t need triaging but need further work-up in case aetiology has still to be identified. This also counts for the excluded patients with delirium, substance abuse or brain injury. Even though this comes with a risk of having omitted cases of vascular and/or mixed dementia.

If one considers only the NoCI as the absolutely unwanted referrals to a memory clinic and the MCI not, as they have a higher risk of developing dementia, the specificity raises to 73% and the PPV to 90% (<21). However, the degree of being unwanted depends on the availability of resources, especially in mid and low income countries where most demented live and up to 90% are not diagnosed (Alzheimer’s Disease International, 2018).

It is still being debated whether the benefits of screening (e.g., early detection allows the improvement of clinical care and management of dementia,) (Baune & Renger, 2014; Pendlebury et al., 2015) outweigh potential harms (e.g., false positive referrals with emotional and financial burden) (Borson et al., 2013; Brunet et al., 2013; Burn et al., 2018; Le Couteur et al., 2013; Lin et al., 2013). The MoCA also comes with its cost; training and assessing-time. Still there are more and more advocacy groups or policy makers that recommend screening, especially for higher risk populations (Alzheimer’s Disease International, 2018; Borson et al., 2013; Cordell et al., 2013; Janssen et al., 2017; Pendlebury et al., 2015). As our patients were believed to be at high risk, and their quality of life seems not to be altered by the assessment (Janssen et al., 2019; McCarten et al., 2011), the use of a short triaging test prior to referral to our memory clinic seems beneficial and may add to a better use of limited resources (Janssen et al., 2019; McCarten et al., 2011). One might question if our setting is comparable to other (non-old age psychiatry) memory clinic settings, as our prevalence of MCI was high due to psychiatric diseases causing cognitive complaints. But we showed that by leaving out all psychiatric causes of MCI, the median stayed 21. A lower prevalence of MCI would result in better PPV, without changing the sensitivity.

Given the above limitations, our overall conclusion is that the MoCA is not suitable for differentiating dementia, but that it is a good tool for screening for MD and MCI even in the old age psychiatry setting and has added value for triaging who is not in need of a specialised diagnostic route. This applies especially in settings where memory clinics are scarce and efforts have to be made to reduce the absolute number of referrals for full diagnostic work-up, without missing those patients in need of further assessment. 90% of those with a MoCA score of <21 will have CI (MD and MCI), while 94% of those with a MoCA of ≥21 will not have dementia.

Acknowledgments

The authors thank Kasper Mens for his help to retrieve the digital data. The research assistants; Andrea Hagg-Koelewijn, Ilse Geurts, Michelle Baars, Veerle van Meijl, Lara Verbeek, Sarah Rooijackers, Mariam Shanurkeyl, Wanisha Latchmansing, Cagla Celik, for their help collecting and digitalising the test results. Jamie Nolan for his help editing the manuscript. We thank the patients and their families for being able to present this data.

Disclosure statement

No potential conflict of interest was reported by the author(s).