The effect of illiteracy on performance in screening tools for dementia: A meta-analysis

ABSTRACT

Introduction

Research indicates that many tools designed for screening dementia are affected by literacy level. The objective of this study was to estimate the overall effects of this confounding factor. A meta-analysis was conducted to evaluate differences in performance in dementia screening tools between literate and illiterate individuals.

Method

Electronic databases were searched from 1975 to June 2021 to identify empirical studies examining performance in dementia screening tools in literate and illiterate individuals over 50 years old. Data for effect sizes, participant demographic information, and study information were extracted.

Results

We identified 27 studies methodologically suitable for meta-analysis. Multi-level random-effects modeling demonstrated a significant overall effect, with literate participants scoring significantly higher than illiterate participants (g = −1.2, 95% CI = −1.47, −0.95, p < .001). Moderator analyses indicated significant effects of test type and the presence of cognitive impairment on the extent of the difference in performance between literate and illiterate participants. The difference in performance between groups was smaller in screening tests modified for illiterate individuals (p < .01), and in individuals with cognitive impairment (p < .001).

Conclusions

Our findings substantiate the unsuitability of many dementia screening tools for individuals who are illiterate. The results of this systematic review and meta-analysis emphasize the need for the development and validation of tools that are suitable for individuals of all abilities.

KEYWORDS:

• Dementia
• literacy
• meta-analysis
• neuropsychology

Despite advances in the provision of education in recent decades, a significant proportion of the world’s population still lack basic literacy skills. In 2019, UNESCO reported that approximately 9% (102 million) of young people and 14% (750 million) of adults are illiterate (UIS, 2019). Elderly illiteracy rates are higher still, with 22% (141 million) of adults over 65 unable to read or write (UIS, 2017).

The main reasons for illiteracy can be broadly divided into two categories: reasons pertaining to health or to social circumstances. Health reasons for illiteracy include learning difficulty or disability and neurological conditions. Social reasons include the absence of an education system, social or cultural disapproval of literacy, child labor, and poverty (Ardila et al., 2010).

It should be noted that, although literacy is highly related to schooling, reading and writing skills can be obtained outside of education. It cannot be assumed, therefore, that an individual who did not attend formal education is illiterate (Ardila & Rosselli, 2007). Individuals who are literate in their native language but unable to read or write proficiently in the language of their country of residence may also be regarded as functionally illiterate. In this sense, linguistic and cultural factors contribute to how illiteracy is defined across different contexts (Vagvolgyi et al., 2016).

Dementia and illiteracy

Assessment of dementia in individuals who are illiterate is complex. It has been shown that individuals who are illiterate are significantly more likely to receive a diagnosis of dementia (Nitrini et al., 2009). There are numerous potential explanations for this.

One possible explanation relates to brain development. The acquisition of literacy skills affects the functional and structural development of the brain (Ardila et al., 2010). The cognitive reserve hypothesis proposes that the neural networks in the brains of illiterate individuals may be more susceptible to disruption or may struggle to compensate for cognitive dysfunction (Manly et al., 2003). This theory is debated within the field of cross-cultural neuropsychology, however, as many have argued that it fails to adequately consider potential confounds, such as testing bias (Ostrosky-Solis, 2007).

Traditional tests used to screen for and diagnose dementia were not developed for individuals who are unable to read or write (Ardila & Rosselli, 2007). Screening tests, such as the Mini-Mental State Examination (MMSE; Folstein et al., 1975), assess specific skills that are enhanced by the process of learning to read (Ostrosky-Solis, 2007). For example, learning to read trains remembering strategies, visuospatial perception, logical reasoning, and fine movements. Individuals who are illiterate often possess a different skill set, one that is more procedural, pragmatic and sensory oriented. However, these skills are less likely to be tested in the process of cognitive assessment. Thus, individuals who are illiterate are at a disadvantage when assessed in the cognitive domains that benefit from literacy skills, even when assessment of these domains does not directly involve reading or writing (Kosmidis, 2018).

Furthermore, individuals who are illiterate usually lack familiarity with testing procedures. People who go through an education system are socialized into a value system which places importance on working alone, memorization, doing your best to succeed without any obvious immediate benefit to your daily functioning (Nell, 2000). These values may not be held by individuals who did not go through such a system. Thus, differences in performance in cognitive tests may not be completely reflective of differences in cognitive ability, but rather differences in test-taking abilities and familiarity.

Dementia screening tools

The MMSE is one of the most frequently used screening tools for dementia across the globe, both in research and in clinical practice (Nieuwenhuis-Mark, 2010). However, many studies have demonstrated that the MMSE is affected by educational level, language of administration, and culture (e.g., Black et al., 1999; Goudsmit et al., 2018; Nielsen et al., 2012). Studies have attempted to account for some of these confounds by lowering the cutoff score for some populations (Black et al., 1999; Cassimiro et al., 2017). However, evidence suggests that while modifying the cutoff score in the MMSE may increase specificity, it reduces sensitivity (Ostrosky-Solis, 2007).

In light of this evidence, a number of alternative tools have been developed, such as the Literacy Independent Cognitive Assessment (LICA; Choi et al., 2011), the Community Screening Instrument for Dementia (CSI-D; Hall et al., 2000), the Rowland Universal Dementia Assessment Scale (RUDAS; Storey et al., 2004), and the European Cross-Cultural Neuropsychology Test Battery (Nielsen, Segers, et al., 2018b). These tools were designed for use in multicultural and illiterate/low educated populations. They differ from traditional tools largely by having more of a focus on abilities acquired in everyday life (e.g., experience with shopping for groceries) rather than relying on school-dependent skills and abilities (Nielsen, Segers, et al., 2018b).

Rationale and objective

The number of individuals living with dementia across the globe is estimated to increase from 50 million in 2018 to 152 million in 2050, a 204% increase (WHO, 2019). The reasons for this increase are complex and include factors such as increased life expectancy, improvements in the reporting of health data, and an increase in knowledge about dementia (Alzheimer’s Research UK, 2018).

The prevalence of dementia in low-income countries is currently lower than in high income countries because the life expectancy is lower. As life expectancy increases in low-income countries, so too will the prevalence of dementia. The most drastic increase in the number of people living with dementia is therefore estimated to occur in low income countries (Prince et al., 2015). The lowest literacy rates are also in these countries (UIS, 2019). In addition, increases in migration and globalization across the world has resulted in more diverse populations with wider ranges of native languages and literacy levels in middle- and higher-income countries (Franzen & European Consortium on Cross-Cultural Neuropsychology, 2021; Nielsen et al., 2012). Finding accurate and appropriate methods of diagnosing dementia in individuals who are illiterate is therefore increasingly relevant. In order to inform the development and promotion of such methods, it is important to understand the extent to which the current methods of screening dementia are affected by illiteracy.

This review aims to identify, evaluate, and summarize the findings of all studies exploring how illiterate individuals perform in dementia screening tools compared to literate individuals. We hypothesize that the mean score of illiterate individuals in dementia screening tools will be lower than that of literate individuals.

We also aim to investigate whether any differences found between literate and illiterate individuals are moderated by whether or not the dementia screening tools were designed or modified for individuals with low levels of literacy or education, and by the cognitive status of the participants. To our knowledge, this is the first meta-analysis on this topic.

Methods

Search criteria

The meta-analysis was carried out using PRISMA guidelines (Liberati et al., 2009). The PRISMA checklist is included in Appendix A. The protocol for the review was registered on PROSPERO, with following reference number: CRD42020168,484 (https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=168484).

A comprehensive search was conducted on 24 June 2021using OVID databases (MEDLINE, PsycINFO, and EMBASE), Scopus, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Google Scholar from 1975 (year of publication of the MMSE; Folstein et al., 1975) to June 2021. The search strategy included combinations of the following phrases:

(dement* OR “cognitive impairment” OR “alzheimer*”) AND (screening OR mmse OR “mini mental” OR “moca” or “montreal cognitive assessment” OR “GPCOG” OR “General Practitioner Assessment of Cognition” OR “mini-cog” OR “mini cog” OR “addenbrooke’s cognitive assessment” OR “ace-r” OR “ace-iii”) AND (literate OR illiterate OR literacy OR illiteracy OR “reading abilit*” OR “reading comprehension”).

Eligibility criteria

Studies that met the following criteria were included in the meta-analysis:

1. Quantitative study

2. Measured performance of illiterate and literate adults over 50 years old in a dementia screening tool. Although most studies on dementia focus on participants over 65 years, it is recognized that dementia also affects people under the age of 65 (Van der Flier & Scheltens, 2005). Many research studies reflect this in their age criteria (e.g., Goudsmit et al., 2020; Muangpaisan et al., 2015; Nielsen, Segers, 2018a; Zhou et al., 2006). In order avoid a potential cultural bias toward studies conducted in countries whereby 65 years old is deemed the cutoff for older age, an inclusive age range was chosen.

3. Illiterate participants were illiterate for social reasons (e.g., poverty, lack of education, culture group). Only studies that explicitly stated that participants were illiterate were included; participants with no education were not assumed to be illiterate. Studies that included participants who were illiterate as a result of health conditions (e.g., learning disability, motor/sensory/neurological conditions) were excluded.

4. Published in English in a peer-reviewed journal

5. Reported sufficient information to allow for the calculation of the effect size and standard error of difference in performance between the two groups (literate vs. illiterate adults) in the screening tool. The information required for eligibility included mean values, standard deviations and/or standard errors of the mean for each group. Where sufficient information was not available, authors were contacted to request the required data.

Study selection

The lead author assessed all non-duplicate titles and abstracts for suitability for inclusion using Rayyan software (https://www.rayyan.ai/). Full-text manuscripts were assessed using Microsoft Excel V16.51 and Endnote X9 and reasons for exclusion were recorded.

Data extraction

Relevant data were extracted from the studies by the lead author using Microsoft Excel. The extracted data included the year, study design, screening tool used, language of administration, study country, sample size, age range, gender breakdown, cognitive impairment diagnosis, years of education, primary findings, statistical analysis used, and the mean and standard deviation of performance in the cognitive screening tool in literate and illiterate participants.

Quality assessment

The lead author assessed the quality of all included studies, and a random 50% of the papers were assessed by an independent researcher to ensure reliability (kappa = .83, indicating near perfect agreement). Disagreements were resolved by consensus. Study quality was assessed using the NIH Quality Assessment of Case-Control Studies tool (National Institute of Health, 2014). The NIH Quality Assessment of Case-Control Studies tool does not include specific rules for calculating the quality of the studies. As a general guideline, studies that met 8 or more of the criteria were graded as good, those that met between 6 and 7 of the criteria were graded as fair, and those that met 5 of less of the criteria were graded as poor.

Of the 12 questions in the tool, two questions were excluded, as they were not relevant to the studies in this review. The question regarding concurrent controls was excluded because ‘cases’ in this review referred to illiterate individuals. Illiterate participants did not become illiterate at a specific moment in time, therefore making it impossible to select concurrent control participants. The question regarding confirming exposure prior to the development of the condition that defined a participant as a case was also excluded for the same reason. The condition that defined a participant as a ‘case’ was illiteracy, which did not develop over time.

Studies with a quality rating of ‘poor’ due to unclear selection of participants or ambiguous differentiation of literate and illiterate participants were excluded from the meta-analysis. As the methods used to delineate the number of participants in each group were questionable in these studies, they were excluded from the quantitative analysis to reduce the risk of bias across the studies.

Statistical analyses

Analyses and plots were carried out in RStudio (V1.3.959; RStudio Team, 2020), using R packages ‘meta’ (Balduzzi et al., 2019), ‘metafor’ (Viechtbauer, 2010), and ‘ggplot2’ (Wickham, 2016). The effect sizes of interest were those obtained from comparisons of independent groups (literate vs. illiterate) in terms of cognitive screening test scores. Studies that reported mean scores and standard deviations and/or standard errors of the mean were included in the meta-analysis, and standard errors of the mean were converted to standard deviation prior to calculation of effect size. Effect sizes were calculated as Hedges’ g. Hedges’ g is similar to the more widely used Cohen’s d in that both measures examine the magnitude of mean differences between groups. However, Hedges’ g includes a correction factor for sample size, making it a more suitable measure for comparisons of groups with significantly different sample sizes (Gaeta & Brydges, 2020).

A multi-level random-effects model was used to weight studies and calculate a summary effect size. A random-effects model was chosen to take into consideration the fact that differences in sample size may create variations in effect sizes across studies. A multi-level model was used to account for dependence within the data. Meta-analytic pooling assumes statistical independence. If there is dependency between effect sizes, this may result in false-positive results. Where authors assess more than one screening tool within the same study, the effect sizes calculated for each tool are not independent (Harrer et al., 2019). Thus, these effect sizes cannot be added to a meta-analytic model without accounting for their dependency. A multi-level model takes these dependencies into account by adding another layer into the structure of the meta-analytic model. This allows the model to account for the fact that some effect sizes are nested within one study. In the present analysis, a three-level model was implemented to model sampling variation for each effect size (Level 1), variation within each study (Level 2), and variation between studies (Level 3; Cheung, 2014; Harrer et al., 2019).

Summary effects were calculated using the restricted maximum likelihood (REML) method. Standard errors and 95% confidence intervals were calculated for each effect size. P-values were calculated to test the null hypotheses and Q and I₂ statistics were used to assess heterogeneity and observed variance respectively. Publication bias and outlier biases were analyzed using Failsafe N (Rosenthal, 1978) and a funnel plot and asymmetry was tested statistically using Egger’s regression test (Egger et al., 1997). To maintain independence, average study effect sizes were used in the funnel plot and in Egger’s regression test.

The main analyses examined the overall difference in performance in cognitive screening tests between literate and illiterate groups. A sensitivity analysis was carried out to assess whether the results differed across studies with different primary aims. To assess this, studies were divided into one of three groups depending on the objectives of the study. The first group included studies that focused on examining the difference in performance between literate and illiterate participants as their primary research question. The second group included studies that focused on this comparison as a secondary research question. The third group included studies that provided data on cognitive test scores for literate and illiterate participants but did not explicitly compare groups.

Moderator analyses were carried out to assess whether the type of cognitive screening test or cognitive status affect the results of the main analyses. Cognitive screening tests were categorized by whether or not they were designed or modified for individuals with low levels of literacy or education. To assess whether cognitive status impacted on the results, participants were categorized as healthy, as having mild cognitive impairment, or as having dementia.

Results

Results of systematic search

The results of the systematic search are summarized in Figure 1. The search yielded 2,219 studies. After removing duplicates, the titles and abstracts of the remaining 1,010 studies were screened and 849 records were excluded. A total of 159 manuscripts were retrieved and the full texts were assessed for eligibility. Of these records, 132 were excluded, with reasons outlined in Figure 1. Twenty-seven studies were identified as meeting the inclusion criteria and were therefore included in the meta-analysis.

Figure 1. PRISMA flow diagram of the study selection process.

Study characteristics

The main characteristics of the included studies are summarized in Table 1. The studies collectively comprised of 38,057 participants, with study sample sizes ranging from 34 to 16,488. Of these participants, 13,117 individuals were identified as illiterate, and 24,940 individuals were classified as literate. The gender breakdown of the collective sample was 59% female (n = 22,689) and 40% male (n = 14,607). Four studies, with 761 total participants, did not report the participants’ gender.

Table 1. Characteristics of included studies

Authors (Year)	Location of study (Language)	Sample size (n) (Illiterate/ Literate)	Age (M(SD) unless otherwise specified)	Gender (n)	Education (n, unless otherwise specified)	Cognitive status (n)	Screening tool(s) assessed	Relevant findings
Balduino et al. (2020)	Brazil (Portuguese)	143 (Illiterate: 23 Literate: 120)	85.85 (4.73)	Not reported	Illiterate: 23 1–4y: 77 >4y: 44	All healthy	MMSE CAMCOG CDT	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants in the MMSE, CAMCOG, and CDT
Black et al. (1999)	USA (Spanish/ English)	2,853 (Illiterate: 546 Literate: 2307)	n per age group 65–69y: 1145 70–74y: 765 75–79y: 464 ≥80y: 479	Female: 1662 Male: 1191	0: 506 1–8y: 1886 9–11y: 185 ≥12y: 276	Not reported	MMSE	Illiterate participants scored significantly lower than literate participants.
Brito-Marques & Cabral-Filho (2004)	Brazil (Portuguese)	232 (Illiterate: 28 Literate: 204)	69.4 (6.8)	Not reported	Illiterate: 28 1–4y: 119 5–8: 85	Not reported	MMSE-mo MMSE-ad	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants in both the MMSE-mo and the MMSE-ad
Caramelli et al. (2007)	Brazil (Portuguese)	205 (Illiterate: 52 Literate: 153)	76.5 (6.6)	Female: 141 Male: 64	Illiterate: 52 1–3y: 46 4–7y: 60 ≥8y: 47	Healthy: 117 Dementia: 88	MMSE	No statistical comparison between literate and illiterate participants. Data extracted for meta-analysis only.
Cassimiro et al. (2017)	Brazil (Portuguese)	164 (Illiterate: 60 Literate: 104)	Median (IR) Illiterate: 70 (67–73) 1–2y: 69 (68–72.5) 3–4y: 70 (66.5–73)	All female	Illiterate: 60 1–2y: 52 3–4y: 52	Not reported	MMSE	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants.
Cesar et al. (2017)	Brazil (Portuguese)	623 (Illiterate: 86 Literate: 537)	n per age group 60–69y: 304 70–79y: 215 ≥80: 104	Female: 393 Male: 230	0: 86 1–3y: 180 4–7y: 213 8–11y: 83 ≥12y: 61	Healthy: 385 Cognitive impairment no dementia: 135 Dementia: 110	ACE-R MMSE	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants in both the ACE-R and the MMSE
Contador et al. (2017)	Spain (Spanish)	3,816 (Illiterate: 481 Literate: 3,335)	73.8 (6.6)	Female: 2,148 Male: 1,668	Illiterate: 481 Ability to read/write: 1,614 Primary school: 1,251 Secondary school: 470	Healthy: 3,654 Dementia: 162	MMSE-37	Illiterate participants scored significantly lower than literate participants
Elbedewy and Elokl (2020)	Egypt (Arabic)	143 (Illiterate: 72 Literate: 71)	67.17 (5.41)	Female: 81 Male: 62	Illiterate: 72 Education: 71	Healthy:73 MCI: 70	MMSE CDT	Scores did not vary significantly according to literacy, gender or cognitive status. Illiterate and literate participants were not directly compared independently of gender/cognitive status.
Goudsmit et al. (2020)	The Netherlands (Native language using interpreter. Majority Turkish/ Arabic)	129 (Illiterate: 67 Literate: 62)	Median (IR) Community controls: 68 (63–74) Patients, intact cognition: 76 (70–78) MCI: 77 (71–82) Dementia: 78 (74–81)	Female: 80 Male: 49	No education: 74 Primary: 38 Secondary: 14 Tertiary: 3	Healthy: 47 MCI: 33 Dementia: 49	MMSE RUDAS	Illiterate participants scored significantly lower than literate participants in the MMSE. There was no difference between groups in the RUDAS.
Hamrick et al. (2013)	USA (English)	219 (Illiterate: 74 Literate: 145)	80 (7)	Not reported	<High school: 111 High school: 108	Healthy: 62 Cognitive impairment: 157	M-MMSE	No statistical comparison between literate and illiterate participants. Data extracted for meta-analysis only.
Katzman et al. (1988)	China (Chinese)	5,030 (Illiterate: 1,350 Literate: 3,680)	n per age group 55–64y: 1496 65–74y: 2180 75y+: 1354	Female: 2825 Male: 2205	Illiterate: 1,350 Elementary: 1,853 Middle school: 1,827	Not reported	CMMS	No statistical comparison between literate and illiterate participants. Data extracted for meta-analysis only.
Kim and Chey (2010)	Korea (Korean)	240 (Illiterate: 28 Literate: 212)	69.1 (8.1)	Female: 166 Male: 74	Illiterate: 28 0y: 18 1–6y: 72 ≥7: 122	Healthy: 240 (Dementia: 28 – not included in analysis)	CDT	Scores varied significantly according to literacy status, such that illiterate participants scored lower than literate participants.
Kim et al. (2014)	Korea (Korean)	203 (Illiterate: 29 Literate: 174)	74 (6.9)	Female: 139 Male: 64	Pure illiterate: 29 Semi illiterate: 67 Literate: 75 High-level literate: 32	All healthy	MMSE	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants
Kochhann et al. (2010)	Brazil (Portuguese)	968 (Illiterate: 72 Literate: 896)	70.6 (7.3)	Female: 633 Male: 335	Illiterate: 72 Lower education: 415 Middle education: 277 Higher education: 204	Healthy: 806 Dementia: 162	MMSE	No statistical comparison between literate and illiterate participants. Data extracted for meta-analysis only.
Leite et al. (2017)	Brazil (Portuguese)	180 (Illiterate: 60 Literate: 120)	74.9 (7.1)	Female: 139 Male: 41	Illiterate: 60 1–2y: 60 3–4y: 60	All healthy	MMSE	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants
Lenardt et al. (2009)	Brazil (Portuguese)	33 (Illiterate: 22 Literate: 11)	79.82 (8.23)	All female	Illiterate: 22 3–10y: 9 >10y: 2	Healthy: 24 Cognitive decline: 9	MMSE	No statistical comparison between literate and illiterate participants. Data extracted for meta-analysis only.
Mokri et al. (2012)	Mexico (Spanish)	167 (Illiterate: 101 Literate: 66)	80.85 (7.55)	Not reported for those with MMSE scores	All 0y education	Not reported	MMSE	Illiterate participants scored significantly lower than literate participants
Muangpaisan et al. (2015)	Thailand (Thai)	4,459 (Illiterate: 348 Literate: 4,111)	64.2 (7.9)	Female: 3,195 Male: 1,264	0y: 349 Study as monk: 9 1–4y: 2619 Primary school: 319 Secondary school: 892 Bachelor degree or higher: 271	All healthy	TMSE	Illiterate participants scored significantly lower than literate participants in each domain in the TMSE except the registration domain.
Nielsen, Segers, etal. (2018b)	Denmark (Danish/ Turkish)	41 (Illiterate: 20 Literate: 21)	62.9 (8.0)	Female: 31 Male: 10	Illiterate group: 0y Literate group: M(SD) = 4.4y (2.3y)	Not reported	RUDAS	There was no difference between the scores of literate and illiterate participants.
Nitrini et al. (2004)	Brazil (Portuguese)	51 (Illiterate: 23 Literate: 28)	73.8 (5.4)	Female: 24 Male: 27	Illiterate group: 0y Literate group: M(SD) = 3.8y (3.3y)	All healthy	MMSE CDT	Illiterate participants scored significantly lower than literate participants in both the MMSE and the CDT
Ortega et al. (2021)	Brazil (Portuguese)	117 (Illiterate: 48 Literate: 69)	76.4 (6.9)	Female: 72 Male: 45	Illiterate group: 0y Literate group: mean 2.9y ± 1.1y	Healthy: 69 Dementia: 48	MMSE CDT	No statistical comparison between literate and illiterate participants in the MMSE. Data extracted for meta-analysis only. Illiterate control participants scored significantly lower than literate control participants in the CDT
Park et al. (2014)	Korea (Korean)	80 (Illiterate: 40 Literate: 40)	74.6 (6.6)	Female: 67 Male: 13	Mean 3.94y ± 4.93y	Healthy: 40 Dementia: 40	MMSE	Healthy illiterate participants scored significantly lower than healthy literate participants. Illiterate participants with dementia scored significantly lower than literate participants with dementia.
Shim et al. (2015)	Korea (Korean)	762 (Illiterate: 140 Literate: 622)	72.2 (6.6)	Female: 474 Male: 288	M(SD) =  6.5y (5.13y)	Healthy: 634 Mild cognitive impairment: 128	K-MMSE LICA	No statistical comparison between literate and illiterate participants. Data extracted for meta-analysis only.
Subramanian et al. (2021)	India (Hindi)	240 (Illiterate: 93 Literate: 147)	63.9 (7.1)	Female: 150 Male: 90	Illiterate: 93 Primary: 64 Middle: 47 High school and above: 36	Healthy: 168 Mild cognitive impairment: 40 Moderate cognitive impairment: 31 Severe cognitive impairment: 1	HMSE	The study found a significant relationship between cognitive status based on HMSE score (<26 & >26) and literacy status (illiterate/literate)
Xu et al. (2003)	China (Chinese)	370 (Illiterate: 74 Literate: 444)	70.23 (6.76)	Female: 158 Male: 212	M(SD) =  4.4y (2.8y)	Healthy: 277 Dementia: 93	CAMSE	Healthy illiterate participants scored significantly lower than healthy literate participants. There was no difference between illiterate and literate participants with dementia.
Youn et al. (2011)	Korea (Korean)	100 (Illiterate: 50 Literate: 50)	73.2 (6.0)	All female	Illiterate group: 0y Literate group: M(SD) = 7.5y (2.4y)	Healthy: 50 Dementia: 50	MMSE	Healthy illiterate participants scored significantly lower than healthy literate participants. There was no difference between illiterate healthy participants and literate participants with AD.
Zhou et al. (2006)	China (Chinese)	16,488 (Illiterate: 9,130 Literate: 7,358)	63.4 (7.6)	Female: 9,813 Male: 6,675	Illiterate: 9,130 1–6y: 6,092 >6y: 1,266	Healthy: 16,114 Dementia: 374	mCMMSE	Scores varied significantly according to education level, such that illiterate participants scored lower than literate participants.

Abbreviations:

n: number of participants

M: Mean

SD: Standard Deviation

y: years

ACE-R: Addenbrooke’s Cognitive Examination-Revised

CAMCOG: Cambridge Cognition Examination

CAMSE: Chinese adapted Mini-Mental State Examination

CDT: Clock Drawing Test

CMMS: Chinese Mini-Mental Status

HMSE: Hindi Mini-Mental State Examination

K-MMSE: Korean Mini-Mental State Examination

LICA: Literacy Independent Cognitive Assessment

M-MMSE: Modified Mini-Mental State Examination

mCMMSE: Modified Chinese Mini-Mental State Examination

MMSE-37: 37-point version of Mini-Mental State Examination

MMSE-ad: Mini-Mental State Examination adapted

MMSE-mo: Mini-Mental State Examination modified

MMSE: Mini-Mental State Examination

RUDAS: Rowland Universal Dementia Assessment Scale

TMSE: Thai Mini-Mental State Examination

Twelve studies either focused on healthy older adults only or did not report the cognitive status of the participants. Fifteen studies included both healthy and cognitively impaired older adults. In these 15 studies, 22,519 individuals were identified as healthy, 604 were classified as having MCI or possible dementia, and 1,176 were reported to have dementia or probable dementia. Five of these studies did not report screening scores separately according to diagnosis and consequently were not included in the moderator analysis.

The majority of the studies (25 of 27 studies) included the MMSE or an adapted version of the MMSE as a cognitive screening tool. Seven studies included a cognitive screening tool that was adapted for use in illiterate/low educated participants. The studies were carried out in 11 different countries and included cognitive screening tools administered in 10 different languages. The most common language was Portuguese (10 studies), followed by Korean (5 studies). Only two studies administered screening tools in English and both of these studies were carried out in the USA.

In 12 of the studies, examining the difference in performance between literate and illiterate participants was the primary research question. In nine studies, this was a secondary research question, with six validation studies, four prevalence studies, two studies designed to obtain norms or cutoff scores, and one assessing factors that contribute to cognitive impairment. Six studies collected and reported data for literate and illiterate participants but did not carry out a statistical comparison between the two groups.

Study quality

Quality assessment ratings for all included studies are outlined in Table 2. Seven studies were rated as good, 20 as fair, and 5 as poor. All but two studies (Lin et al., 2002; Subramanian et al., 2021) clearly defined the aims of the study, and all studies clearly defined the study population. Only one study provided a justification for the sample size (Elbedewy & Elokl, 2020). No study explicitly outlined whether the researchers were blinded to the participants’ literacy status. However, it is likely that maintaining blindedness during the process of cognitive assessment would have been difficult. Confounders were adjusted for in the analyses of 17 studies.

Table 2. Quality assessment ratings based on the NIH Quality Assessment of Case-Control Studies tool

	Overall rating	Clearly defined aim	Clearly defined study population	Sample size justified	Cases & controls from similar population	Consistent inclusion/ exclusion criteria	Cases & controls clearly differentiated	Random selection from eligible participants	Measures defined and valid	Blinding	Confounders adjusted
Balduino et al. (2020)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Black et al. (1999)	Fair	Yes	Yes	No	Yes	CD	No	NA	Yes	NR	Yes
Brito-Marques & Cabral-Filho (2004)	Good	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	Yes
Caramelli et al. (2007)	Fair	Yes	Yes	No	Yes	CD	No	NA	Yes	NR	NA
Cassimiro et al. (2017)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Cesar et al. (2017)	Fair	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	NR	No
Contador et al. (2017)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Elbedewy and Elokl (2020)	Good	Yes	Yes	Yes	Yes	Yes	Yes	NA	Yes	NR	Yes
Gambhir et al. (2014)	Poor	Yes	Yes	No	Yes	CD	No	NA	Yes	NR	No
Goudsmit et al. (2020)	Fair	Yes	Yes	No	Yes	Yes	Yes	No	Yes	NR	Yes
Hamrick et al. (2013)	Good	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	Yes
Hong et al. (2011)	Poor	Yes	Yes	No	Yes	Yes	Yes	CD	CD	NR	No
Julayanont et al. (2015)	Poor	Yes	Yes	No	Yes	CD	CD	NA	CD	NR	Yes
Katzman et al. (1988)	Good	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	NR	Yes
Kim et al. (2010)	Good	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	Yes
Kim et al. (2014)	Fair	Yes	Yes	No	Yes	CD	Yes	NA	Yes	NR	Yes
Kochhann et al. (2010)	Fair	Yes	Yes	No	Yes	CD	CD	NA	Yes	NR	Yes
Leite et al. (2017)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Lenardt et al. (2009)	Fair	Yes	Yes	No	Yes	Yes	No	NA	Yes	NR	No
Lin et al. (2002)	Poor	No	Yes	No	No	CD	No	NA	Yes	NR	Yes
Mokri et al. (2012)	Fair	Yes	Yes	No	Yes	CD	Yes	NA	Yes	NR	Yes
Muangpaisan et al. (2015)	Fair	Yes	Yes	No	Yes	CD	No	NA	Yes	NR	Yes
Nielsen (2018)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Nitrini et al. (2004)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Ortega et al. (2021)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Park et al. (2014)	Good	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	Yes
Shim et al. (2015)	Good	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	Yes
Subramanian et al. (2021)	Fair	No	Yes	Yes	Yes	Yes	No	Yes	Yes	NR	Yes
Umakalyani and Senthilkumar (2018)	Poor	Yes	Yes	No	Yes	Yes	No	CD	CD	NR	No
Xu et al. (2003)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Youn et al. (2011)	Fair	Yes	Yes	No	Yes	Yes	Yes	NA	Yes	NR	No
Zhou et al. (2006)	Fair	Yes	Yes	No	Yes	CD	Yes	NA	Yes	NR	Yes

Abbreviations: NA = Not Applicable, NR = Not Reported, CD = Cannot Determine

Scoring: Good: ≥8 Yes/NA Fair: 6–7 Yes/NA Poor: ≤5 Yes/NA

Meta-analysis

Study selection and characteristics

Seven studies did not include sufficient data for inclusion in the meta-analysis. The authors of six of these studies were contacted. The authors of one study (Devraj et al., 2014) could not be contacted because there were no author contact details specified in the paper. Two authors replied and provided the necessary data for the meta-analysis (Cassimiro et al., 2017; Goudsmit et al., 2020). Five studies were given a quality rating of ‘poor’ and were therefore excluded from the meta-analysis (Devraj et al., 2014; Gambhir et al., 2014; Hong et al., 2011; Julayanont et al., 2015; Umakalyani & Senthilkumar, 2018). The final number of studies included in the meta-analysis was 27.

Cognitive screening scores in literate vs. illiterate older adults

Effect sizes were calculated for the differences in performance in cognitive screening tests between the literate groups and illiterate groups. A significant overall effect was found, with literate participants scoring significantly higher than illiterate participants (g = −1.21, 95% CI = [−1.47, −0.95], p < .001; Table 3; Figure 2). However, significant heterogeneity was found (Q = 591.37, p < .001, I² = 98.05%). There were no distinct outliers, so all studies were included. No statistically significant asymmetry was observed from funnel plots (see, Figure 3) using Egger’s test (p = 0.52). A calculation of Failsafe N indicated that 48,822 undetected negative studies would be needed to change the outcome, suggested low publication bias.

Table 3. Results from the multi-level random-effects model

Study	Screening tool	Hedges’ g	Lower CI	Upper CI	Weight (random)
Balduino et al. 2020	MMSE	−1.1847	−1.2829	−1.0864	1.41%
Balduino et al. 2020	CAMCOG	−1.3028	−1.7154	−0.8903	3.43%
Balduino et al. 2020	CDT	−1.6931	−2.1173	−1.269	3.42%
Black et al., 1999	MMSE	−1.2207	−1.5569	−0.8845	1.32%
Brito-Marques & Cabral-Filho 2004	MMSE-mo	−0.9975	−1.3333	−0.6618	1.32%
Brito-Marques & Cabral-Filho 2004	MMSE-ad	−2.705	−2.9778	−2.4322	4.64%
Caramelli et al. 2007	MMSE	−2.4462	−2.7113	−2.181	4.65%
Cassimrio et al. 2007	MMSE	−1.2659	−1.3656	−1.1662	1.41%
Cesar et al. 2017	ACE-R	−0.8739	−1.2397	−0.5082	3.87%
Cesar et al. 2017	MMSE	−0.2844	−0.6355	0.0666	3.89%
Contador et al. 2017	MMSE-37	−0.521	−0.8053	−0.2368	4.49%
Elbedewy & Elokl, 2020	MMSE	−0.7492	−1.0379	−0.4605	4.48%
Elbedewy & Elokl, 2020	CDT	−1.2188	−1.2856	−1.1521	1.41%
Goudsmit et al., 2020	MMSE	−2.2334	−2.6755	−1.7912	3.21%
Goudsmit et al., 2020	RUDAS	−0.3353	−0.5759	−0.0948	1.37%
Hamrick et al., 2013	M-MMSE	−2.0535	−2.4295	−1.6774	1.30%
Hamrick et al., 2013	MMSE	−1.4283	−1.5416	−1.3149	1.41%
Katzman et al. 1988	CMMS	0	−0.6124	0.6124	1.15%
Kim & Chey, 2010	CDT	−2.4205	−3.1497	−1.6912	1.84%
Kim et al. 2014	MMSE	−2.668	−3.4301	−1.9059	1.80%
Kochhann et al. 2010	MMSE	−1.7862	−1.9904	−1.5821	5.26%
Leite et al., 2017	MMSE	−1.0373	−1.2279	−0.8467	5.28%
Lenardt et al. 2009	MMSE	−0.6846	−0.9441	−0.4251	1.36%
Mokri et al. 2012	MMSE	−1.3542	−1.7895	−0.919	1.27%
Muangpaisan et al., 2015	TMSE	−1.3217	−1.3555	−1.2878	1.42%
Nielsen, Segers, et al. 2018b	RUDAS	−1.804	−2.2972	−1.3109	3.62%
Nitrini et al. 2004	CDT	−1.7033	−2.1915	−1.215	3.64%
Nitrini et al. 2004	MMSE	−1.6598	−2.1461	−1.1736	3.65%
Ortega et al. 2021	MMSE	−0.2695	−0.5988	0.0598	4.12%
Ortega et al. 2021	CDT	−0.3095	−0.6393	0.0203	4.12%
Park et al. 2014	MMSE	−2.4656	−2.9266	−2.0046	3.18%
Shim et al., 2015	K-MMSE	−0.9649	−1.727	−0.2028	1.04%
Shim et al., 2015	LICA	−0.779	−1.1004	−0.4577	1.33%
Subramanian et al., 2021	MMSE	−1.0411	−1.4332	−0.649	3.25%
Xu et al., 2003	CAMSE	−1.2448	−1.7651	−0.7245	3.04%
Youn et al. 2011	MMSE	−0.79	−1.2454	−0.3347	1.25%
Zhou et al., 2006	mCMMSE	−0.8971	−1.169	−0.6252	1.35%
Multi-level random effects model		−1.2098	−1.4696	−0.9500	100%

Abbreviations: CI = confidence interval; ACE-R: Addenbrooke’s Cognitive Examination-Revised; CAMSE: Chinese adapted Mini-Mental State Examination; CAMCOG: Cambridge Cognition Examination; CDT: Clock Drawing Test; CMMS: Chinese Mini-Mental Status; K-MMSE: Korean Mini-Mental State Examination; M-MMSE: Modified Mini-Mental State Examination; mCMMSE: Modified Chinese Mini-Mental State Examination; MMSE-37: 37-point version of Mini-Mental State Examination; MMSE-ad: Mini-Mental State Examination adapted; MMSE-mo: Mini-Mental State Examination modified; MMSE: Mini-Mental State Examination; RUDAS: Rowland Universal Dementia Assessment Scale; TMSE: Thai Mini-Mental State Examination

Figure 2. Forest plot of multi-level random effects model for studies comparing performance in dementia screening tests in literate and illiterate individuals. Abbreviations: ACE-R: Addenbrooke’s Cognitive Examination-Revised; CAMCOG: Cambridge Cognitive Examination; CAMSE: Chinese adapted Mini-Mental State Examination; CDT: Clock Drawing Test; CMMS: Chinese Mini-Mental Status; K-MMSE: Korean Mini-Mental State Examination; M-MMSE: Modified Mini-Mental State Examination; mCMMSE: Modified Chinese Mini-Mental State Examination; MMSE-37: 37-point version of Mini-Mental State Examination; MMSE-ad: Mini-Mental State Examination adapted; MMSE-mo: Mini-Mental State Examination modified; MMSE: Mini-Mental State Examination; RUDAS: Rowland Universal Dementia Assessment Scale; TMSE: Thai Mini-Mental State Examination

Figure 3. Funnel plot for studies comparing performance in dementia screening tests in literate and illiterate individuals.

Subgroup analyses

A sensitivity analysis showed no significant effect of study objectives on the difference in performance between literate and illiterate participants (p = 0.88). Literate participants scored significantly higher than illiterate participants in studies that primarily set out to compare these groups, studies that compared the groups as a secondary analysis, and in studies that did not explicitly compare the groups at all (Table 4).

Table 4. Random effect models sub-grouped according to how studies examined differences between literate and illiterate participants

Subgroup	n	k	Hedges’ g, [95% CI], p	Q test, p	I^2
Primary research question	12	15	−1.43, [−1.87, −0.98], p < 0.001	127.63, p < 0.001	96.9%
Secondary research question	9	14	−1.09, [−1.58, −0.59], p < 0.001	324.62, p < 0.001	96.75%
Research question not explicitly asked	6	8	−0.97, [−1.41, −0.54], p < 0.01	113.37, p < 0.001	95.04%

n = number of studies, k = number of effect sizes

A moderator analysis showed a significant effect of test type on the difference in performance between literate and illiterate participants (p = .004). Where screening tests were designed or modified for illiterate individuals or individuals with low education, the difference in performance between the two groups was smaller, demonstrated by a lower Hedges’ g. However, literate participants still obtained significantly higher scores than illiterate participants in adapted screening tests (g = −0.81, 95% CI = [−1.32, −0.29], p = .008; Table 5).

Table 5. Random effect models sub-grouped according to test type

Test design	n	k	Hedges’ g, [95% CI], p	Q test, p	I^2
Designed for general use	24	30	−1.29, [−1.56, −1.03], p < 0.001	482.59, p < 0.001	97.81%
Modified/designed for use in illiterate/low education populations	7	7	−0.81, [−1.32, −0.29], p < 0.01	76.99, p < 0.001	94.89%

n = number of studies, k = number of effect sizes

In order to examine the effect of cognitive impairment, a further analysis was carried out using only the 10 studies that included both participants with and without cognitive impairment or dementia. Similarly, in this analysis, a significant overall effect was found with literate participants scoring higher than illiterate participants (g = −1.58, 95% CI = [−2.1, −1.07], p < .001). Again, significant heterogeneity was found (Q = 293.57, p < .001, I² = 92.9%). A moderator analysis showed a significant effect of cognitive impairment on the difference in performance between literate and illiterate participants (p < .001). The difference in performance between groups was greater in healthy participants than in participants with cognitive impairment and dementia. However, there was still a significant difference between literate and illiterate groups in participants with cognitive impairment/dementia (Table 6).

Table 6. Random effect models sub-grouped according to cognitive status

Cognitive status	n	k	Hedges’ g, [95% CI], p	Q test, p	I^2
Healthy	10	13	−1.57, [−2.19, −0.95], p < 0.001	215.16, p < 0.001	95.34%
Mild cognitive impairment	4	7	−0.87, [−1.69, −0.05], p < 0.05	34.534, p < 0.001	90.81%
Dementia	7	7	−0.9, [−1.44, −0.37], p < 0.01	26.42, p < 0.001	76.38%

n = number of studies, k = number of effect sizes

Discussion

Main findings

The results of this systematic review and meta-analysis indicated that illiterate groups scored significantly lower than literate groups in dementia screening tools. The analyses demonstrated that there was less disparity between literate and illiterate populations in tests that are designed or adapted for use with individuals who are illiterate or have low education levels, including modified versions of the MMSE as well as the RUDAS (Goudsmit et al., 2020; Nielsen, 2018) and LICA (Shim et al., 2015). However, in many of these tests, literate participants continued to outperform illiterate participants. The analysis also indicated that the difference between literate and illiterate participants in performance in dementia screening tools was greater when these participants did not have a cognitive impairment.

Interpretation and analysis

Our findings are in line with literature on the topic of illiteracy and cognitive assessment, which suggests that illiterate individuals are at a significant disadvantage when assessed using formal assessment methods (Ardila et al., 2010). Dementia screening tools assess many cognitive domains that rely on skills that are directly or indirectly related to literacy, such visuospatial function, logical reasoning, and fine motor skills (Kosmidis, 2018). It corresponds that individuals who never learned to read or write tend to perform poorer in these domains than those who did.

The most common dementia screen test studied in this review was the MMSE, with 25 of the 27 studies including either the original version or an adapted version of the MMSE. The persistence and predominance of the MMSE within research, despite widespread acknowledgment of its many limitations (Nieuwenhuis-Mark, 2010), represents a significant obstacle in the pursuit of appropriate methods for screening dementia. This meta-analysis adds further weight to the proposal that the MMSE is unsuitable for use in many populations and future studies should move away from relying on the MMSE as a screen for cognitive impairment.

The results indicate that, when tests are designed or adapted for use in illiterate populations or populations with low education, there is less disparity in performance between literate and illiterate groups. This analysis should be interpreted with a degree of caution, however, as the group of adapted screening tools consisted of a range of different tools. These tools are relatively new compared to the MMSE and require further assessment to determine their validity and utility across different population groups. One tool that stood out as potentially promising was the RUDAS, with both Goudsmit et al. (2020) and Nielsen (2018) demonstrating that literate and illiterate participants performed similarly in this tool. The RUDAS was developed for use in culturally and linguistically diverse populations. It has been validated in 16 countries, in at least 16 languages (Komalasari et al., 2019).

It should be noted that only two studies included in this review administered cognitive assessments in English, both of which were conducted in the USA (Black et al., 1999; Hamrick et al., 2013). There were no studies conducted in the UK. The dearth of studies on this topic in English-speaking, higher-income countries is likely related to the higher literacy rate of these countries (UIS, 2019). However, the demographic landscape of many countries, including the UK, is changing, and becoming increasingly diverse (Franzen & European Consortium on Cross-Cultural Neuropsychology, 2021; Office for National Statistics, 2018). It is therefore important that even countries with high literacy rates have the resources to provide services that are culturally and educationally competent.

The results also demonstrated that, where individuals were cognitively impaired at the point of administration of the cognitive screen, there was less disparity between literate and illiterate groups. It is possible, that as cognition deteriorates, literate and illiterate groups may become more similar in terms of their cognitive functioning. It is plausible for example, that literate individuals with dementia may become more impaired in domains related to literacy (e.g., visuospatial function) as their cognition deteriorates (Kim & Chey, 2010). The purpose of screening tools, however, is to provide an initial indication of cognitive changes. It is therefore important that such tools are sensitive to subtle changes. Disparity between literate and illiterate healthy individuals is therefore problematic, as screening tools are most valuable at the point where it is unclear whether an individual is healthy or is beginning to show some cognitive changes (Xu et al., 2003).

Limitations

There are several limitations to this systematic review and meta-analysis. The quality of the studies included in the review is mixed, with only seven of the included studies rated as “good.” The main methodological issues with the studies included lack of sample size justifications and unclear criteria for differentiating between literate and illiterate participants. We attempted to minimize quality issues by only including studies with a rating of “fair” or “good” in the meta-analysis. The quality assessment ratings may also be subject to individual bias. Steps were taken to minimize bias, such as the addition of a second rater.

There are a number of potential sources of heterogeneity in the meta-analysis, as reflected by high I² values. While heterogeneity is to be expected in a meta-analysis, it is important to explore potential sources of variance (Higgins, 2008).

Firstly, there was heterogeneity across the study objectives and designs. Whereas some studies were designed specifically to answer the question of whether literate and illiterate groups differed in terms of performance in a screening tool, this was not the sole focus of every study. A sensitivity analysis was carried out to explore this potential source of heterogeneity. This analysis indicated that the results were not affected by the discrepancies across the studies in terms of their primary objectives, and high I² values remained when random effects models were carried out by subgroup.

There was also heterogeneity across study populations. Some studies included all healthy participants, whereas others included both healthy participants and participants with MCI and/or dementia. Participants with a wide range of cognitive abilities were therefore included in the main analysis of the meta-analysis, potentially limiting the generalizability of the findings. A moderator analysis was carried out to explore this potential limitation further and indicated that literate and illiterate participants still differed significantly when healthy participants and cognitively impaired participants were analyzed separately.

Finally, there was heterogeneity across the screening tools. For the purpose of this systematic review and meta-analysis, all screening tools were grouped together. Although the MMSE was the most common tool examined across the studies, there existed significant variation in the languages and versions of the MMSE used. The analyses therefore included a variety of different tools administered across a range of different countries and languages. Combining effect sizes for individual studies with different outcome measures may also limit the findings.

Implications and future directions

The results of this systematic review and meta-analysis support the hypothesis that many of the tools used for screening dementia are unsuitable for use in individuals who are illiterate. These findings have significant clinical implications, as they suggest that many of the widely used cognitive screening tools are not fit for purpose for many individuals. Many countries across the world are becoming increasingly multi-cultural and as a result, clinicians in higher income countries, such as the UK, are encountering more individuals from lower income countries with high illiteracy rates (Nielsen et al., 2011). Health services have a duty to provide culturally competent and person-centered care. It is therefore imperative that clinicians are aware of the considerations that should be given when assessing dementia in illiterate individuals. Definitive guidelines around assessing illiterate individuals cannot yet be recommended based on the current research. However, some general recommendations include selecting tools that were designed specifically for use in multicultural and/or illiterate populations and emphasizing information about changes relative to past functioning and the reasons for their illiteracy. This information should be gathered from interview with individuals and, where possible, with informants who have known the individual in question for a significant length of time (Kosmidis, 2018; Nell, 2000).

This review also highlights the substantial heterogeneity in research on dementia screening tools, making it difficult to compare results across studies. This is widely recognized as a problem within the field of dementia research (Costa et al., 2017). Recent calls for consensus in the use of assessment tools for dementia highlight the importance of a harmonized approach (Costa et al., 2017; Logie et al., 2015; Paulino Ramirez Diaz et al., 2005). One important aspect of harmonization involves ensuring that tools selected and developed for widespread use are suitable for use across many different populations. Such tools should be cross-culturally valid and should not be affected by education or literacy level (Costa et al., 2017; Logie et al., 2015). Tools that minimize the effect of education, literacy, and culture have been developed (Choi et al., 2011; Hall et al., 2000; Nielsen et al., 2018; Storey et al., 2004). However, the literature review and data analysis in the present study indicates that more research is required to validate these tools and determine their suitability across a wide range of settings. Such research should focus on determining the sensitivity and specificity of the tools in terms of differentiating between healthy aging, mild cognitive impairment and various types of dementia, the effect of translation, and the impact on confounding variables.

Although the results of this meta-analysis support the hypothesis that many tools are unsuitable for use in individuals who are illiterate, it should be acknowledged that examining mean differences alone does not fully test this hypothesis. To substantiate the hypothesis further, studies investigating whether the relation between test scores and external factors such as level of functioning differs between literate and illiterate individuals. Furthermore, it would be helpful to explore whether the factor structure of cognitive screening tools varies between literate and illiterate groups. For example, using multi-group confirmatory factor analysis would allow researchers to examine whether cognitive tools lack invariance when used with illiterate individuals. Lack of invariance demonstrates that a tool such as a cognitive test triggers systematically different responding in different groups (Brown et al., 2015). This is a method used in cross-cultural research to investigate the cross-cultural validity of certain research tools (Brown et al., 2015; Losada et al., 2012). Unfortunately, none of the studies found in this meta-analysis addressed this question. Further research investigating whether cognitive screening tools lack invariance in illiterate groups would help to further bolster the results of this study.

Conclusions

This meta-analysis collated existing data which supports the view that many dementia screening tools are unsuitable for individuals who are illiterate. This finding emphasizes the need for the development and use of tools that are suitable for all individuals, regardless of their literacy ability, education or cultural background. The development of screening tools that are unaffected by literacy level is complicated by the fact that many cognitive domains implicated in dementia are influenced by literacy skills. Furthermore, individuals who are illiterate are less likely to be familiar with test-taking procedures, which may impact their performance in formal cognitive tests. Despite these confounding factors, tools that minimize the effect of education, literacy, and culture have been developed. However, further research is required to validate such tools and determine their suitability across a wide range of settings. Although further research is still required in order to substantiate the suitability of these tools in some settings, clinicians assessing dementia in individuals with low levels of literacy should consider using such tools where appropriate and should place particular emphasis on information gathering to inform diagnostic decision-making.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Appendix

Appendix A: PRISMA checklist of items to include when reporting a systematic review or meta-analysis

Table

Section/topic	Item No	Checklist item	Reported on page No
Title
Title	1	Identify the report as a systematic review, meta-analysis, or both	1
Abstract
Structured summary	2	Provide a structured summary including, as applicable, background, objectives, data sources, study eligibility criteria, participants, interventions, study appraisal and synthesis methods, results, limitations, conclusions and implications of key findings, systematic review registration number	2
Introduction
Rationale	3	Describe the rationale for the review in the context of what is already known	3–7
Objectives	4	Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS)	6–7
Methods
Protocol and registration	5	Indicate if a review protocol exists, if and where it can be accessed (such as web address), and, if available, provide registration information including registration number	7
Eligibility criteria	6	Specify study characteristics (such as PICOS, length of follow-up) and report characteristics (such as years considered, language, publication status) used as criteria for eligibility, giving rationale	8–9
Information sources	7	Describe all information sources (such as databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched	7–8
Search	8	Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated	8
Study selection	9	State the process for selecting studies (that is, screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis)	9
Data collection process	10	Describe method of data extraction from reports (such as piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators	9
Data items	11	List and define all variables for which data were sought (such as PICOS, funding sources) and any assumptions and simplifications made	9
Risk of bias in individual studies	12	Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis	10
Summary measures	13	State the principal summary measures (such as risk ratio, difference in means).	10
Synthesis of results	14	Describe the methods of handling data and combining results of studies, if done, including measures of consistency (such as I2) for each meta-analysis	10–12
Risk of bias across studies	15	Specify any assessment of risk of bias that may affect the cumulative evidence (such as publication bias, selective reporting within studies)	11
Additional analyses	16	Describe methods of additional analyses (such as sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified	12
Results
Study selection	17	Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram	12
Study characteristics	18	For each study, present characteristics for which data were extracted (such as study size, PICOS, follow-up period) and provide the citations	Table 1
Risk of bias within studies	19	Present data on risk of bias of each study and, if available, any outcome-level assessment (see item 12).	Table 2
Results of individual studies	20	For all outcomes considered (benefits or harms), present for each study (a) simple summary data for each intervention group and (b) effect estimates and confidence intervals, ideally with a forest plot	Table 1, Figure 2
Synthesis of results	21	Present results of each meta-analysis done, including confidence intervals and measures of consistency	14–16
Risk of bias across studies	22	Present results of any assessment of risk of bias across studies (see item 15)	15
Additional analysis	23	Give results of additional analyses, if done (such as sensitivity or subgroup analyses, meta-regression [see item 16])	15–16
Discussion
Summary of evidence	24	Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (such as health care providers, users, and policy makers)	16–19
Limitations	25	Discuss limitations at study and outcome level (such as risk of bias), and at review level (such as incomplete retrieval of identified research, reporting bias)	19–20
Conclusions	26	Provide a general interpretation of the results in the context of other evidence, and implications for future research	20–22
Funding
Funding	27	Describe sources of funding for the systematic review and other support (such as supply of data) and role of funders for the systematic review	23