Normative data for executive function tests in an Ecuadorian Waranka minority population

Abstract

Objective: To generate normative data (ND) for executive functions tests in the Waranka minority population of Ecuador. Method: Four-hundred participants aged 6–17 completed the Symbol-Digit Modalities Test (SDMT), Trail-Making Test (TMT), Modified-Wisconsin Card Sorting Test (M-WCST), and Test of Colors-Words (STROOP). Scores were normed using multiple linear regressions, including age, age², natural logarithm of mean parent education (MPE), sex, bilingualism, and two-way interactions as predictors. Results: Age by MPE and Age² by MPE interactions arose for SDMT, so that children with illiterate parents scored lower than those with literate parents. Girls scored higher in SDMT. All TMT and M-WCST scores were influenced by age². Age by MPE interaction was found for TMT–A, so that children with higher MPE went faster; and age by bilingualism interaction for TMT–B, so that more bilingual children needed less time. Stroop-Word and Color were influenced by age² by MPE interaction, so that children, while older, scored higher, especially those with higher MPE. Also, age² by sex interaction arose, so that girls increased scores curvilinearly while boys linearly. Word-Color was influenced by age, while Stroop-interference by age². Age by MPE interaction was found for MCST-Categories and Perseveration, so that perseverations decreased to then increased, especially in those with illiterate parents. M-WCST-Category scores increased to then decrease later on age in children with illiterate parents. Z-scores calculated through indigenous ND were significantly lower than generated through non-indigenous norms. Conclusions: ND for minority populations are critical since Waranka sample performed worse when using non-indigenous norms for z-score calculation.

Keywords:

• Normative data
• children
• executive functions
• indigenous population
• regression models

Introduction

Executive functions (EF) encompass a set of cognitive functions (e.g. attentional control, working memory, inhibition, or cognitive flexibility) that allow the planning and organization of goal-directed behavior (Lezak et al., 2012; Muir et al., 2023; Stuss, 2011). EF development occurs during childhood and adolescence, parallel to the maturation of the human brain, especially the frontal lobes (Petersen & Posner, 2012; Posner & Petersen, 1990; Ribeiro et al., 2023). Indeed, recent evidence suggests that adequate EF development depends on coordinated changes in large-scale functional organization in the brain systems of ventral attention, dorsal attention, and frontoparietal control (Keller et al., 2022). It also requires an appropriate balance between an integrated connectivity and a segregated connectivity pattern (Lopez et al., 2020; Wang et al., 2020). Integration between areas would allow more efficient neural communication (Baum et al., 2020; Oligschläger et al., 2017), while segregation would enable more specialized processing (Chen et al., 2017; Murphy et al., 2019).

Adequate EF during childhood and adolescence is a hallmark of healthy development, may be important for resilience to psychopathology (Keller et al., 2022), and is essential for adaptive behavior, particularly in novel situations with complex task demands. Deficits in any of the EF domains are found in several disorders such as traumatic brain injury (e.g. Shah et al., 2017), attention deficit hyperactivity disorder (e.g. Kingdon et al., 2016), autism spectrum disorder (e.g. Christoforou et al., 2023), learning difficulties (e.g. Pullen et al., 2023) and conduct disorder (e.g. Gioia et al., 2002). Furthermore, deficits in EF are prominent in almost all mental illnesses, suggesting that poor EF may be a transdiagnostic vulnerability factor for various forms of psychopathology (Shanmugan et al., 2016).

Currently, neuroimaging techniques may fail in detecting brain network deficits crucial for EF (Horowitz-Kraus et al., 2023). In turn, an adequate neuropsychological assessment that includes EF tests might allow clinicians to detect deficits in different domains of EF in childhood and adolescence. Among Latin American neuropsychologists, the most widely used instruments are the Modified Wisconsin Card Sorting Test (M-WCST), Stroop Color and Word Test (SCWT), Trail Making test (TMT), and Digit and Symbol Modalities Test (SDMT) (Oliveras-Rentas et al., 2020). Several studies have been conducted to generate normative data for children and adolescents for these tests in Spanish-speaking countries, including Spain (e.g. del Pino et al., 2016; Peña-Casanova et al., 2009, 2012), Argentina, Bolivia, Chile, Cuba, El Salvador, Guatemala, Honduras, Mexico, Paraguay, Perú and Puerto Rico (e.g. Arango-Lasprilla et al., 2015a, 2015b, 2015c; Rivera et al., 2015).

Neuropsychology in Ecuador is currently growing, with greater availability of services and research efforts (Ramos-Galarza, 2017). In 2021, Rodríguez-Lorenzana et al. provided specific normative data for the aforementioned tests of attention and EF in Ecuadorian adults. It was an important milestone with relevant clinical and research implications for the country because due to the lack of normative data in Ecuador, many neuropsychologists used norms from other countries, which may not hold for Ecuadorians (Mascialino et al., 2022). However, their sample was composed of monolingual people from urban areas in the metropolitan district of Quito, which could lead, as they acknowledge, to the overestimation or underestimation of normative data from other population areas of Ecuador (Rodríguez-Lorenzana et al., 2020). Although Spanish is the official language, approximately 34% of the population is indigenous and speak their own languages (around 18 native languages), with Kichwa/Quichua (86.2%) being the predominant one (Haboud et al., 2016).

Bilingualism influence on EF measures is still on debate. Recently, a systematic review conducted by Giovannoli et al. (2020) concluded that the current evidence is not enough to sustain the existence of a bilingual effect, or even to identify the factors involved in the bilingual effect, mostly due to the wide differences between the studies (e.g. sample size, neuropsychological tests, methodological aspects, EF subdomains measured). Moreover, bilingual effect is difficult to study as bilingual populations included in the studies differ (e.g. language competence, use, exposure) (Leivada et al., 2021), and also because bilingualism has been treated as categorical variable when it is not (Anderson et al., 2020).

There is a particular dearth of standards for underrepresented minority (URM) populations (Mindt et al., 2019). The evidence suggests that the use of neuropsychological tests, even those with good psychometric properties, in these populations entails measurement errors in, for example, diagnosis and interpretation of the results (Antinori et al., 2007). It is well known that a different cultural background will result in the development of different patterns of abilities (Ardila & Moreno, 2001). This is particularly problematic because studies such as Valcour et al. (2006) and Shimizu et al. (2011) found a significant association between minority status and worse neurocognitive outcomes.

Also, most neuropsychological tests have been developed according to Western culture values, so the constructs that are relevant for Western culture may not be for others. Given the impact that culture has on cognitive test performance (Al-Jawahiri & Nielsen, 2021; Li et al., 2022; Mendoza et al., 2022), the generation of normative data for non-Western populations becomes essential (Ardila et al., 2010; Fujii, 2018).

In fact, Fujii (2018) reported that cross-cultural neuropsychological evaluation should not be limited to quantitative factors, but also to cultural factors included in his ECLECTIC model. These factors are, first, level and quality of education, with special attention to illiterate population, as these people may not be familiar with neuropsychological testing that fit more with a Western academic environment (Ardila et al., 2010). Second, acculturation, as the closer a URM culture is to Western culture, the more valid the test will be (Arentoft et al., 2015; Boone et al., 2007; Coffey et al., 2005; Flores et al., 2017). Third, language, considering how many languages the people speak and which is(are) their main language(s). Economic level of the country is other relevant factor, as well as communicative style, that is, whether the people remain silent out of shame or out of respect for the authority that the neuropsychologist constitutes. The last two factors are the testing situation (the familiarity they may have with this type of evaluation) and the culture's conception of intelligence (whether or not the functions evaluated are relevant for them).

There are very few studies that explore neuropsychological performance in URM in Latin America and they are mostly limited to the 2000s. In 2001, Ardila & Moreno administered a set of neuropsychological tests, including the M-WCST, to a sample of 20 indigenous people from Arauco, Colombia. They found that, on some tests, indigenous people's performance was adequate, while in others (e.g. M-WCST) participants were unable to understand the task, and, therefore, it was impossible to administer. Educational level, age, and cultural relevance were related to performance such that the higher the education level, the younger the age, and the more the test was culturally meaningful and relevant, which lead to better understanding of the task, the better participants performed.

Ostrosky-Solís et al. (2004b) analyzed the influence of education and culture on the NEUROPSI test battery in a sample Maya indigenous people compared to non-indigenous people from Mexico. They found that illiterate indigenous people showed better performance in visuoperceptive tasks, but worse performance in verbal memory. Similar results were obtained when exploring the performance of 7 non-indigenous, 14 Maya and 6 indigenous Pame people from Mexico. Illiterate indigenous people showed better performance in visuoperceptive and constructive tasks, but lower scores in verbal memory compared to non-indigenous illiterate people. Moreover, Maya indigenous people with 1 to 4 years of education obtained higher scores in visual detection, visual memory and delay verbal memory compared to Maya indigenous people with no education (Ostrosky-Solís et al., 2004a). In both studies, the results were explained in terms of the demands imposed by the cultural environment or education. In this lens, skills related to verbal memory are not used in the daily life of indigenous people, but visuospatial skills are of great relevance. On the contrary, in the urban context, skills related to verbal memory are used to a greater extent than visuoperceptive skills. Finally, Fierro et al. (2018) studied the performance of 29 indigenous adolescents from Guaranda, Ecuador on the NEUROPSI test battery. They found participants performed below what was expected according to age and level of education in language, attention and reading. In contrast, orientation and writing were within normal ranges.

As can be seen, studies that address the influence of culture on the neuropsychological profile of URM populations have shown a different performance pattern compared to non-indigenous people. Therefore, collecting normative data specific to URM is essential. For these reasons, the purpose of this study was to generate normative data for four tests that evaluate EF in a group of 400 children and adolescents from the Ecuadorian indigenous community of Waranka. The Waranka people are geographically distributed in the Bolívar province of Ecuador. They are located at an altitude of 2500 to 3500 meters above sea level. They represent 40% of the total population of Bolívar, are distributed in 216 communities, and their native language is Northern Quichua (CONAIE, 2014). Waranka children and adolescents attend to intercultural bilingual schools, where teaching is taught in Spanish and Kichwa/Quichua. It integrates an intercultural vision in accordance with the geographical, cultural and linguistic diversity of the country. It promotes the respect, care and preservation of communities and nationalities identities (Constitución de la República del Ecuador [Constitution of The Republic of Ecuador], 2021).

Method

Participants

The sample consisted of 417 healthy indigenous children and adolescents who were recruited from the Waranka indigenous community of Ecuador. Two participants were removed due to incomplete information; therefore, the final sample size was 415. The majority of the sample was female (53.7%), the age average was 11.66 (SD = 3.23), and all attended public intercultural schools. The mean parent education (MPE) was 7.9 (SD = 3.61) years. Households reported an average income of US$299.52, (SD = US$197.74; Median = US$245; range = US$50–US$1700). Participants lived at an altitude between 2980 to 4250 meters above sea level.

Bilingualism was assessed using the Language and Social Background Questionnaire (LSQB) for children (Anderson et al., 2020) that includes questions about languages' proficiency, use and exposure. A composite score is calculated by computing the sum of these domains, ranging from 0 to 15, with higher composite score values corresponded to more balanced bilingualism (Anderson et al., 2020). Language proficiency mean score was 1.66 (SD = 0.81; range = 0.60–3); mean use score was 2.34 (SD = 1.54; range = 0–6), mean exposure score was 2.87(SD = 1.56, range = 0–6), and mean total composite score was 6.88 (SD = 3.45; range = 1–15). More demographic characteristics of the sample are shown in Table 1.

Table 1. Demographic characteristics of the sample (N = 415).

Age	n Total	Sex				MPE
		Girls		Boys
		n	%	n	%	Mean	SD
6	28	13	3.13	15	3.61	8.84	4.11
7	23	9	2.17	14	3.37	9.54	3.99
8	27	10	2.41	17	4.1	8.65	3.79
9	35	11	2.65	24	5.78	9.09	2.77
10	43	25	6.02	18	4.34	8.14	3.73
11	47	20	4.82	27	6.51	8.15	3.25
12	50	20	4.82	30	7.23	8.75	3.85
13	36	18	4.34	18	4.34	7.53	3.60
14	28	16	3.86	12	2.89	6.61	2.96
15	33	18	4.34	15	3.61	6.73	3.11
16	27	12	2.89	15	3.61	6.19	2.74
17	38	20	4.82	18	4.34	6.47	3.75

Note. MPE=Mean Parent Education; SD=Standard Deviation.

To be eligible for study, all participants must have met the following requirements: a) be between 6 to 17 years old, b) be born and live in the Waranka indigenous community, c) have an IQ ≥80 on the Test of Non-verbal Intelligence (TONI-2; Brown et al., 2000), and d) have a score of <19 on the Children's Depression Inventory (CDI; Kovacs, 1992). Participants were excluded if they reported: (a) learning problems, (b) a history of psychoactive substance use (among adolescents), and (c) a score < 5 in the Alcohol Use Disorder Identification Test (AUDIT-C; Bush et al., 1998).

Measures

Symbol digit modalities test (SDMT; Smith, 2002)

The SDMT is an experimental task for the detection of possible brain dysfunctions in children and adults (Ramos-Galarza et al., 2018). It evaluates processing speed, visual-motor coordination, divided memory, and attention. The test takes 90 s to scan an array of symbols and substitute them with a number associated with each symbol (Arango-Lasprilla et al., 2017a). The total score is the number of correct substitutions from 0 to 110.

Trail making test (TMT A-B; Reitan & Wolfson, 1985)

The TMT evaluates attentional processes and visual-motor tracking (Lezak, 1995). The TMT is a brief paper-and-pencil test, consisting of two parts: TMT-A and B. In the first part, the participant must draw a line, as fast as possible, connecting 25 numbers in ascending order. In TMT-B, the participant alternates between numbers and letters (1-A, 2-B, 3-C, etc.), which is significantly more difficult than part A (Drane et al., 2002). The final score is the time, in seconds, to complete the task in each part.

Modified Wisconsin card sorting test (M-WCST; Schretlen, 2010)

The M-WCST is considered a complex planning and execution test that assesses EF (Arango-Lasprilla et al., 2017b). The test is individual, does not require previous training, and does not have a time limit, although it usually lasts between 10 and 15 min. The M-WCST contains 48 attribute cards and 4 stimulus cards to assess perseveration and abstract reasoning. Scores are obtained for the number of correct categories, number of perseverative errors, and total number of errors (Schretlen, 1997).

Test of colors and words (STROOP; Golden, 2007)

The Stroop test measures different aspects of EF, such as cognitive flexibility, information processing speed, selective attention, and cognitive inhibition (Rivera et al., 2017). The test consists of 100 stimuli, divided into three parts. In the first part, the participant must read as many words as possible in 45 s, where the score is the number of words read correctly. In the second part, the number of correctly named color items is scored. Finally, in the third part, the participants must name the color of the ink, inhibiting the reading of the word, so that an interference index is calculated.

Procedure

The participants completed the following tests as a part of a neuropsychological battery: SDMT, TMT, M-WCST, and Stroop. Several methods were used to reduce bias and maintain consistency, such as randomizing the order in which the tests were administered to each participant and using a template in Microsoft Excel to limit biased input information. This study was approved by the Ethics Committee of the Public University of Navarre, considering the declaration of Helsinki.

The research group from Public University of Navarre trained the data collection research group from Ecuador, led by M.J., a neuropsychologist researcher, in the administration and scoring of the test. For recruitment, the research team first made an appointment with community leaders to explain the project. Once community leaders accepted, they called the parents from the community so that the researcher could explain the study and answer questions about the process. The parents/guardians who agreed to participate signed the informed consent and completed a sociodemographic, medical history, and health status questionnaire for their child.

The test was administered individually in the intercultural bilingual schools and lasted approximately between 90 and 120 min. Data collection started in July 2022 and ended in December 2022. All the participants were volunteers and received no financial compensation.

Statistical analyses

Demographic variables' effect on neuropsychological performance

The sample was divided into sub-sample 1 (n = 337) and sub-sample 2 (n = 78) for cross-validation. This is a method that helps to ensure the accuracy of model predictions with new data that have not been used to create the model. Subsample 1 was used to determine the effects of demographic variables on the SDMT, the TMT A-B, the Stroop Color and Word Test (Total Word, Total Color, and Word-Color interference), and the M-WCST (Categories, Perseverations, and Total Errors) using means of multiple linear regression analyses (Rivera et al., 2017, 2019, 2021), where for each test scores were assumed as $y\sim N_n(\mu ,{\sigma }^2{I}_n)$ and $\mu =\breve{X}\beta$. The full regression models included the following predictors: age, age², ln(MPE), sex, bilingualism, and two-way interactions between these variables. To avoid multicollinearity, age and quadratic age were calculated through orthogonal polynomials of second-degree, warranting the independence assumption between covariables (Kennedy & Gentle, 1980). Natural logarithm (ln) of MPE was examined assuming that scores do not increase linearly as a function of MPE; instead each MPE one-point increase supposed a slower neuropsychological score increase each time. Given that illiterate people were included (MPE = 0), and natural logarithm of 0 is minus infinitive, a constant ($k=1$) was included for each MPE value. Sex was coded as girls = 0 and boys= 1, and treated as factor. This equation represents the full regression model: Yi = β0 + βi  ⋅  (age)i + β2  ⋅  (age)i2 + β3  ⋅  [In(MPE)i] + β4  ⋅  (sex)i + β5  ⋅  (bilingualism)i + βk  ⋅  Interactionsi + εi, with the term ${\mathit{\text{Interactions}}}_i$ referring to all two-way interactions between the fixed effects. The model assumes that the residuals ${\epsilon }_i$ are normally distributed with mean 0 and variance ${\sigma }_{\epsilon }^2$, i.e. ${\epsilon }_i\sim N(0,{\sigma }_{\epsilon }^2)$. A square root or natural logarithm transformation was used for the control of the lack of normality in the models.To select the predictors of the final model, we used the best subset selection method. Best subset selection involves trying out all possible combinations of the p predictors ($2^p)$, assessing their goodness of fit using performance metrics, and selecting the model with the best performance as the final model. This study used the performance metrics Bayesian information criterion (BIC) (James et al., 2013). The following assumptions were evaluated for the multiple linear regression models: (a) the absence of multicollinearity by computing the Variance Inflation Factor (VIF) (≤10) and the collinearity tolerance values (≤1), (b) homoscedasticity by applying the Breusch Pagan test, (c) normality of the standardized residuals by using the Kolmogorov Smirnov test, and (d) a lack of influential values by computing maximum Cook's distance. For a linear regression model's prediction to be as accurate and unbiased as possible, it is important that four key assumptions hold and, if not, transformations will be used to make models fit these assumptions.

Normative data procedure

To generate normative data, we followed the four steps described by Rivera et al. (2019). First, using the final regression model, the predictive value of the score was calculated $\left({\widehat{Y}}_i={\widehat{\beta }}_0+{\widehat{\beta }}_1{X}_{1i}+{\widehat{\beta }}_2{X}_{2i}+\dots +{\widehat{\beta }}_K{X}_{\mathit{\text{Ki}}}\right)$. Second, the residual value of the model was obtained $\left(e_i=Y_i-{\widehat{Y}}_i\right)$. Third, the residual value obtained was standardized $\left(z_i=e_i/{\mathit{\text{SD}}}_e\right)$. For the M-WCST Perseverative errors, M-WCST Total errors, and TMT (A and B) scores, it is necessary to multiply the standardized residual value by minus one because a higher score reflects worse performance $\left(z_i={[e}_i/{\mathit{\text{SD}}}_e]·-1\right)$. Finally, the percentile value was obtained by using the standard normal cumulative distribution function (if the assumption of normal residuals in the normative sample was fulfilled) or the empirical cumulative distribution function of the standardized values (if the standardized residuals were not normally distributed in the normative sample).

Comparative z-scores

The Z-scores $\left(z_i\right)$ for subsample 2 were calculated using the final regression model obtained from subsample 1. These $z_i$ were used to perform comparisons between Waranka indigenous people who lived in rural communities in Guaranda versus non-indigenous people from Quito who lived in an urban area (Arango-Lasprilla et al., 2017a, 2017b, 2017c) using the Wilcoxon signed-rank test. R Project for Statistical Computing for Linux (version 4.3.3; R Development Core Team, 2021) was used for all analyses.

Results

Demographic variables' effect on neuropsychological performance

Assumptions

All final models had no multicollinearity (maximum generalized and scaled VIF value was 4.29, which does not exceed the cutoff of 10). The maximum Cook's distance was 0.13, meaning there were no influential values on the models. The Breusch Pagan test suggested homoscedasticity in all models (p' values >.01), except for √SDMT and √Stroop Color. The lack of homoscedasticity in these two models was corrected by estimating the residual standard deviation $\left({\mathit{\text{SD}}}_e\right)$ value in each quartile of the predicted scores in each regression model (see Table 2, last column). The Kolmogorov–Smirnov test suggested normality in all models' scores (p' values >.01) except for the √Stroop—interference, ln(M-WCST Categories), and ln(M-WCST Total errors). As a result, the empirical cumulative distribution function of the standardized residual was used for these scores.

Table 2. Final multiple linear regression models.

Score	Predictor	$\beta$	Std. error	t	Sig.	Adjusted R^2	SDe (residual)
							${\widehat{Y}}_i$	SDe
√SDMT	Intercept	5.188	0.223	23.235	< 0.001	.470	< 4.81	1.140
	Age	7.708	3.563	2.164	0.031
	Age^2	−3.057	3.465	−0.882	0.378		4.81–5.54	0.946
	ln(MPE)	0.102	0.098	1.033	0.302
	Sex	−0.266	0.103	−2.563	0.010		5.54–5.96	0.671
	Age*ln(MPE)	3.554	1.660	2.140	0.033
	Age^2*ln(MPE)	−1.365	1.611	−0.848	0.397		> 5.96	0.895
ln(TMT–A)	Intercept	3.597	0.082	43.553	< 0.001	.185	All values	0.365
	Age	0.087	1.359	0.064	0.949
	Age^2	3.068	0.374	8.205	< 0.001
	ln(MPE)	−0.055	0.038	−1.448	0.149
	ln(MPE)*Age	−0.437	0.638	−0.686	0.493
ln(TMT–B)	Intercept	3.905	0.054	71.449	< 0.001	.168	All values	0.415
	Age	−0.587	1.053	−0.558	0.577
	Age^2	3.090	0.461	6.691	< 0.001
	LSBQ	0.016	0.007	2.264	0.024
	Age*LSBQ	0.145	0.14	1.034	0.301
√Stroop Word	Intercept	7.368	0.302	24.383	0.000	.511	< 7.06	1.640
	Age	19.381	5.545	3.495	0.001
	Age^2	−7.874	5.336	−1.475	0.141		7.08–8.25	1.090
	ln(MPE)	0.282	0.133	2.116	0.035
	Sex	−0.029	0.136	−0.214	0.831		8.26–8.93	0.921
	Age*ln(MPE)	0.892	2.453	0.364	0.716
	Age^2*ln(MPE)	−1.033	2.358	−0.438	0.662		> 8.93	0.947
	Age*Sex	0.184	2.409	0.076	0.939
	Age^2*Sex	7.007	2.406	2.913	0.004
√Stroop Color	Intercept	6.673	0.188	35.514	< 0.001	.424	All values	0.735
	Age	13.599	3.448	3.944	< 0.001
	Age^2	−1.823	3.318	−0.549	0.583
	ln(MPE)	0.092	0.083	1.108	0.268
	Sex	0.014	0.084	0.162	0.871
	Age*ln(MPE)	−0.706	1.525	−0.463	0.643
	Age^2*ln(MPE)	−0.756	1.467	−0.515	0.606
	Age*Sex	−1.506	1.498	−1.005	0.315
	Age^2*Sex	5.636	1.496	3.768	< 0.001
√Stroop Color-Word	Intercept	5.261	0.040	129.973	< 0.001	.193	All values	0.724
	Age	6.396	0.725	8.820	< 0.001
Stroop Interference	Intercept	1.797	0.500	3.594	< 0.001	.165	All values	9.153
	Age	−62.773	9.180	−6.838	< 0.001
	Age^2	42.906	9.180	4.674	< 0.001
ln(M-WCST-Categories)	Intercept	1.532	0.093	16.423	< 0.001	.097	All values	0.412
	Age	−0.927	1.535	−0.604	0.546
	Age^2	−2.127	0.422	−5.036	< 0.001
	ln(MPE)	−0.023	0.043	−0.554	0.580
	ln(MPE)*Age	1.070	0.720	1.485	0.139
ln(M-WCST-Perseveration errors)	Intercept	1.945	0.169	11.489	< 0.001	.039	All values	0.748
	Age	1.858	2.786	0.667	0.505
	Age^2	2.579	0.766	3.366	0.001
	ln(MPE)	−0.052	0.078	−0.675	0.499
	ln(MPE)*Age	−1.504	1.307	−1.150	0.250
ln(M-WCST-Total errors)	Intercept	2.699	0.037	71.206	< 0.001	.041	All values	0.693
	Age	−0.972	0.695	−1.398	0.163
	Age^2	2.660	0.695	3.823	< 0.001

Note. SDMT = Symbol Digit Modalities Test; TMT = Trail Making Test; M-WCST = Modified Wisconsin Cart Sorting Test; MPE = Mean Parent Education; LSQB = Language and Social Background Questionnaire; B = represents unstandardized regression weights; R²= Coefficient of determination.

SDMT

The final multiple linear regression model for √SDMT was significant. Age by ln(MPE), and Age² by ln(MPE) interactions were found, so that, children with illiterate parents scored higher before age 9, but later their performance stayed below those children with literate parents (see Figure 1(a)). Also, sex was related to √SDMT, such that girls achieved higher scores than boys. The predictor model explains 47% ($R^2$ = .470) of the participants' performance (see Table 2).

Figure 1. Predicted mean scores for test-scores. (A) Predicted mean scores for SDMT as a function of Age by ln(MPE) interaction. (B) Predicted mean scores for TMT-A as a function of Age by ln(MPE) interaction. (C) Predicted mean scores for TMT-B as a function of Age by LSBQ interaction. (D) Predicted mean scores for Stroop-Word test as a function of Age by ln(MPE) interaction. (E) Predicted mean scores for Stroop-Word test as a function of Age by Sex interaction. (F) Predicted mean scores for M-WCST Perseveration as a function of Age by ln(MPE) interaction.Note. SDMT = Symbol Digit Modalities Test; TMT = Trail Making Test; M-WCST = Modified Wisconsin Cart Sorting Test; MPE = Mean Parent Education; LSQB = Language and Social Background Questionnaire.

TMT A-B

The final multiple linear regression models for ln(TMT—A) and ln(TMT—B) were significant. ln(TMT—A) was influenced by quadratic age, and age by ln(MPE) interaction, so that, children with higher MPE went faster than those with less MPE, but around age 11–12 these differences increased (see Figure 1(b)). The predictor model explained 18% ($R^2$ = .185) of the participants' performance. ln(TMT—B) was also affected by quadratic age, and age by bilingualism interaction, so that children with higher bilingualism score at LSBQ needed less time in the task compared those with lower LSBQ score (see Figure 1(c)). The predictor model explained 16% ($R^2$ = .168) of the participants' performance (see Table 2).

Stroop test

The final multiple regression models for √Stroop Word, √Color, and √Color-word scores were significant. √Stroop Word and √Color scores were influenced by age² by ln(MPE) interaction, so that children, while getting older, achieved higher scores, especially those with higher MPE (see Figure 1(d)). Also, age² by sex interaction arose, so that girls increased their scores curvilinearly while boys linearly (see Figure 1(e)). √Color-Word scores were influenced by linear age, while Stroop interference was by age². The models explained between 50% and 16% of the participants' performance (see Table 2).

M-WCST

The final multiple regression models for the M-WCST ln(Categories) score, ln(M-WCST Perseveration), and ln(Total errors) scores were significant. All scores were affected by age². Moreover, M-WCST ln(Categories) and ln(M-WCST Perseveration) were affected by age by ln(MPE) interaction, so that perseverations decreased until around aged 9 to then increased, especially in those children with illiterate parents (see Figure 1(f)). Regarding M-WCST ln(Categories), number of completed categories increased until around aged 13 to then decrease, for children with literate parents; while in the case of children with illiterate parents, this decreased started later, around 11 years old. The final models explained between 4% and 9% of the participants' performance (see Table 2).

Calculator of normative data

The four-step normative procedure explained above offers the clinician the ability to determine a z-score and percentile for a participant who has a specific test score. To enhance user-friendliness, the authors created a calculator in Microsoft Excel in which the clinician must include the following information: age, education, raw score and sex. This tool is freely available for all users and can be downloaded at https://neuropsychologylearning.com/datos-normativos-archivos-descargables/.

Comparative z-scores

Wilcoxon Signed-Rank Tests showed significant differences between z-score distributions depending on indigenous or non-indigenous normative data used for their estimation (see Figure 2). Specifically, z-scores calculated through indigenous normative data were significantly lower (p value were between 1.715 × 10⁻¹⁴ and 5.9 × 10⁻³) in comparison to z-scores generated through non-indigenous norms. Moreover, across all test-scores, the estimated z-scores derived from norms established with the indigenous population were consistently centered around zero. This consistency enhances the reliability of the model, aligning with expectations for a clinically healthy sample.

Figure 2. Comparison between z-scores generated through indigenous versus non-indigenous normative data.

Discussion

Due to the roles that EF play in healthy development for children and adolescents, the importance of neuropsychological tests in measuring EF, and the lack of normative data for these tests among URM populations in Latin America, this study generated normative data for four EF tests (the SDMT, TMT A-B, Stroop Color and Word Test, and M-WCST) from a sample of 400 children and adolescents from the Waranka Ecuadorian indigenous community. We found significant differences between the z-score distribution based on normative data for the Waranka sample compared to the z-score distribution based on non-indigenous normative data for all tests, suggesting the importance of collecting normative data for specific Latin American URM groups. The lower z-score distributions are most likely due to cultural differences that lead to lower performance on neuropsychological tests predominately designed for Western culture rather than neuropsychological deficits, which is consistent with previous research (Al-Jawahiri & Nielsen, 2021; Li et al., 2022; Mendoza et al., 2022). We also found that age and/or squared age were consistent predictors of performance on the four neuropsychological tests such that generally performance increased with age, although we did observe a curvilinear relationship between age and performance on several of the tests. Sex and parental educational levels were not consistent predictors of EF test performance across tests as single variables, although there were some significant interactions between age and parental education for several of the tests and sex and age for the Stroop Word and Color tests, suggesting the importance of collecting these variables for normative data estimation.

In terms of age effects, for the SDMT, quadratic age positively impacted performance, although the effect plateaued around 14–15 years. For the TMT, both parts A and B were associated with quadratic age, displaying a curvilinear relationship between age and scores where scores initially decline until around ages 11–13 and then begin increasing. A quadratic age was associated with better Stroop Word, Color and Color-Word scores, while both age and quadratic age were associated with Stroop Interference. Finally, quadratic age was also a significant predictor of performance on the M-WCST, exhibiting a clear curvilinear pattern where scores decreased until around ages 11–13 and then began increasing after that. While this effect of age on EF performance in children and adolescents is common in the broader neuropsychological literature, the only known study connecting age to EF test performance in indigenous children and adolescents is that of Ardila and Moreno (2001), who found that younger children and adolescents in their indigenous sample from Arauco, Colombia performed better than older adolescents and young adults up to age 30 on many neuropsychological tests, including the M-WCST. However, it is important to note that in their study, the relatively older participants had 0 years of education and were focused on agriculture and household activities, so the effect was likely due to Western cultural socialization rather than age. As a result, the current study's findings are the first in the literature finding either increasing EF with increasing age or a curvilinear relationship between age and scores across several neuropsychological tests in a sample of indigenous children and adolescents, suggesting that age should serve as a covariate in normative assessments with this and similar communities.

In contrast to age, standalone variables for sex and parental educational levels generally were not significantly associated with performance on the various EF tests. This finding corresponds with other studies assessing neuropsychological performance among Latin American URM populations, as none of the other similar studies found sex or parental education level to be important predictors of performance (Ardila & Moreno, 2001; Fierro et al., 2018; Ostrosky-Solís et al., 2004a, 2004b). The exception for parental education was the Stroop-Word test, where scores were higher for children whose parents had more education. As found in previous research (Ardila & Moreno, 2001; Ostrosky-Solís et al., 2004b), these children might have performed better because they were more exposed to a home learning environment or because their parents' level of education was correlated with their own, both of which would better prepare them for cognitive tests and the Western academic environment these tests are modeled after. The exception for sex was the SDMT, where girls performed significantly better than boys. The better performance of girls is consistent with findings from Paraguay and Spain (Arango-Lasprilla et al., 2017c; Jinabhai et al., 2004). According to Roivainen's (2011) review regarding gender differences in processing speed, these differences may be due to females having better visual processing speed, as well as females having an advantage in reading and writing. In addition, Roivainen (2011) hypothesized that one of the mechanisms by which gender and education affect processing speed is through reading and writing speed.

Regarding interaction terms, for the SDMT, TMT-A, Stroop-Word, M-WCST Preservations, and M-WCST Total Errors tests, parental education had a significant interaction with age, such that children whose parents had more education generally performed better. As discussed above, the increased performance for children whose parents have more education is likely due to their parents' education level being correlated with them having more educational opportunities or schooling, which would better prepare them for cognitive tests (Ardila et al., 2010; Ardila & Moreno, 2001; Ostrosky-Solís et al., 2004b). For the TMT-B test, we found a significant interaction between age and bilingualism, where children with a higher level of bilingualism completed the task faster. While other studies have both not been able to establish a consistent relationship between bilingualism and neuropsychological test performance nor explain why bilingualism might impact performance on these tests (Anderson et al., 2020; Giovannoli et al., 2020; Leivada et al., 2021), our finding highlights the importance of measuring bilingualism when generating normative data for these assessments, and are in line with previous studies that showed the better performance of bilinguals in some EF subdomains (Bialystok et al., 2012). We also found a quadratic age and sex interaction for the Stroop Word and Color tests in which female performance increased nonlinearly with age while male performance increased linearly. As discussed above, this finding is consistent with literature identifying that girls perform better than boys on other similar cognitive tests (Arango-Lasprilla et al., 2017c; Jinabhai et al., 2004; Roivainen, 2011).

All models met the assumptions for no multicollinearity and no overly influential values based on the diagnostic tests run. The results from the Breusch Pagan test showed all models were homoscedastic except for the SDMT and Stroop-Word. We corrected this heteroskedasticity by estimating the residual standard deviation $\left({\mathit{\text{SD}}}_e\right)$ value for each quartile of the predicted scores in the affected models. Based on the Kolmogorov-Smirnov test, the standardized residual values of all models followed the normal distribution except for the Stroop—Interference, M-WCST Categories and Total errors. To address this non-normality for these two models, we used the empirical cumulative distribution function of the standardized residual when estimating scores rather than the standard normal cumulative distribution function. We also used a square root transformation and ln(Y) for the test scores so that the data distribution from these models would better fit the standard normal distribution.

Clinical and research implications

One key finding from the present study was that the norms for the Waranka indigenous child and adolescent population on three of the four neuropsychological tests in this study were significantly lower than the non-indigenous population. This highlights the importance for future research to collect normative data for neuropsychological tests for other Latin American URM and indigenous groups. In order for clinicians who work with these populations to obtain the most accurate representations of patients' cognitive abilities, they should calculate their scores for neuropsychological tests based on the norms generated specific to that larger demographic group. To this end, the current study's normative data calculator for the Warankan population can help clinicians input basic demographic information about their patient (age, education, raw score and sex) and receive a z-score and percentile for them on any of the four EF tests considered. These norms could be used for Waranka children and adolescents who live in other regions. Likewise, they could be used for Ecuadorian Kichua/Quichua speakers who present similar sociodemographic data (especially those referring to socioeconomic status, educational level and linguistic competence).

This study would not be possible without a respectful community-based approach to working with the minority community through collaborating with Waranka community leaders. In Ecuador, these leaders respond to their own hierarchical organization (Altmann, 2017). They are elected in a general assembly, and their duty is to represent and oversee the wellbeing of their community. Therefore, any activity that is intended to engage the indigenous community must first be approved by these community leaders. Respecting Warankan customs and lifestyle is necessary to promote inclusive and meaningful research. We encourage future research to promote similar community engagement when working with minority communities.

Constraints on generality, limitations and future directions

One limitation of the present study is that it only collected normative data for the Waranka community, so these norms might not apply well to other indigenous Latin American communities. However, if no norms exist, the current norms may be the best available until more accurate norms can be created—in this case, the z-scores and percentages derived for individual patients' performance should be interpreted with extreme caution. Another limitation is that the present study only collected normative data for four neuropsychological tests that assess EF, and there are many other tests that will need to be normed. A final limitation is that the current sample is relatively young with a maximum age of 17, so the normative data collected will not apply to older individuals.

Conclusion

Despite these limitations, the present study underscores the importance of collecting normative data for neuropsychological tests specific to Latin American URM and indigenous groups, as we found the Warankan sample consistently performed worse on three of the four neuropsychological tests considered relative to the non-indigenous population. Given this finding, it is important that researchers collect normative data for other neuropsychological tests and Latin American URM and indigenous populations so that clinicians can better serve these patients by more accurately identifying those at need of additional treatment, supports and rehabilitation. To collect these norms, we recommend neuropsychologists take a similar approach to the present study by building meaningful researcher-community partnerships that foster mutual trust, community engagement and may provide a strong foundation for future clinician-patient relationships. When collecting normative data, it is critical researchers assess the assumptions underlying their regression models and perform transformations to make these assumptions hold when necessary so that the normative data can be as accurate and unbiased as possible. To obtain the best assessment of cognitive ability when working with indigenous patients, clinicians should use norms specific to their demographic group. The normative data calculator we release can aid clinicians in obtaining normalized scores for the Waranka community, and future research should create similar normative data calculators for other Latin American URM populations and neuropsychological tests.

Acknowledgements

Open access funding provided by Universidad Pública de Navarra.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not—for—profit sectors. The author D.A.E. disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by The Spanish Ministry of Science, Innovation and Universities under Grant for the training of university teachers (grant number FPU19/00726).